Control method and device for head-up display device in vehicle, vehicle and storage medium

Abstract

The invention discloses a control method and device for a head-up display device in a vehicle, the vehicle and a readable storage medium. The method comprises the steps that current eyeball state information of eyeballs of a driver driving the vehicle is collected; determining current focusing state information of eyeballs based on the current eyeball state information; determining projection parameters of the head-up display device based on the current focusing state information; and controlling the head-up display device based on the projection parameters. According to the method, the projection effect of the head-up display device in the vehicle is improved.

Description

Technical Field

[0001] The present application relates to the field of vehicle technology, and more specifically, to a control method, device, vehicle, and storage medium for a head-up display device in a vehicle. Background Art

[0002] A head-up display (HUD), also known as a head-up display (HUD), projects key vehicle information into the driver's field of view. However, the prior art suffers from poor projection quality. Summary of the Invention

[0003] The present application proposes a control method, device, vehicle, and storage medium for a head-up display device in a vehicle, so as to improve the control efficiency of the head-up display device in the vehicle.

[0004] In a first aspect, an embodiment of the present application provides a method for controlling a head-up display device in a vehicle, the method comprising: Collecting current eye state information of the driver's eyes; Determining current focus state information of the eyeball based on current eyeball state information; determining projection parameters of the head-up display device based on the current focus state information; The head-up display device is controlled based on the projection parameters.

[0005] In a second aspect, an embodiment of the present application further provides a control device for a head-up display device in a vehicle, the device comprising: A collection module, used for collecting current eye state information of the driver's eyes; A first determining module is used to determine the current focus state information of the eyeball based on the current eyeball state information; a second determining module, configured to determine projection parameters of the head-up display device based on the current focus state information; The control module is used to control the head-up display device based on the projection parameters.

[0006] In a third aspect, an embodiment of the present application further provides a vehicle, comprising: one or more processors; a memory; a head-up display device; and one or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the one or more processors, and the one or more programs are configured to execute the above method.

[0007] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, which stores a program code executable by a processor, and when the program code is executed by the processor, the processor executes the above method.

[0008] The present application provides a control method, device, vehicle and computer-readable storage medium for a head-up display device in a vehicle. In the present application, current focus state information indicating the current focus state of the driver's eyes is determined based on the current eye state information of the driver's eyes, and then projection parameters of the head-up display device are determined based on the current focus state information of the driver, and the head-up display device is controlled based on the projection parameters of the head-up display device. As a result, the purpose of adjusting the projection parameters of the head-up display device in real time in combination with the driver's eye state is achieved, so that the projection parameters of the head-up display device are adjusted to match the driver's eye state, and then when the head-up display device is projected according to the projection parameters of the head-up display device, the image projected by the head-up display device is easier for the driver to see clearly, thereby improving the utilization rate of the driver's attention and greatly improving the projection effect and driving safety.

[0009] Other features and advantages of the embodiments of the present application will be described in the following description and, in part, will become apparent from the description or be understood by practicing the embodiments of the present application. The objectives and other advantages of the embodiments of the present application can be achieved and obtained through the structures particularly pointed out in the written description, claims, and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0011] Figure 1 A schematic diagram of a vehicle hardware environment suitable for an embodiment of the present application is shown.

[0012] Figure 2 A flow chart of a method for controlling a head-up display device in a vehicle proposed in one embodiment of the present application is shown.

[0013] Figure 3 A schematic diagram illustrating a switching process of a control function for implementing a method for controlling a head-up display device in a vehicle in an embodiment of the present application is shown.

[0014] Figure 4 A schematic diagram showing a process for determining driver eye information in an embodiment of the present application is shown.

[0015] Figure 5 A schematic diagram of a process for determining pupil radius change information and corneal area change information in an embodiment of the present application is shown.

[0016] Figure 6A schematic diagram showing a process for determining behavioral tendency information in an embodiment of the present application is shown.

[0017] Figure 7 A schematic diagram showing a process for determining projection parameters in an embodiment of the present application is shown.

[0018] Figure 8 A schematic diagram of a vehicle function determination process in an embodiment of the present application is shown.

[0019] Figure 9 A structural block diagram of a control device for a head-up display device in a vehicle proposed in one embodiment of the present application is shown. DETAILED DESCRIPTION

[0020] In order to enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application for which protection is claimed, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work fall within the scope of protection of the present application.

[0021] It should be noted that similar reference numerals and letters represent similar items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further defined or explained in subsequent drawings. At the same time, in the description of this application, the terms "first", "second", etc. are only used to distinguish the description and should not be understood as indicating or implying relative importance.

[0022] Reference Figure 1 , Figure 1 A schematic diagram of a vehicle hardware environment applicable to an embodiment of the present application is shown. The vehicle 100 includes a head-up display device 110, one or more (only one is shown in the figure) processors 112 and a memory 113.

[0023] The head-up display device 110 is used to project key vehicle information onto the driver's front visible area (for example, onto the windshield) while the vehicle is driving.

[0024] The processor 112 may be a microcontroller unit (MCU) having a built-in memory 113 . The memory 113 stores a program that can execute the contents of the following embodiments, and the processor 112 can execute the program stored in the memory 113 .

[0025] The processor 112 may include one or more processors. The processor 112 utilizes various interfaces and circuits to connect various components within the vehicle 100 and execute various functions and process data by running or executing instructions, programs, code sets, or instruction sets stored in the memory 113 and accessing data stored in the memory 113.

[0026] The memory 113 may include random access memory (RAM) or read-only memory (ROM). The memory 15 may be used to store instructions, programs, code, code sets, or instruction sets. The memory 15 may include a program storage area and a data storage area. The program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playback function, an image playback function, etc.), and instructions for implementing the following method embodiments.

[0027] Of course, the vehicle 100 may also include an on-board collection device, which collects eye state information of the driver's eyes, environmental information around the vehicle, and in-vehicle environmental information, etc., so as to control the head-up display device based on the eye state information, and determine the information that the head-up display device needs to display based on the environmental information around the vehicle and in-vehicle environmental information.

[0028] See also Figure 2 , Figure 2 A flow chart of a method for controlling a head-up display device in a vehicle, proposed in one embodiment of the present application, is shown. The method is applied to a vehicle including a head-up display device. The method includes: S110: Collect current eye state information of the driver driving the vehicle.

[0029] Generally speaking, when a driver is driving a vehicle, the vehicle can project specific content on the vehicle's windshield through its own head-up display device, so that the driver can view the specific content displayed on the windshield. The specific content can be, for example, text, images, etc.

[0030] Of course, the driver's eye state information belongs to the driver's personal information. Therefore, before collecting the driver's eye state information, the driver's consent must be obtained. For example, the vehicle is pre-installed with a control function that implements the control method of the head-up display device in the vehicle in this application. When the driver turns on the control function, it prompts that the control function needs to obtain the driver's eye state information and continues to ask the driver whether to agree to turn it on. If the driver agrees to turn it on, it means that the driver agrees to collect the driver's eye state information and turn off the control function; if the driver does not agree to turn it on, it means that the driver agrees to collect the driver's eye state information and turn on the control function to implement the control method of the head-up display device in the vehicle in this application.

[0031] Of course, it is not difficult to understand that for a vehicle, there may be multiple drivers driving the vehicle. For each driver, when the driver drives the vehicle for the first time, the driver can be prompted whether to turn on the control function, and the driver can be prompted that the control function requires obtaining the eye status information of the driver's eyeballs.

[0032] The vehicle may include an OMS (Occupant Monitoring System), which may include at least one pre-installed camera to collect eye state information of the driver through the OMS. During the driver's driving of the vehicle, the collected eye state information of the eye at each current moment is the current eye state information.

[0033] In this embodiment, the eye state information may include pupil radius and corneal area. Thus, the current eye state information includes the pupil radius and corneal area of the driver at the current moment.

[0034] S120: Determine the current focus state information of the eye based on the current eye state information.

[0035] The current focus state information refers to the focus state information of the driver's eyes at the current moment. The focus state information may include one-way focus of the eyes, one-way defocus of the eyes, focus of the eyes from the surrounding directions to the center, defocus of the eyes from the surrounding directions to the center, defocus of the eyes from the center to the surrounding directions, focus of the eyes from the center to the surrounding directions, etc.

[0036] In some embodiments, the current focus state information of the driver's eyes at the current moment can be determined based on the driver's current eye state information at the current moment. For example, for each focus state information, corresponding to an eye state information range, the focus state information corresponding to the eye state information range within which the current eye state information is located is determined as the current focus state information.

[0037] In yet other embodiments, a change in focus state information may be determined based on the driver's current eye state information at the current moment, and then the driver's current focus state information at the current moment may be determined based on the change in focus state information. For example, for each piece of focus state information, a range of eye state information changes may be associated with the focus state information, and the focus state information corresponding to the range of eye state information changes within which the focus state information changes may be determined as the current focus state information.

[0038] In some embodiments, S120 may further include: obtaining historical eye state information at a moment before the current eye state; and determining current focus state information based on a difference between the historical eye state information and the current eye state information.

[0039] For each current moment t, the eye state information at the moment t-1 before the current moment is obtained as the historical eye state information, and then the difference between the current eye state information at the current moment t and the historical eye state information at the moment t-1 before is compared to determine the current focus state information of the driver's eyes at the current moment t.

[0040] As mentioned above, the eye state information may include pupil radius and corneal area. Correspondingly, the above-mentioned determination of the current focus state information based on the difference between the historical eye state information and the current eye state information includes: determining the pupil radius change information of the eye based on the difference between the pupil radius in the current eye state information and the pupil radius in the historical eye state information; determining the corneal area change information of the eye based on the difference between the corneal area in the current eye state information and the corneal area in the historical eye state information; and determining the current focus state information based on the pupil radius change information and the corneal area change information.

[0041] The pupil radius change information of the eyeball can be obtained by obtaining the difference between the pupil radius in the current eyeball status information and the pupil radius in the historical eyeball status information, or the pupil radius change information of the eyeball can be obtained by obtaining the difference between the pupil radius in the current eyeball status information and the pupil radius in the historical eyeball status information, and calculating half, one third or one quarter of the difference, etc. as the pupil radius change information of the eyeball; similarly, the difference between the corneal area in the current eyeball status information and the corneal area in the historical eyeball status information can be directly obtained as the corneal area change information of the eyeball, or the corneal area in the current eyeball status information and the corneal area in the historical eyeball status information can be obtained, and calculating half, one third or one quarter of the difference, etc. as the corneal area change information of the eyeball.

[0042] Afterwards, the current focusing state information of the driver's eyeball at the current moment can be determined based on the pupil radius change information and the corneal area change information.

[0043] For example, pupil radius change information may be represented as ΔR, and corneal area change information may be represented as ΔS. Focus state information may include unidirectional eye focus, unidirectional eye defocus, eye focus from surrounding directions to the center, eye defocus from surrounding directions to the center, eye defocus from the center to surrounding directions, and eye focus from the center to surrounding directions. In this case, based on the pupil radius change information and the corneal area change information, determining the current focus state information of the driver's eye at the current moment may include at least one of the following: △R<0 and △S=0, it is determined that the current focus state is unidirectional eye focus; △R>0 and △S=0, it is determined that the current focus state is unidirectional defocus of the eye; △R<0 and △S>0, it is determined that the current focus state is that the eyeball is focused from the surrounding directions to the center; △R>0 and △S>0, it is determined that the current focus state is that the eyeball is defocused from the center to the surrounding directions; △R>0 and △S<0, it is determined that the current focus state is that the eyeball is defocused from the center to the surrounding directions; △R<0 and △S<0, it is determined that the current focusing state is that the eyeball focuses from the center to the surrounding directions.

[0044] S130: Determine projection parameters of the head-up display device based on the current focus state information.

[0045] Each piece of focus state information may correspond to a projection parameter adjustment value. Based on the current projection parameter adjustment value corresponding to the current focus state information, the projection parameter at the current time t may be adjusted to obtain the projection parameter at the next time t+1, which serves as the projection parameter determined in S130. The projection parameter at the next time t+1 may be obtained by summing the projection parameter at the current time t and the current projection parameter adjustment value.

[0046] In this embodiment, the projection parameters of the head-up display device may include projection brightness and projection height, wherein the projection height refers to the height of the content projected by the head-up display device on the windshield.

[0047] In some implementations, S130 may further include: A1. Performing behavior analysis based on the current focus state information to obtain the driver's current behavior information; A2. Analyze the behavior tendency based on the current behavior information to obtain the driver's current behavior tendency information; A3. Determine projection parameters of the head-up display device based on the current behavior tendency information.

[0048] The behavior information corresponding to the focus state information may be determined based on requirements and test results, and then the behavior information corresponding to the current focus state information may be obtained as the current behavior information of the driver at the current moment.

[0049] For example, when the focus state information includes eye unidirectional focus, eye unidirectional defocus, eye focus from surrounding directions to the center, eye defocus from surrounding directions to the center, eye defocus from the center to surrounding directions, and eye focus from the center to surrounding directions, the correspondence between the focus state information and the driver's behavior information may include: The behavioral information corresponding to the unidirectional eye focus is: the driver's attention shifts from the road conditions directly in front of the car to the information displayed inside the car; The behavioral information corresponding to the unidirectional defocus of the eyeball is: the driver's focus shifts from the information display inside the car to the road conditions directly in front of the car; The corresponding behavioral information of the eyeball focusing from the surrounding directions to the center is: the driver's attention shifts from other places outside the car to the information display in front of the car (such as the instrument panel, etc.); The behavior information corresponding to the defocus of the eyes from the surrounding directions to the center is that the driver's attention shifts from other places in the car to the road conditions in front of the car; The behavior information corresponding to the defocus of the eyeball from the center to the surrounding directions is: the driver's attention shifts from the information displayed inside the car to other road conditions outside the car except for the front; When the eyes focus from the center to the surrounding directions, the corresponding behavioral information is: the driver's attention point shifts from the front outside the car to other places inside the car except the front.

[0050] Of course, in some embodiments, the driver's behavior can also be analyzed based on the current focus state information using a behavior analysis model to obtain the driver's current behavior information. The behavior analysis model can be a long short-term memory neural network, a deep neural network, an encoder, a large language model, etc.

[0051] Similarly, based on demand and test results, etc., the behavior tendency information corresponding to the driver's behavior information can be determined, and then the behavior tendency information corresponding to the current behavior information can be obtained as the current behavior tendency information of the driver at the current moment.

[0052] For example, when the driver's behavior information includes six types of behavior information corresponding to the six types of focus state information, the behavior tendency information corresponding to each type of behavior information is as follows: When the driver's focus shifts from the road conditions directly ahead to the information displayed inside the vehicle, the corresponding behavioral tendency information is: the driver is currently in a focused state, the distance between vehicles is small, and he may be nervous and anxious; When the driver's focus shifts from the in-car information display to the road conditions directly ahead, the corresponding behavioral tendency information is: the driver is currently in a relaxed state, the distance between cars is large, and there may be a tendency to fatigue driving; When the behavior information indicates that the driver's focus shifts from other places outside the vehicle to the information displayed directly in front of the vehicle, the corresponding behavior tendency information is: the driver is currently in the information monitoring state; When the driver's attention shifts from other places in the car to the road conditions directly in front of the car, the corresponding behavior tendency information is: the driver is currently in a state of recovering attention; The behavior information is that when the driver's focus shifts from the information displayed inside the car to other road conditions outside the car except for the road ahead, the corresponding behavior tendency information is: the driver may have a tendency to change lanes; When the behavior information indicates that the driver's focus shifts from the front of the vehicle to other places inside the vehicle except the front, the corresponding behavior tendency information indicates that the driver may be distracted or have a tendency to be absent-minded.

[0053] Of course, in some embodiments, a behavioral tendency analysis model can also be used to analyze the driver's behavioral tendency based on the current behavioral information to obtain the driver's current behavioral tendency information. The behavioral tendency analysis model can be a long short-term memory neural network, a deep neural network, an encoder, a large language model, etc.

[0054] After obtaining the current behavior tendency information of the driver at the current moment, the projection parameters of the head-up display device at the next moment can be determined based on the current behavior tendency information.

[0055] In some embodiments, A3 may include: determining the projection parameters of the head-up display device based on current behavioral tendency information, the rate of change of the projection parameters of the head-up display device, the current projection parameters of the head-up display device at the current moment, and the current projection parameter change value of the head-up display device at the current moment.

[0056] Among them, the rate of change of the projection parameters of the head-up display device is used to indicate the change pattern of the projection parameters of the head-up display device. In some embodiments, the parameter difference between the maximum projection parameter and the minimum projection parameter of the head-up display device can be calculated, and the ratio of the parameter difference to the preset value is calculated as the rate of change of the projection parameters of the head-up display device, wherein the preset value can be, for example, 10 or 100.

[0057] In yet other embodiments, obtaining the rate of change of projection parameters of the head-up display device may include: obtaining first eye state information and second eye state information of an eye; the first eye state information being eye state information when the eye is gazing at the vehicle's windshield; the second eye state information being eye state information when the eye is gazing at a target location; and determining the rate of change of the projection parameters of the head-up display device based on the first eye state information, the second eye state information, and the maximum and minimum projection parameters of the head-up display device. The target location is different from the location of the windshield; the target location may be a location directly in front of the driver at a specified distance from the driver, such as 300 meters or 350 meters.

[0058] Generally speaking, when a driver drives a vehicle for the first time, the vehicle can output prompt information to prompt the driver to look at the vehicle's windshield or look at a target location, thereby collecting the driver's first eye state information and second eye state information.

[0059] In which, when the eye state information includes the pupil radius, the difference between the pupil radius in the second eye state information and the pupil radius in the first eye state information can be calculated as the pupil radius difference, the difference between the maximum projection parameter and the minimum projection parameter can be calculated as the parameter difference, and then the ratio of the parameter difference to the pupil radius difference can be calculated as the rate of change of the projection parameter.

[0060] For example, when the projection parameter is projection brightness, the pupil radius in the second eye state information can be calculated The pupil radius in the first eye state information The difference ( ), as the pupil radius difference, calculate the minimum projection brightness With minimum projection brightness The difference ( ), as the parameter difference, and then calculate the ratio of the parameter difference to the pupil radius difference , as the projection brightness change rate .

[0061] Similarly, for example, when the projection parameter is the projection height, the pupil radius in the second eye state information can be calculated The pupil radius in the first eye state information The difference ( ), as the pupil radius difference, calculate the minimum projection height Minimum projection height The difference ( ), as the parameter difference, and then calculate the ratio of the parameter difference to the pupil radius difference , as the projection height change rate .

[0062] As mentioned above, the maximum projection parameter and the minimum projection parameter are usually the factory values of the head-up display device and will not change, while the pupil radius in the second eye state information The pupil radius in the first eye state information It will vary with different drivers. Therefore, for the same vehicle, the rate of change of the projection parameters determined for each driver may also be different.

[0063] The current projection parameter change value of the head-up display device at the current moment refers to the projection parameter change value of the head-up display device at the current moment, which can be the difference between the projection parameter of the head-up display device at the current moment t and the projection parameter at the previous moment t-1. For example, if the projection parameter is projection brightness, the front projection brightness change value ,in, is the projection brightness of the head-up display device at the current time t. Similarly, the projection parameter is the projection height, and the front projection height change value is ,in, is the projection height of the head-up display device at the current time t.

[0064] In this embodiment, the product of the current projection parameter change value of the head-up display device at the current moment and the rate of change of the projection parameter of the head-up display device can be determined as the projection parameter adjustment value, and a calculation method is selected based on the current behavioral tendency information. Then, using the selected calculation method, the projection parameter of the head-up display device (that is, the projection parameter at the next moment t+1) is calculated based on the projection parameter adjustment value and the current projection parameter of the head-up display device at the current moment.

[0065] For example, when the current behavior tendency information is "the driver is currently in a focused state, the distance between vehicles is small, and he may be nervous and anxious" or "the driver is currently in an information monitoring state", the selected calculation method is to sum the projection parameter adjustment value and the current projection parameter of the head-up display device at the current moment; when the current behavior tendency information is "the driver is currently in a relaxed state, the distance between vehicles is large, and he may be fatigued driving tendency", "the driver is currently in a state of recovering concentration" or "the driver may have a tendency to change lanes", the selected calculation method is to find the difference between the projection parameter adjustment value and the current projection parameter of the head-up display device at the current moment (the current projection parameter of the head-up display device at the current moment minus the projection parameter adjustment value); when the current behavior tendency information is "the driver may be distracted and absent-minded", the calculation method is to obtain the current projection parameter of the head-up display device at the current moment as the projection parameter of the head-up display device (that is, the projection parameter at the next moment t+1).

[0066] That is, when the current behavior tendency information is "the driver is currently focused, the distance between vehicles is small, and he may be nervous or anxious" or "the driver is currently in information monitoring mode," the calculation process of the head-up display projection brightness is as follows: The calculation process of the projection height of the head-up display device is as follows: When the current behavior tendency information is "the driver is currently relaxed, the distance between vehicles is large, and there may be a tendency to drive fatigued," "the driver is currently recovering focus," or "the driver may have a tendency to change lanes," the calculation process for the head-up display projection brightness is as follows: The calculation process of the projection height of the head-up display device is as follows: When the current behavior tendency information is "the driver may be distracted or absent-minded", the calculation process of the head-up display device's projection brightness is as follows: The calculation process of the projection height of the head-up display device is as follows: .

[0067] Generally speaking, when a driver's eyes are focused on distant objects, their attention is on the road ahead rather than the interior of the vehicle. Therefore, the projection parameters need to be increased to attract the driver's attention through high-parameter display content, allowing the driver to focus on the content projected on the windshield by the head-up display. Conversely, when a driver's eyes are focused on nearby objects, their attention is on the interior of the vehicle. Therefore, the projection parameters can be reduced to attract the driver's attention through low-parameter display content, allowing the driver to focus on the content projected on the windshield by the head-up display. This means that the distance the driver's eyes focus is positively correlated with the projection parameters.

[0068] When the driver's eyes focus on something far away, the pupil radius is larger, and when the driver's eyes focus on something close, the pupil radius is smaller. That is, the pupil radius of the driver's eyes is positively correlated with the distance of the driver's eyes. Based on the positive correlation between the aforementioned projection parameters and the distance of the driver's eyes, it can be seen that the pupil radius of the driver's eyes is also positively correlated with the projection parameters. Therefore, in this application, when the pupil radius is When , the projection parameter takes the maximum value, and the pupil radius is When , the projection parameter is taken as the minimum value, and then the current projection parameter is adjusted based on the change rate of the projection parameter and the current projection parameter change value.

[0069] As mentioned above, the formula for calculating projection parameters (including 、 ) we can see that the rate of change of the projection parameter is negative, and in the triangle △R (that is, ) is a positive value, which means that the driver's pupil radius becomes larger and the driver's focusing distance changes from small to large. At this time, the projection parameter needs to be increased. Therefore, the projection parameter at the next moment is determined by taking the difference between the current projection parameter and the product (the rate of change of the projection parameter and the current projection parameter change value) to increase the projection parameter. Similarly, when △R is a negative value, which means that the pupil radius becomes smaller and the driver's focusing distance changes from large to small, the projection parameter needs to be lowered. Therefore, the projection parameter at the next moment is determined by taking the sum of the current projection parameter and the product (the rate of change of the projection parameter and the current projection parameter change value) to reduce the projection parameter.

[0070] It is worth mentioning that every time a driver gets in the car and drives it, the vehicle's OMS system will identify his or her facial features. If it is recognized that the current driver is not the owner of the vehicle, the first eye state information and the second eye state information of the current driver will be re-collected and stored in the driver's eye information storage system, so as to determine the change rate of each driver's respective projection parameters.

[0071] However, to ensure that the memory occupied by information storage does not affect the functional operation of other key systems of the vehicle, the driver eye storage system of the same vehicle can accommodate the first eye state information and second eye state information of up to N drivers (N is a natural number, for example, N is 5 or 6). When the facial features of the identified N+1th driver are different from those of the previous N drivers, the driver eye information storage system will sequentially clear the first eye state information and second eye state information of the first driver other than the vehicle owner, and so on.

[0072] Alternatively, if the vehicle storage space is sufficient, the first eye state information and the second eye state information of all drivers driving the vehicle can be stored.

[0073] S140: Control the head-up display device based on the projection parameters.

[0074] As mentioned above, the obtained projection parameters are actually the projection parameters at the next time t+1. Therefore, when the next time t+1 is reached, the head-up display device can be controlled based on the projection parameters so that the head-up display device projects according to the determined projection parameters.

[0075] For example, the projection parameters include projection brightness and projection height When the next moment t+1 is reached, the head-up display device is controlled according to the projection brightness , project the content, and project the projected content at the projection height The corresponding position.

[0076] In this embodiment, based on the current eye state information of the driver's eyes, current focus state information indicating the current focus state of the driver's eyes is determined, and then based on the current focus state information of the driver, the projection parameters of the head-up display device are determined, and the head-up display device is controlled based on the projection parameters of the head-up display device. In this way, the purpose of adjusting the projection parameters of the head-up display device in real time in combination with the driver's eye state is achieved, so that the projection parameters of the head-up display device are adjusted to match the driver's eye state, and then when the head-up display device is projected according to the projection parameters of the head-up display device, the image projected by the head-up display device is easier for the driver to see clearly, thereby improving the utilization rate of the driver's attention and greatly improving the projection effect and driving safety.

[0077] Secondly, behavior analysis and behavior tendency analysis are performed based on the current focus state information to obtain the driver's current behavior information and the driver's current behavior tendency information. Then, based on the driver's current behavior tendency information, the projection parameters of the head-up display device are determined. Therefore, in the process of determining the projection parameters of the head-up display device, the driver's behavior tendency is taken into consideration, so that the projection parameters of the head-up display device are more consistent with the driver's behavior tendency. Therefore, when the head-up display device is controlled according to the determined projection parameters, the head-up display device can project according to the driver's behavior tendency, so that the content projected by the head-up display device is easier for the driver to view, further improving the utilization rate of the driver's attention, and further improving the projection effect and driving safety of the head-up display device.

[0078] Furthermore, in the present application, the real-time changes in the focus state information of the driver's eyes are determined based on the real-time changes in the eye state information, so that the determined current focus state information of the driver can more accurately indicate the focus state of the driver's eyes, and the current focus state information is more accurate, thereby making the projection parameters determined based on the current focus state information more accurate, further improving the projection effect.

[0079] In some embodiments, after A2 in the aforementioned embodiment, the method may further include: enabling a vehicle function corresponding to the current behavior tendency information.

[0080] A corresponding vehicle function may be set for each behavioral tendency information of the driver, and then after the current behavioral tendency information is determined, the vehicle function corresponding to the current behavioral tendency information may be enabled.

[0081] For example, when the driver's behavior tendency information is "the driver is currently in a focused state, the distance between vehicles is small, and he may be nervous and anxious", the corresponding vehicle function may be a music playback function that plays soothing music and a safety assisted driving mode that should be turned on but is not turned on (such as a voice assistant function, a navigation function, a lane line prompt function, etc.). For another example, when the driver's behavior tendency information is "the driver is currently in a relaxed state, the distance between vehicles is large, and he may be fatigued driving prone", the corresponding vehicle function may be OMS to detect in real time whether the driver is yawning, squinting, etc., and remind the driver to prevent fatigue driving when yawning, squinting, etc. occurs (which can be voice prompts, light prompts, or text prompts, etc.). For another example, when the driver's behavior tendency information is "the driver may have a tendency to change lanes", the corresponding vehicle function may be a safety assisted driving mode that should be turned on but is not turned on (such as a voice assistant function, a lane change prompt function, and an automatic turning signal on and off function, etc.).

[0082] Of course, the vehicle functions corresponding to each behavioral tendency information mentioned above are only examples. More or fewer vehicle functions can be set for each behavioral tendency information based on needs, which will not be elaborated here.

[0083] In this embodiment, corresponding behavioral tendencies are determined for different focus state information, and then other vehicle functions in the vehicle are linked based on the behavioral tendencies, thereby improving the functions of the smart cockpit system, making the smart cockpit system more functional and providing a better driver experience.

[0084] In some embodiments, when the driver's current behavioral tendency information indicates "the driver may be distracted or absent-minded," if it is determined that the duration of the corneal area change information ΔS < 0 reaches a specified time, the driver may be prompted to pay attention to the road conditions, including but not limited to voice, text, or light. The specified time period may be 3 seconds, for example.

[0085] In other words, when the driver's current behavioral tendency information is "the driver may be distracted or absent-minded", if it is determined that the duration of the corneal area change information △S<0 reaches a specified time, it means that the driver is distracted or absent-minded for a long time, and there is a safety hazard in the driver's driving process. At this time, the driver is reminded so that the driver can regain his attention in time and ensure the driver's driving safety.

[0086] For example, the operation process of the control function of the control method of the head-up display device in the vehicle integrated with the present application is as follows: Figure 3 shown.

[0087] The vehicle OMS monitors changes in the driver's face and, when it is determined that the driver is using the vehicle for the first time, asks the driver whether to turn on the control function. If turned on, the head-up display device is controlled based on the control function. If not turned on, the control function is turned off and projection is performed according to the preset projection strategy, wherein the preset projection strategy is the projection strategy of the head-up display device (HUD) factory setting. The preset projection strategy is preset by the manufacturer of the head-up display device, and is generally determined by combining the original HUD maximum projection brightness Lmax, the original HUD minimum projection brightness Lmin, the original HUD maximum projection height Hmax, the original HUD minimum projection height Hmin and external environmental information. The preset projection strategy is used to determine the projection parameters: projection brightness and projection height.

[0088] The process of collecting the eyeball state information (the first eyeball state information and the second eyeball state information) in the above embodiment is as follows: Figure 4 shown.

[0089] The pupil radius R01 of the driver when looking at the front windshield and the pupil radius R02 of the driver when looking at the target position are collected through an eye positioning system (such as OMS), and the collected R01 and R02 are stored in the driver eye information storage system in the vehicle.

[0090] The tracking process of pupil radius change information and corneal area change information in the above embodiment is as follows: Figure 5 shown.

[0091] The driver's current pupil radius Rt and current corneal area St at the current moment, as well as the driver's historical pupil radius Rt0 and historical corneal area St0 at the previous moment are collected through an eye tracking system (such as OMS). Based on the current pupil radius Rt and the historical pupil radius Rt0, pupil radius change information △R=Rt-Rt0 is obtained, and based on the current corneal area St and the historical corneal area St0, pupil radius change information △S=St-St0 is obtained.

[0092] The process of determining the behavioral tendency information in the above embodiment is as follows: Figure 6 shown.

[0093] Accordingly, the process of determining the projection parameters corresponding to each behavioral tendency information in the above embodiment is as follows: Figure 7 As shown in FIG. , Lt is the projection brightness at the current moment, Lt+1 is the projection brightness at the next moment after the current moment, Ht is the projection height at the current moment, Ht+1 is the projection height at the next moment after the current moment, and Lt+1 and Ht+1 are the projection parameters determined in S130.

[0094] Accordingly, the process of determining the vehicle function corresponding to each behavior tendency information in the above embodiment is as follows: Figure 8 shown.

[0095] See attached Figure 9 , Figure 9 The following is a block diagram of a control device for a head-up display device in a vehicle, according to one embodiment of the present application. The control device 900 for a head-up display device in a vehicle includes: The collecting module 910 is used to collect the current eye state information of the driver driving the vehicle; A first determining module 920 is configured to determine current focus state information of the eye based on the current eye state information; A second determining module 930 is configured to determine projection parameters of the head-up display device based on the current focus state information; The control module 940 is configured to control the head-up display device based on the projection parameters.

[0096] Optionally, the second determination module 930 is further used to perform behavior analysis based on the current focus state information to obtain the driver's current behavior information; perform behavior tendency analysis based on the current behavior information to obtain the driver's current behavior tendency information; and determine the projection parameters of the head-up display device based on the current behavior tendency information.

[0097] Optionally, the second determination module 930 is further used to determine the projection parameters of the head-up display device based on current behavioral tendency information, the change rate of the projection parameters of the head-up display device, the current projection parameters of the head-up display device at the current moment, and the current projection parameter change value of the head-up display device at the current moment.

[0098] Optionally, the second determination module 930 is also used to obtain first eye state information and second eye state information of the eye; the first eye state information is the eye state information of the eye when the eye is looking at the windshield of the vehicle; the second eye state information is the eye state information of the eye when the eye is looking at a target position, wherein the target position is different from the position of the windshield; based on the first eye state information, the second eye state information, the maximum projection parameters and the minimum projection parameters of the head-up display device, the change rate of the projection parameters of the head-up display device is determined.

[0099] Optionally, the first determination module 920 is further configured to obtain historical eye state information at a moment before the current eye state; and determine current focus state information based on a difference between the historical eye state information and the current eye state information.

[0100] Optionally, the first determination module 920 is also used to determine the pupil radius change information of the eyeball based on the difference between the pupil radius in the current eyeball state information and the pupil radius in the historical eyeball state information; determine the corneal area change information of the eyeball based on the difference between the corneal area in the current eyeball state information and the corneal area in the historical eyeball state information; and determine the current focusing state information based on the pupil radius change information and the corneal area change information.

[0101] Optionally, the control module 940 is further configured to activate a vehicle function corresponding to the current behavior tendency information.

[0102] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices and modules can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0103] In addition, the functions in the various embodiments of the present application may be integrated into a single processing module, or each module may exist physically separately, or two or more modules may be integrated into a single module. The aforementioned integrated modules may be implemented in the form of hardware or software functional modules.

[0104] On the other hand, the present application also provides a computer-readable storage medium, which stores program code. The program code can be called by a processor to execute the method described in the above method embodiment.

[0105] The computer-readable storage medium may be an electronic memory such as flash memory, EEPROM (Electrically Erasable Programmable Read-Only Memory), EPROM, a hard disk, or a cluster of ROMs. Alternatively, the computer-readable storage medium includes a non-transitory computer-readable storage medium. The computer-readable storage medium has storage space for program code for executing any of the method steps described above. This program code can be read from or written to one or more computer program products. The program code may be compressed, for example, in a suitable format.

[0106] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for controlling a head-up display device in a vehicle, characterized in that: The method comprises: collecting current eye state information of the driver driving the vehicle; Determining current focus state information of the eyeball based on the current eyeball state information; determining projection parameters of the head-up display device based on the current focus state information; The head-up display device is controlled based on the projection parameters.

2. The method according to claim 1, characterized in that The determining, based on the current focus state information, projection parameters of the head-up display device includes: Performing behavior analysis based on the current focus state information to obtain current behavior information of the driver; Performing a behavior tendency analysis based on the current behavior information to obtain the driver's current behavior tendency information; Based on the current behavior tendency information, projection parameters of the head-up display device are determined.

3. The method according to claim 2, characterized in that The determining of the projection parameters of the head-up display device based on the current behavior tendency information includes: The projection parameters of the head-up display device are determined based on the current behavior tendency information, the change rate of the projection parameters of the head-up display device, the current projection parameters of the head-up display device at the current moment, and the current projection parameter change value of the head-up display device at the current moment.

4. The method according to claim 3, characterized in that Before determining the projection parameters of the head-up display device based on the current behavior tendency information, the rate of change of the projection parameters of the head-up display device, the current projection parameters of the head-up display device at the current moment, and the change value of the current projection parameters of the head-up display device at the current moment, the method further includes: Obtaining first eye state information and second eye state information of the eye; the first eye state information is the eye state information when the eye is gazing at the windshield of the vehicle; the second eye state information is the eye state information when the eye is gazing at a target position; wherein the target position is different from the position of the windshield; A change rate of the projection parameter of the head-up display device is determined based on the first eyeball state information, the second eyeball state information, a maximum projection parameter, and a minimum projection parameter of the head-up display device.

5. The method according to any one of claims 1 to 4, characterized in that The determining, based on the current eye state information, the current focusing state information of the eye includes: Obtaining historical eye state information at a moment before the current moment of the eye; The current focus state information is determined based on a difference between the historical eye state information and the current eye state information.

6. The method according to claim 5, characterized in that The determining the current focus state information based on the difference between the historical eye state information and the current eye state information includes: determining pupil radius change information of the eyeball based on a difference between the pupil radius in the current eyeball state information and the pupil radius in the historical eyeball state information; determining corneal area change information of the eyeball based on a difference between the corneal area in the current eyeball state information and the corneal area in the historical eyeball state information; The current focus state information is determined based on the pupil radius change information and the cornea area change information.

7. The method according to any one of claims 2 to 4, characterized in that After performing the behavior tendency analysis based on the current behavior information to obtain the current behavior tendency information of the driver, the method further includes: A vehicle function corresponding to the current behavior tendency information is activated.

8. A control device for a head-up display device in a vehicle, characterized in that: The device comprises: a collection module, configured to collect current eye state information of the driver driving the vehicle; A first determining module is configured to determine current focus state information of the eye based on the current eye state information; a second determining module, configured to determine projection parameters of the head-up display device based on the current focus state information; A control module is configured to control the head-up display device based on the projection parameters.

9. A vehicle, characterized in that: include: one or more processors; Memory; Head-up display; One or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the one or more processors, and the one or more applications are configured to execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores program codes executable by a processor, and when the program codes are executed by the processor, the processor executes the method according to any one of claims 1 to 7.

Images