Vehicle interaction method and device, vehicle and storage medium

Abstract

The application relates to a vehicle interaction method and device, a vehicle and a storage medium, wherein the method comprises the following steps: recognizing pupil information of a driver, determining a current driving state and an interaction intention of the driver based on the pupil information, and determining a driving risk coefficient of the driver based on a current driving parameter of the vehicle; determining display content of a head-up display of the vehicle in combination with the driving risk coefficient, the current driving state and the interaction intention; tracking the pupil information to determine an actual interaction intention of the driver in combination with the tracked pupil information and / or a gesture action of the driver, and controlling the vehicle to perform a corresponding interaction action according to the actual interaction intention and the display content. Thus, the technical problem that, in the related art, vehicle interaction is mostly dependent on manual touch control or voice operation, the driver needs to divert his / her line of sight or be distracted, and safety hazards are easily caused is solved.

Description

Technical Field

[0001] This application relates to the field of electronic digital data processing technology, and in particular to a vehicle interaction method, device, vehicle, and storage medium. Background Technology

[0002] With the development of vehicle intelligence and vehicle networking services, the number of devices and information in vehicles has increased significantly compared to the past, and the in-vehicle systems are becoming increasingly complex. At the same time, the demand for human-vehicle interaction is also increasing, resulting in a higher cognitive load for drivers. Traditional human-machine interfaces (physical buttons, knobs, touch screens) cannot solve the problems under the new circumstances. In related technologies, vehicle interaction mostly relies on manual touch or voice operation, which requires drivers to shift their eyes or be distracted, easily causing safety hazards, and urgently needs to be improved. Summary of the Invention

[0003] This application provides a vehicle interaction method, device, vehicle, and storage medium to solve the technical problem in the related art that vehicle interaction mostly relies on manual touch or voice operation, which requires the driver to shift their gaze or be distracted, easily leading to safety hazards.

[0004] The first aspect of this application provides a vehicle interaction method, comprising the following steps: identifying the driver's pupil information; determining the driver's current driving state and interaction intention based on the pupil information; determining the driver's driving risk coefficient based on the vehicle's current driving parameters; determining the display content of the vehicle's head-up display by combining the driving risk coefficient, the current driving state, and the interaction intention; tracking the pupil information to determine the driver's actual interaction intention by combining the tracked pupil information and / or the driver's hand gestures; and controlling the vehicle to perform corresponding interaction actions according to the actual interaction intention and the display content.

[0005] Optionally, in one embodiment of this application, determining the driver's current driving state and interaction intention based on the pupil information includes: extracting the driver's iris texture from the pupil information, and calculating the driver's iris contraction rate using the iris texture; and determining the current driving state based on the iris contraction rate.

[0006] Optionally, in one embodiment of this application, determining the driver's driving risk coefficient based on the vehicle's current driving parameters includes: obtaining the vehicle's sudden steering angle change and rapid acceleration based on the current driving parameters; and calculating the vehicle's driving risk coefficient based on the sudden steering angle change and the rapid acceleration.

[0007] Optionally, in one embodiment of this application, determining the display content of the vehicle's head-up display by combining the driving risk coefficient, the current driving state, and the interaction intent includes: determining whether the driver meets preset head-up display interaction conditions based on the interaction intent; if the preset head-up display interaction conditions are met, determining whether the driving risk coefficient is greater than a preset coefficient threshold; if the driving risk coefficient is greater than the preset coefficient threshold, increasing the transparency of the head-up display to a preset transparency, compressing the display content to a preset size, and disabling applications that do not meet the preset safety interaction conditions; if the driving risk coefficient is less than or equal to the preset coefficient threshold, matching a corresponding display scheme based on the current driving state, and adjusting the display content based on the display scheme.

[0008] Optionally, in one embodiment of this application, determining the driver's actual interaction intention by combining the tracked pupil information and / or the driver's gestures includes: determining the driver's focus target based on the tracked pupil information; determining the driver's eye tremor frequency if the focus target is within the display area of ​​the head-up display; if the eye tremor frequency is within a preset safety range, determining the actual interaction intention based on the focus target; otherwise, ignoring the focus target.

[0009] Optionally, in one embodiment of this application, after recognizing the driver's pupil information, the method further includes: acquiring the vehicle's interior light intensity information; determining whether the pupil information meets a preset credibility condition based on the interior light intensity information; if the preset credibility condition is not met, determining the actual interaction intent based on the gesture and / or the driver's voice command.

[0010] A second aspect of this application provides a vehicle interaction device, comprising: a recognition module for recognizing a driver's pupil information, determining the driver's current driving state and interaction intention based on the pupil information, and determining the driver's driving risk coefficient based on the vehicle's current driving parameters; a first determination module for determining the display content of the vehicle's head-up display by combining the driving risk coefficient, the current driving state, and the interaction intention; and an interaction module for tracking the pupil information to determine the driver's actual interaction intention by combining the tracked pupil information and / or the driver's hand gestures, and controlling the vehicle to perform corresponding interaction actions according to the actual interaction intention and the display content.

[0011] Optionally, in one embodiment of this application, the recognition module includes: a first extraction unit, configured to extract the driver's iris texture from the pupil information and calculate the driver's iris contraction rate using the iris texture; and a first determination unit, configured to determine the current driving state based on the iris contraction rate.

[0012] Optionally, in one embodiment of this application, the identification module includes: a second extraction unit, configured to obtain the sudden change in steering angle and the rapid acceleration of the vehicle based on the current driving parameters; and a calculation unit, configured to calculate the driving risk coefficient of the vehicle based on the sudden change in steering angle and the rapid acceleration.

[0013] Optionally, in one embodiment of this application, the determining module includes: a first determining unit, configured to determine whether the driver meets preset head-up display interaction conditions based on the interaction intent; a second determining unit, configured to determine whether the driving risk coefficient is greater than a preset coefficient threshold if the preset head-up display interaction conditions are met; a first adjusting unit, configured to increase the transparency of the head-up display to a preset transparency, compress the displayed content to a preset size, and disable applications that do not meet the preset safety interaction conditions if the driving risk coefficient is greater than the preset coefficient threshold; and a second adjusting module, configured to match a corresponding display scheme based on the current driving state and adjust the displayed content based on the display scheme if the driving risk coefficient is less than or equal to the preset coefficient threshold.

[0014] Optionally, in one embodiment of this application, the interaction module includes: a second determining unit, configured to determine the driver's focus target based on the tracked pupil information; a third determining unit, configured to determine the driver's eye tremor frequency when the focus target is within the display area of ​​the head-up display; and a fourth determining unit, configured to determine the actual interaction intention based on the focus target when the eye tremor frequency is within a preset safety range, otherwise, ignore the focus target.

[0015] Optionally, in one embodiment of this application, it further includes: an acquisition module, configured to acquire interior light intensity information of the vehicle; a judgment module, configured to determine whether the pupil information meets a preset credibility condition based on the interior light intensity information; and a second determination module, configured to determine the actual interaction intention based on the gesture and / or the driver's voice command if the preset credibility condition is not met.

[0016] A third aspect of this application provides a vehicle, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the vehicle interaction method as described in the above embodiments.

[0017] A fourth aspect of this application provides a computer-readable storage medium storing computer instructions for causing the computer to perform the vehicle interaction method as described in the above embodiments.

[0018] A fifth aspect of this application provides a computer program product, including a computer program that, when executed, implements the above-described vehicle interaction method.

[0019] This application's embodiments can analyze the driver's pupil information to determine the driver's current driving state and interaction intentions. Based on the vehicle's driving risk coefficient, it dynamically matches the content displayed on the vehicle's head-up display (HUD) to achieve targeted display of interactive content. Furthermore, it tracks pupil information and combines this tracking information with / or the driver's hand gestures to determine the driver's actual interaction intentions, thereby achieving intelligent human-vehicle interaction. This reduces the probability of driver distraction, improves the driver's driving experience, and ensures driving safety. Therefore, it solves the technical problem in related technologies where vehicle interaction largely relies on manual touch or voice operation, requiring the driver to shift their gaze or become distracted, easily leading to safety hazards.

[0020] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0021] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:

[0022] Figure 1 This is a flowchart of a vehicle interaction method provided according to an embodiment of this application;

[0023] Figure 2 This is a schematic diagram illustrating the principle of a vehicle interaction method according to an embodiment of this application;

[0024] Figure 3 This is a flowchart of a vehicle interaction method provided according to an embodiment of this application;

[0025] Figure 4 This is a schematic diagram of the installation position of an eye-tracking device according to an embodiment of this application;

[0026] Figure 5This is a schematic diagram of the overall vehicle layout according to an embodiment of this application;

[0027] Figure 6 This is a schematic diagram illustrating the principle of gesture control according to an embodiment of this application;

[0028] Figure 7 This is a schematic diagram illustrating the principle of interaction combining gaze and gesture actions according to an embodiment of this application;

[0029] Figure 8 This is a schematic diagram of the structure of a vehicle interaction device according to an embodiment of this application;

[0030] Figure 9 This is a structural schematic diagram of a vehicle provided according to an embodiment of this application. Detailed Implementation

[0031] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.

[0032] The following description, with reference to the accompanying drawings, describes a vehicle interaction method, device, vehicle, and storage medium according to embodiments of this application. Addressing the technical problem mentioned in the background art, where vehicle interaction often relies on manual touch or voice operation, requiring the driver to shift their gaze or become distracted, easily leading to safety hazards, this application provides a vehicle interaction method. In this method, the driver's pupil information can be analyzed to determine the driver's current driving state and interaction intention. Based on the vehicle's driving risk coefficient, the display content of the vehicle's head-up display is dynamically matched to achieve targeted display of interactive content. Furthermore, pupil information is tracked, and combined with the tracked pupil information and / or the driver's hand gestures, the driver's actual interaction intention is determined, thereby achieving intelligent human-vehicle interaction, reducing the probability of driver distraction, improving the driver's driving experience, and ensuring driving safety. Thus, this solves the technical problem in related technologies where vehicle interaction often relies on manual touch or voice operation, requiring the driver to shift their gaze or become distracted, easily leading to safety hazards.

[0033] Specifically, Figure 1 This is a flowchart illustrating a vehicle interaction method provided in an embodiment of this application.

[0034] like Figure 1 As shown, the interaction method of this vehicle includes the following steps:

[0035] In step S101, the driver's pupil information is identified, and based on the pupil information, the driver's current driving state and interaction intention are determined, and the driver's driving risk coefficient is determined based on the vehicle's current driving parameters.

[0036] In actual implementation, this application embodiment can install an eye-tracking device in the vehicle to identify the driver's pupil information, including iris texture, pupil position, eye tremors, etc. Combined with the physiological signals of the driver's pupils, this application embodiment can determine the driver's current driving state (such as relaxed driving state, tense driving state, etc.) and interaction intention (such as interacting with the head-up display, not interacting with the head-up display, etc.).

[0037] In addition, this application embodiment can also obtain the vehicle's current driving parameters to determine whether the vehicle is in a driving state, and calculate the driving risk coefficient when the vehicle is in a driving state.

[0038] Optionally, in one embodiment of this application, determining the driver's current driving state and interaction intention based on pupil information includes: extracting the driver's iris texture from the pupil information and calculating the driver's iris contraction rate using the iris texture; and determining the current driving state based on the iris contraction rate.

[0039] In some embodiments, the information density of the head-up display can be proactively adjusted by analyzing changes in iris texture (contraction when tense / expansion when relaxed) in real time to predict the driver's interaction needs (such as navigation anxiety, entertainment relaxation).

[0040] This application embodiment can utilize an eye-tracking device to capture the driver's iris texture via an infrared camera and calculate the iris contraction rate (IRR):

[0041]

[0042] For example, when IRR < 0.9, the current driving state can be determined to be tense; when IRR > 1.1, the current driving state can be determined to be relaxed.

[0043] Optionally, in one embodiment of this application, after recognizing the driver's pupil information, the method further includes: acquiring the vehicle's interior light intensity information; determining whether the pupil information meets a preset credibility condition based on the interior light intensity information; if the preset credibility condition is not met, determining the actual interaction intent based on gesture actions and / or the driver's voice commands.

[0044] It is understandable that when the light is too strong, it is difficult to accurately identify the driver's pupil information, and the content of the head-up display cannot be clearly displayed. Therefore, this application embodiment can determine whether the identified pupil information is reliable based on the vehicle's internal light intensity information. If it is not reliable, this application embodiment can directly complete the human-vehicle interaction through voice commands and / or gestures.

[0045] In step S102, the content to be displayed on the vehicle's head-up display is determined by combining the driving risk coefficient, the current driving state, and the interaction intent.

[0046] Furthermore, embodiments of this application can combine driving risk factor (to determine display density), current driving state (to determine the application to be displayed), and interaction intent (to determine whether display is required) to determine the specific display content of the head-up display, including display size, display brightness, transparency, and application to be displayed, so as to dynamically adjust the interaction content for different situations.

[0047] Optionally, in one embodiment of this application, determining the driver's driving risk coefficient based on the vehicle's current driving parameters includes: obtaining the vehicle's sudden steering angle change and rapid acceleration based on the current driving parameters; and calculating the vehicle's driving risk coefficient based on the sudden steering angle change and rapid acceleration.

[0048] As one possible implementation method, embodiments of this application can calculate the Driving Risk Ratio (DRS) in real time:

[0049] DRS = α·Sudden change in steering angle + β·Rapid acceleration

[0050] Where α and β are sensor weighting coefficients.

[0051] In step S103, pupil information is tracked to determine the driver's actual interaction intention by combining the tracked pupil information and / or the driver's hand gestures, and the vehicle is controlled to perform corresponding interaction actions based on the actual interaction intention and the displayed content.

[0052] In actual implementation, this application embodiment can track pupil information to determine whether the driver's focus target has changed, etc. Based on the tracked pupil information, this application embodiment can determine the driver's actual interaction intention. For example, if the driver's gaze is focused on a certain application for a long time, the application will be launched. In order to increase the credibility of the recognition results, this application embodiment can also combine gesture actions to further confirm the driver's intention, and then control the vehicle.

[0053] Optionally, in one embodiment of this application, determining the driver's actual interaction intention by combining the tracked pupil information and / or the driver's hand gestures includes: determining the driver's focus target based on the tracked pupil information; determining the driver's eye tremor frequency if the focus target is within the display area of ​​the head-up display; if the eye tremor frequency is within a preset safety range, determining the actual interaction intention based on the focus target; otherwise, ignoring the focus target.

[0054] In other embodiments, after detecting that the gaze is focused on the target object for a certain period of time, such as 0.3 seconds, the frequency of eye tremors is analyzed (e.g., 8-12 Hz for natural focusing):

[0055] If the tremor frequency is within the normal range (e.g., 10±2Hz), directly use the focused target as the actual interaction intent and execute the default operation of that object (e.g., focus on the song icon to play).

[0056] If the frequency is abnormal (e.g., <6Hz or >15Hz), it is determined to be unintentional staring, and the operation is ignored.

[0057] Optionally, in one embodiment of this application, the display content of the vehicle's head-up display is determined by combining the driving risk coefficient, the current driving state, and the interaction intent, including: determining whether the driver meets the preset head-up display interaction conditions based on the interaction intent; if the preset head-up display interaction conditions are met, determining whether the driving risk coefficient is greater than a preset coefficient threshold; if the driving risk coefficient is greater than the preset coefficient threshold, increasing the transparency of the head-up display to a preset transparency, compressing the display content to a preset size, and disabling applications that do not meet the preset safety interaction conditions; if the driving risk coefficient is less than or equal to the preset coefficient threshold, matching the corresponding display scheme based on the current driving state, and adjusting the display content based on the display scheme.

[0058] First, the embodiments of this application can determine whether the driver intends to interact with the head-up display based on the interaction intent. For example, based on the frequency of eye tremors, it can be determined whether the current focus target is an intentional gaze or an unintentional gaze. If it is an intentional gaze, the focus target's landing point is determined to be the interaction area of ​​the head-up display or a normal gaze at the road surface. When the landing point is determined to be in the interaction area, it is determined that there is an intention to interact with the head-up display.

[0059] Based on this, the embodiments of this application can determine the display content of the head-up display according to the driving risk coefficient. For example, if the driving risk coefficient is high, the transparency of the head-up display can be increased and the display size of the head-up display can be reduced to avoid the head-up display affecting the driver's normal observation of the road.

[0060] If the driving risk factor is low, the displayed content can be adjusted according to the driving state. For example, in a tense state, the head-up display interface can be simplified, retaining only navigation and warning information; in a relaxed state, the entertainment controls can be expanded.

[0061] In addition, it can also display pre-set personalized content for each driver based on pupil recognition.

[0062] Combination Figures 2-6 As shown, the working principle of the vehicle interaction method of this application embodiment is explained in detail with reference to one embodiment.

[0063] The embodiments of this application can be combined with Figure 2 The structure shown enables human-vehicle interaction, and the execution flow can be as follows: Figure 3 As shown.

[0064] The eye-tracking device mainly consists of an infrared lamp, a camera, an ECU controller, a PCB circuit board, and a housing. Figure 4 As shown, it can be installed on the dashboard, steering wheel, or A-pillar area. After the vehicle is powered on, it monitors the movement of the driver's eyes and the pupil's focusing area in real time, and transmits the captured image information to the control ECU for raw image information processing. After processing, it obtains the driver's gaze focus point information and sends this information to the data interaction processing ECU via CAN / CAN-FD / Ethernet or other transmission methods. Based on the above installation location, the vehicle layout of this embodiment can be as follows: Figure 5 As shown, the eye-tracking device is used to obtain the driver's pupil information to determine the interaction target with the head-up display, and to confirm the actual intention by combining it with gestures.

[0065] like Figure 6 As shown, the gesture button device mainly consists of physical or virtual buttons such as volume up, volume down, left movement, right movement, mode selection / confirmation, etc. It can be installed in the steering wheel area. After the vehicle is powered on, it senses the driver's gesture behavior information and processes the information before sending it to the data interaction processing ECU through transmission methods such as LIN / CAN / CAN-FD / Ethernet.

[0066] A HUD (Head-Up Display) system is a device that projects the in-vehicle system interface onto the windshield or external glass panel through a projection device and a mirror, forming a virtual image. It is usually installed in front of the driver's cab to display information such as speed and HMI information, allowing the driver to obtain speed and in-vehicle HMI information without having to look down at the instrument panel and the main unit.

[0067] The HMI functional area is mainly composed of icons for calls, music, navigation, air conditioning, etc. Figure 7As shown, when the driver's gaze enters the HMI function area, the icons within the area brighten and increase in size. When the driver's gaze falls on a specific icon, that icon further enlarges within 50ms, while other icons return to their initial size and brightness. If the driver's gaze does not leave the HMI function area within 50ms, the driver enters the corresponding icon's function interface (e.g., looking at the music icon enters the music selection interface). Once a function is selected, the driver's intention is confirmed through interaction with gesture information. If the driver's gaze leaves the function area after selection or during selection, the initial HMI function area screen returns to its original state after 50ms.

[0068] After the vehicle is powered on, the eye-tracking device identifies the driver's pupil information. If the identification is successful and matches the pupil information stored in the vehicle, the HMI switches to the theme previously set by the driver. If the identification fails, the default HMI theme interface is entered.

[0069] In these two ways, eye-tracking devices enable the HUD interface to proactively respond to user needs: 1. By matching the driver's pupils, the HMI theme mode can be quickly adjusted to cater to the driver's behavioral preferences; 2. The driver's gaze indicates their willingness to interact. When there is no intention to interact, the driver's gaze is outside the functional area, and the HUD interface displays high transparency and a small size, which does not interfere with the driver's vision, reducing driver distraction and making driving safer. In this process, eye tracking establishes the interaction rule of "the target the driver looks at is the object being controlled," replacing manual selection of the target interaction object with natural gaze focus. This allows the driver to directly use general gestures to control the in-vehicle equipment without entering functional sub-levels, making the interaction process more convenient and effortless.

[0070] The vehicle interaction method proposed in this application analyzes the driver's pupil information to determine the driver's current driving state and interaction intention. Based on the vehicle's driving risk coefficient, it dynamically matches the content displayed on the vehicle's head-up display to achieve targeted display of interactive content. Furthermore, it tracks pupil information and combines this tracking information with / or the driver's hand gestures to determine the driver's actual interaction intention, thereby achieving intelligent human-vehicle interaction. This reduces the probability of driver distraction, improves the driver's driving experience, and ensures driving safety. Thus, it solves the technical problem in related technologies where vehicle interaction largely relies on manual touch or voice operation, requiring the driver to shift their gaze or become distracted, easily leading to safety hazards.

[0071] Next, the interactive device for a vehicle according to an embodiment of this application is described with reference to the accompanying drawings.

[0072] Figure 8 This is a block diagram of the vehicle's interactive device according to an embodiment of this application.

[0073] like Figure 8As shown, the vehicle's interactive device 10 includes: an identification module 100, a first determination module 200, and an interactive module 300.

[0074] Specifically, the recognition module 100 is used to recognize the driver's pupil information, determine the driver's current driving state and interaction intention based on the pupil information, and determine the driver's driving risk coefficient based on the vehicle's current driving parameters.

[0075] The first determining module 200 is used to determine the content to be displayed on the vehicle's head-up display by combining the driving risk coefficient, the current driving state, and the interaction intent.

[0076] The interaction module 300 is used to track pupil information, combine the tracked pupil information and / or the driver's hand gestures to determine the driver's actual interaction intention, and control the vehicle to perform corresponding interaction actions based on the actual interaction intention and the displayed content.

[0077] Optionally, in one embodiment of this application, the identification module 100 includes a first extraction unit and a first determination unit.

[0078] The first extraction unit is used to extract the driver's iris texture from the pupil information and to calculate the driver's iris contraction rate using the iris texture.

[0079] The first determining unit is used to determine the current driving state based on the iris contraction rate.

[0080] Optionally, in one embodiment of this application, the identification module 100 includes a second extraction unit and a calculation unit.

[0081] The second extraction unit is used to obtain the sudden change in steering angle and rapid acceleration of the vehicle based on the current driving parameters.

[0082] The calculation unit is used to calculate the driving risk coefficient of the vehicle based on sudden changes in steering angle and rapid acceleration.

[0083] Optionally, in one embodiment of this application, the determining module 200 includes: a first determining unit, a second determining unit, a first adjusting unit, and a second adjusting module.

[0084] The first judgment unit is used to determine whether the driver meets the preset head-up display interaction conditions based on the interaction intent.

[0085] The second judgment unit is used to determine whether the driving risk coefficient is greater than the preset coefficient threshold when the preset head-up display interaction conditions are met.

[0086] The first adjustment unit is used to increase the transparency of the head-up display to a preset transparency, compress the displayed content to a preset size, and disable applications that do not meet the preset safety interaction conditions when the driving risk coefficient is greater than a preset coefficient threshold.

[0087] The second adjustment module is used to match the corresponding display scheme based on the current driving state when the driving risk coefficient is less than or equal to the preset coefficient threshold, and to adjust the display content based on the display scheme.

[0088] Optionally, in one embodiment of this application, the interaction module 300 includes: a second determining unit, a third determining unit, and a fourth determining unit.

[0089] The second determining unit is used to determine the driver's focus target based on the tracked pupil information.

[0090] The third determining unit is used to determine the frequency of the driver's eye tremors when the focused target is within the display area of ​​the head-up display.

[0091] The fourth determining unit is used to determine the actual interaction intent based on the focusing target when the eye tremor frequency is within a preset safe range; otherwise, the focusing target is ignored.

[0092] Optionally, in one embodiment of this application, the vehicle's interaction device 10 further includes: an acquisition module, a judgment module, and a second determination module.

[0093] The acquisition module is used to acquire information about the interior light intensity of the vehicle.

[0094] The judgment module is used to determine whether the pupil information meets the preset confidence conditions based on the internal light intensity information.

[0095] The second determining module is used to determine the actual interaction intent based on gestures and / or the driver's voice commands when the preset trust conditions are not met.

[0096] It should be noted that the foregoing explanation of the vehicle interaction method embodiment also applies to the vehicle interaction device of this embodiment, and will not be repeated here.

[0097] The vehicle interaction device proposed in this application can analyze the driver's pupil information to determine the driver's current driving state and interaction intention. Based on the vehicle's driving risk coefficient, it dynamically matches the display content of the vehicle's head-up display to achieve targeted display of interactive content. Furthermore, it tracks pupil information and combines this tracking information with / or the driver's hand gestures to determine the driver's actual interaction intention, thereby achieving intelligent human-vehicle interaction, reducing the probability of driver distraction, improving the driver's driving experience, and ensuring driving safety. This solves the technical problem in related technologies where vehicle interaction largely relies on manual touch or voice operation, requiring the driver to shift their gaze or become distracted, easily leading to safety hazards.

[0098] Figure 9 A schematic diagram of the structure of a vehicle provided in an embodiment of this application. The vehicle may include:

[0099] The memory 901, the processor 902, and the computer program stored on the memory 901 and capable of running on the processor 902.

[0100] When the processor 902 executes the program, it implements the vehicle interaction method provided in the above embodiments.

[0101] Furthermore, the vehicle also includes:

[0102] Communication interface 903 is used for communication between memory 901 and processor 902.

[0103] The memory 901 is used to store computer programs that can run on the processor 902.

[0104] The memory 901 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0105] If the memory 901, processor 902, and communication interface 903 are implemented independently, then the communication interface 903, memory 901, and processor 902 can be interconnected via a bus to complete communication between them. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 9 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0106] Optionally, in a specific implementation, if the memory 901, processor 902, and communication interface 903 are integrated on a single chip, then the memory 901, processor 902, and communication interface 903 can communicate with each other through an internal interface.

[0107] The processor 902 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application.

[0108] This embodiment also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the vehicle interaction method described above.

[0109] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the vehicle interaction method provided in this embodiment of the invention.

[0110] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0111] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0112] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or N executable instructions for implementing custom logic functions or processes, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of this application pertain.

[0113] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0114] It should be understood that the various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, the N steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0115] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0116] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0117] The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of this application.

Claims

1. A vehicle interaction method, characterized in that, Includes the following steps: Identify the driver's pupil information, determine the driver's current driving state and interaction intention based on the pupil information, and determine the driver's driving risk coefficient based on the vehicle's current driving parameters; The content to be displayed on the vehicle's head-up display is determined by combining the driving risk coefficient, the current driving state, and the interaction intent. The pupil information is tracked, and the actual interaction intention of the driver is determined by combining the tracked pupil information and / or the driver's hand gestures. The vehicle is then controlled to perform corresponding interaction actions based on the actual interaction intention and the displayed content. The step of determining the driver's current driving state and interaction intention based on the pupil information includes: extracting the driver's iris texture from the pupil information, and calculating the driver's iris contraction rate using the iris texture; and determining the current driving state based on the iris contraction rate. The step of determining the display content of the vehicle's head-up display by combining the driving risk coefficient, the current driving state, and the interaction intent includes: determining whether the driver meets preset head-up display interaction conditions based on the interaction intent; if the preset head-up display interaction conditions are met, determining whether the driving risk coefficient is greater than a preset coefficient threshold; if the driving risk coefficient is greater than the preset coefficient threshold, increasing the transparency of the head-up display to a preset transparency, compressing the display content to a preset size, and disabling applications that do not meet preset safety interaction conditions; if the driving risk coefficient is less than or equal to the preset coefficient threshold, matching a corresponding display scheme based on the current driving state, and adjusting the display content based on the display scheme.

2. The method according to claim 1, characterized in that, Determining the driver's driving risk coefficient based on the vehicle's current driving parameters includes: The sudden change in steering angle and rapid acceleration of the vehicle are obtained based on the current driving parameters; The driving risk coefficient of the vehicle is calculated based on the sudden change in steering angle and the rapid acceleration.

3. The method according to claim 1, characterized in that, The determination of the driver's actual interaction intention by combining the tracked pupil information and / or the driver's hand gestures includes: The driver's focus target is determined based on the tracked pupil information; When the focused target is within the display area of ​​the head-up display, determine the frequency of the driver's eye tremors; If the frequency of the eye tremor is within a preset safe range, the actual interaction intention is determined based on the focusing target; otherwise, the focusing target is ignored.

4. The method according to claim 1, characterized in that, After identifying the driver's pupil information, the process also includes: Obtain the interior light intensity information of the vehicle; Based on the internal light intensity information, determine whether the pupil information meets the preset credibility conditions; If the preset trust conditions are not met, the actual interaction intent is determined based on the gesture and / or the driver's voice command.

5. A vehicle interaction device, characterized in that, include: The recognition module is used to recognize the driver's pupil information, determine the driver's current driving state and interaction intention based on the pupil information, and determine the driver's driving risk coefficient based on the vehicle's current driving parameters. The determination module is used to determine the content displayed on the vehicle's head-up display by combining the driving risk coefficient, the current driving state, and the interaction intent; An interaction module is used to track the pupil information, combine the tracked pupil information and / or the driver's hand gestures to determine the driver's actual interaction intention, and control the vehicle to perform corresponding interaction actions based on the actual interaction intention and the displayed content. The recognition module includes: a first extraction unit, configured to extract the driver's iris texture from the pupil information and calculate the driver's iris contraction rate using the iris texture; and a first determination unit, configured to determine the current driving state based on the iris contraction rate. The determining module includes: a first determining unit, used to determine whether the driver meets preset head-up display interaction conditions based on the interaction intent; a second determining unit, used to determine whether the driving risk coefficient is greater than a preset coefficient threshold if the preset head-up display interaction conditions are met; a first adjusting unit, used to increase the transparency of the head-up display to a preset transparency, compress the displayed content to a preset size, and disable applications that do not meet the preset safety interaction conditions if the driving risk coefficient is greater than the preset coefficient threshold; and a second adjusting module, used to match a corresponding display scheme based on the current driving state and adjust the displayed content based on the display scheme if the driving risk coefficient is less than or equal to the preset coefficient threshold.

6. A vehicle, characterized in that, include: A memory, a processor, and a computer program stored in the memory and executable on the processor, the processor executing the program to implement the vehicle interaction method as described in any one of claims 1-4.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, The program is executed by the processor to implement the vehicle interaction method as described in any one of claims 1-4.

8. A computer program product, comprising a computer program, characterized in that, When the computer program is executed, it is used to implement the vehicle interaction method as described in any one of claims 1-4.

Images