Holographic projection interaction method and device based on visual tracking, vehicle and medium

Abstract

This application relates to the field of human-computer interaction technology, and in particular to a holographic projection interaction method, device, vehicle, and medium based on vision tracking. The method includes: detecting the current driving state of the vehicle; determining the working state and holographic projection position of multiple rotating cameras and / or multiple eye-tracking cameras based on the driving state; and when the multiple rotating cameras are enabled, collecting the user's facial rotation angle using the multiple rotating cameras; determining at least one target eye-tracking camera based on the facial rotation angle and the working state of the multiple eye-tracking cameras, acquiring camera data collected by the at least one target eye-tracking camera, and performing holographic projection based on the holographic projection position and the camera data. This solves the problems of eye-tracking technology in related technologies being unable to adapt to users' multi-angle rotations and having low interactive fun in vehicles, thus increasing the fun of vehicle-machine interaction, improving the visual interaction experience, and enhancing user comfort.

Description

Technical Field

[0001] This application relates to the field of human-computer interaction technology, and in particular to a holographic projection interaction method, device, vehicle, and medium based on visual tracking. Background Technology

[0002] The widespread application of 360-degree holographic display technology signifies that holographic projection technology has entered a mature stage. Currently, this technology is used in numerous fields such as cultural relic display, tourism, education, healthcare, and the military, and its application scope continues to expand. Automobiles, as the most complex mass-produced product, are well-suited to meet the requirements of holographic projection in terms of equipment, networks, and data processing. Statistics show that humans acquire 80% of information through vision, but driving scenarios require minimizing visual involvement in human-computer interaction, thus weakening the enjoyment of human-computer interaction and failing to differentiate between the interaction needs of leisure and driving scenarios.

[0003] In related technologies, eye-tracking devices are used to collect users' eye parameter information to determine the driver's gaze information inside the vehicle. However, this cannot meet the problem of the driver's multi-angle rotation, does not process data according to the driver's driving state, and has large data errors. Related technologies have also proposed an electronic device with eye-tracking function, but it cannot be applied to vehicles and cannot adapt to the driver's flexible movements to obtain eye angles, which urgently needs to be solved. Summary of the Invention

[0004] This application provides a holographic projection interaction method, device, vehicle, and medium based on visual tracking, which solves the problems in related technologies such as the inability of eye-tracking technology to adapt to the user's multi-angle rotation and the low fun of vehicle-machine interaction, thereby increasing the fun of vehicle-machine interaction, improving the visual interaction experience, and enhancing user comfort.

[0005] The first aspect of this application provides a holographic projection interaction method based on visual tracking, comprising the following steps: detecting the current driving state of the vehicle; determining the working state and holographic projection position of multiple rotating cameras and / or multiple eye-tracking cameras according to the driving state, and when the multiple rotating cameras are in an enabled state, using the multiple rotating cameras to collect the user's facial rotation angle; determining at least one target eye-tracking camera according to the facial rotation angle and the working state of the multiple eye-tracking cameras, acquiring camera data collected by the at least one target eye-tracking camera, and performing holographic projection based on the position of the holographic projection and the camera data.

[0006] Based on the above technical means, the problems of eye-tracking technology being unable to adapt to users' multi-angle rotation and low fun of vehicle-machine interaction have been solved, thereby increasing the fun of vehicle-machine interaction, improving the visual interaction experience, and enhancing user comfort.

[0007] Furthermore, the driving state includes at least one of driving state, waiting state, and leisure state.

[0008] Based on the aforementioned technical means, different visual tracking and holographic projection methods are used for different driving states to enhance the user's interactive experience in different scenarios.

[0009] Furthermore, determining the working status of multiple rotating cameras and / or multiple eye-tracking cameras and the position information of the holographic projection based on the driving state includes: if the driving state is the driving state, then controlling the working state of both the rotating cameras and the eye-tracking cameras to be disabled; if the driving state is the waiting state, then controlling the working state of the rotating cameras to be disabled, and controlling some of the eye-tracking cameras to be disabled, while the remaining ones are enabled; if the driving state is the leisure state, then controlling the working state of both the rotating cameras and the eye-tracking cameras to be enabled.

[0010] Based on the aforementioned technical means, different visual tracking methods are adopted according to different driving states to ensure that there is no interference while driving, limited participation while waiting, and a full-range experience while entertaining, thus ensuring that the vehicle's primary attribute as a means of transportation is further enhanced by its entertainment attributes.

[0011] Furthermore, the plurality of eye-tracking cameras includes a first to a sixth eye-tracking camera. The step of determining at least one target eye-tracking camera based on the facial rotation angle and the operating state of the plurality of eye-tracking cameras includes: if the facial rotation angle is within a first preset angle range, then the at least one target eye-tracking camera is the first eye-tracking camera; if the facial rotation angle is within a second preset angle range, then the at least one target eye-tracking camera is the second eye-tracking camera; if the facial rotation angle is within a third preset angle range, then the at least one target eye-tracking camera is both a third and a fourth eye-tracking camera; if the facial rotation angle is within a fourth preset angle range, then the at least one target eye-tracking camera is the fifth eye-tracking camera; and if the facial rotation angle is within a fifth preset angle range, then the at least one target eye-tracking camera is the sixth eye-tracking camera.

[0012] Based on the aforementioned technical means, different eye-tracking cameras are activated according to the facial rotation angle, reducing the cost of calculating the holographic projection position and improving accuracy.

[0013] Furthermore, after determining the working status and holographic projection position of multiple rotating cameras and / or multiple eye-tracking cameras based on the driving state, the method further includes: if all of the multiple rotating cameras are in the disabled state, then determining the at least one target eye-tracking camera from the multiple eye-tracking cameras based on a preset viewing angle range.

[0014] Based on the aforementioned technical means, when multiple rotating cameras are disabled, an eye-tracking camera is activated to calculate the driver's eye angle according to the driver's perspective.

[0015] Furthermore, the holographic projection based on the position of the holographic projection and the camera data includes: converting the camera data to a local coordinate system and obtaining the user's gaze focus in the local coordinate system; adjusting the position of the holographic projection to the optimal position according to the gaze focus, and determining the current projection scene combination based on preset business logic, wherein the current projection scene combination includes projection content; and performing holographic projection based on the projection content and the optimal position.

[0016] Based on the aforementioned technical means, the optimal position for holographic projection is determined according to the focal point of the user's gaze, thereby improving the user's visual experience.

[0017] Furthermore, after performing holographic projection based on the position of the holographic projection and the camera data, the method further includes: determining the current wake-up mode of the holographic projection; if the current wake-up mode is a passive wake-up mode, then matching a target scene according to the urgency of the current wake-up event and switching the current scene of the holographic projection to the target scene; if the current wake-up mode is an active wake-up mode, then determining the target scene according to the user's current communication method and switching the current scene of the holographic projection to the target scene.

[0018] Based on the aforementioned technical means, the human-computer interaction experience can be enhanced by setting up different holographic projection scenarios.

[0019] Furthermore, before detecting the driving status of the current vehicle, the method further includes: initializing the local coordinate system, wherein the local coordinate system uses the vehicle body as a reference system; determining whether the current vehicle is starting for the first time or whether the driver's seat has been adjusted; if the current vehicle is starting for the first time or the driver's seat has been adjusted, establishing a viewing coordinate system with the adjusted driver's seat as the origin; and mapping the coordinates of the rotating camera and the eye-tracking camera in the local coordinate system to the viewing coordinate system based on the position of the adjusted driver's seat.

[0020] Based on the above technical means, the calculation logic is simplified and the calculation cost is saved by establishing a view coordinate system and a local coordinate system.

[0021] A second aspect of this application provides a vision-tracking-based holographic projection interactive device, comprising: a detection module for detecting the current driving state of a vehicle; a data acquisition module for determining the working state and holographic projection position of a plurality of rotating cameras and / or a plurality of eye-tracking cameras based on the driving state, and acquiring the user's facial rotation angle using the plurality of rotating cameras when the plurality of rotating cameras are in an enabled state; and a projection module for determining at least one target eye-tracking camera based on the facial rotation angle and the working state of the plurality of eye-tracking cameras, acquiring camera data acquired by the at least one target eye-tracking camera, and performing holographic projection based on the holographic projection position and the camera data.

[0022] Furthermore, the driving state includes at least one of driving state, waiting state, and leisure state.

[0023] Furthermore, the acquisition module is also configured to: if the driving state is the driving state, then control the working states of the rotating camera and the eye-tracking camera to be both disabled; if the driving state is the waiting state, then control the working state of the rotating camera to be disabled, and control some of the eye-tracking cameras to be disabled, while the remaining ones are enabled; if the driving state is the leisure state, then control the working states of the rotating camera and the eye-tracking camera to be both enabled.

[0024] Furthermore, the plurality of eye-tracking cameras includes a first to a sixth eye-tracking camera. The projection module, which determines at least one target eye-tracking camera based on the facial rotation angle and the operating state of the plurality of eye-tracking cameras, is further configured to: if the facial rotation angle is within a first preset angle range, then the at least one target eye-tracking camera is the first eye-tracking camera; if the facial rotation angle is within a second preset angle range, then the at least one target eye-tracking camera is the second eye-tracking camera; if the facial rotation angle is within a third preset angle range, then the at least one target eye-tracking camera is both a third and a fourth eye-tracking camera; if the facial rotation angle is within a fourth preset angle range, then the at least one target eye-tracking camera is the fifth eye-tracking camera; and if the facial rotation angle is within a fifth preset angle range, then the at least one target eye-tracking camera is the sixth eye-tracking camera.

[0025] Furthermore, after determining the working status of multiple rotating cameras and / or multiple eye-tracking cameras and the position of the holographic projection based on the driving state, the projection module is further configured to: if all of the multiple rotating cameras are in the disabled state, then determine the at least one target eye-tracking camera from the multiple eye-tracking cameras based on a preset viewing angle range.

[0026] Furthermore, the projection module is also used to: convert the camera data to a local coordinate system, obtain the user's gaze focus in the local coordinate system; adjust the position of the holographic projection to the optimal position according to the gaze focus, and determine the current projection scene combination based on preset business logic, wherein the current projection scene combination includes projection content; and perform holographic projection based on the projection content and the optimal position.

[0027] Furthermore, after performing holographic projection based on the position of the holographic projection and the camera data, the projection module is further configured to: determine the current wake-up method of the holographic projection; if the current wake-up method is a passive wake-up method, match a target scene according to the urgency of the current wake-up event, and switch the current scene of the holographic projection to the target scene; if the current wake-up method is an active wake-up method, determine the target scene according to the user's current communication method, and switch the current scene of the holographic projection to the target scene.

[0028] Furthermore, before detecting the driving status of the current vehicle, the detection module is also used to: initialize the local coordinate system, wherein the local coordinate system uses the vehicle body as a reference system; determine whether the current vehicle is starting for the first time or whether the driver's seat has been adjusted; if the current vehicle is starting for the first time or the driver's seat has been adjusted, establish a viewing coordinate system with the adjusted driver's seat as the origin; and map the coordinates of the rotating camera and the eye-tracking camera in the local coordinate system to the viewing coordinate system based on the position of the adjusted driver's seat.

[0029] A third aspect of this application provides a vehicle comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the vision-tracking-based holographic projection interaction method as described in the above embodiments.

[0030] A fourth aspect of this application provides a computer-readable storage medium having a computer program stored thereon, which is executed by a processor to implement the vision-tracking-based holographic projection interaction method as described in the above embodiments.

[0031] Therefore, this application determines the working status of multiple rotating cameras and / or multiple eye-tracking cameras and the position of the holographic projection based on the driving state. When the multiple rotating cameras are enabled, they collect the user's facial rotation angle. Based on the facial rotation angle and the working status of the multiple eye-tracking cameras, at least one target eye-tracking camera is determined, and the camera data collected by the at least one target eye-tracking camera is acquired. Holographic projection is then performed based on the position of the holographic projection and the camera data. This solves the problems of eye-tracking technology in related technologies being unable to adapt to the user's multi-angle rotation and having low interactive fun in vehicle systems, thus increasing the fun of vehicle system interaction, improving the visual interaction experience, and enhancing user comfort.

[0032] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0033] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:

[0034] Figure 1 This is a schematic diagram illustrating communication between visual tracking-based holographic projection functional units according to an embodiment of this application;

[0035] Figure 2 This is a flowchart of a holographic projection interaction method based on visual tracking provided in an embodiment of this application;

[0036] Figure 3 This is a schematic diagram of a local coordinate system and a visual coordinate system according to an embodiment of this application;

[0037] Figure 4 This is a schematic diagram illustrating the effective range of visual tracking of waiting status according to an embodiment of this application;

[0038] Figure 5 This is a schematic diagram illustrating the transition of driving states according to an embodiment of this application;

[0039] Figure 6 This is a schematic diagram of the arrangement of infrared cameras for view positioning according to an embodiment of this application;

[0040] Figure 7 This is a stereoscopic mounting diagram of an eye-tracking camera according to an embodiment of this application;

[0041] Figure 8 This is a schematic diagram of a multi-functional eye-tracking camera cooperation scheme according to an embodiment of this application;

[0042] Figure 9This is a schematic diagram of projection point positioning according to an embodiment of this application;

[0043] Figure 10 This is a schematic diagram illustrating the association of a holographic projection scene according to an embodiment of this application;

[0044] Figure 11 This is a flowchart of a vision-tracking-based holographic projection interaction method according to a specific embodiment of this application;

[0045] Figure 12 This is a block diagram of a vision-tracking-based holographic projection interactive device according to an embodiment of this application;

[0046] Figure 13 This is a structural schematic diagram of a vehicle according to an embodiment of this application.

[0047] Explanation of reference numerals in the attached figures: 10- Holographic projection interactive device based on visual tracking, 100- Detection module, 200- Acquisition module, 300- Projection module, 1301- Memory, 1302- Processor, 1303- Communication interface. Detailed Implementation

[0048] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.

[0049] The following description, with reference to the accompanying drawings, describes a vision-tracking-based holographic projection interaction method, apparatus, vehicle, and medium according to embodiments of this application. Addressing the issues mentioned in the background section regarding the inability of eye-tracking technology to adapt to multi-angle user rotations and the low level of engagement in vehicle-to-everything (V2X) interaction, this application provides a vision-tracking-based holographic projection interaction method. In this method, the working states and holographic projection positions of multiple rotating cameras and / or multiple eye-tracking cameras are determined based on the driving state. When the multiple rotating cameras are enabled, they are used to collect the user's facial rotation angles. Based on the facial rotation angles and the working states of the multiple eye-tracking cameras, at least one target eye-tracking camera is determined, and camera data collected by the at least one target eye-tracking camera is acquired. Holographic projection is then performed based on the holographic projection position and the camera data. This solves the problems of eye-tracking technology's inability to adapt to multi-angle user rotations and the low level of engagement in V2X interaction, increasing the engagement of V2X interaction, improving the visual interaction experience, and enhancing user comfort.

[0050] In this embodiment, the module that acquires the user's eye position information is called the visual tracking unit, the system that processes the holographic projection is called the holographic projection unit, and the vehicle-mounted system acts as an interaction bridge between the two units to manage their interaction protocol, such as... Figure 1 As shown.

[0051] Specifically, Figure 2 This is a flowchart illustrating a holographic projection interaction method based on visual tracking, provided in an embodiment of this application.

[0052] like Figure 2 As shown, this vision-tracking-based holographic projection interaction method includes the following steps:

[0053] In step S201, the current driving status of the vehicle is detected.

[0054] Driving status can include driving status, waiting status, and leisure status.

[0055] Specifically, if the current driving status is driving, then the detected current driving status of the vehicle is driving; if the current driving status is waiting, then the detected current driving status of the vehicle is waiting. To avoid redundancy, this will not be elaborated on in detail here.

[0056] Furthermore, in some embodiments, before detecting the current driving state of the vehicle, the method further includes: initializing a local coordinate system, wherein the local coordinate system uses the vehicle body as a reference system; determining whether the current vehicle is starting for the first time or whether the driver's seat has been adjusted; if the current vehicle is starting for the first time or the driver's seat has been adjusted, establishing a view coordinate system with the adjusted driver's seat as the origin; and mapping the coordinates of the rotating camera and the eye-tracking camera in the local coordinate system to the view coordinate system based on the position of the adjusted driver's seat.

[0057] Specifically, once the vehicle is manufactured, the positions of the rotating camera and eye-tracking camera used for data collection, as well as the position of the holographic projection device, are determined. This embodiment establishes a local coordinate system using the vehicle body as a reference and initializes the local coordinate system data, which includes the coordinates of all cameras, the holographic projection device, and the hardware devices within the maximum projectable range. The position of each camera in the local coordinate system is determined based on the hardware configuration. When the vehicle is first started or after the driver's seat is adjusted, a viewing coordinate system is established with the adjusted driver's seat as the origin. The coordinates of the rotating camera and eye-tracking camera in the local coordinate system are mapped to the viewing coordinate system, and a transformation matrix is ​​generated using the seat-related position information. This transformation matrix completes the initialization of the mutual conversion between the local coordinate system and the viewing coordinate system, facilitating data processing and image display positioning. The viewing coordinate system is particularly useful for calculating the user's facial rotation angle and eye angle obtained by the rotating camera and eye-tracking camera. The local coordinate system and the viewing coordinate system are as follows: Figure 3 As shown.

[0058] In this diagram, 3(a) is a top view and 3(b) is a perspective view. The outermost rectangle represents the vehicle body, and the inner squares represent the driver's seat. The origin of the viewpoint coordinate system is taken from the center of the driver's seat, and this information can be obtained through the hardware system. Since dynamically constructing the coordinate system can make angle calculations more accurate, but frequent construction will affect processing speed, the timing of establishing the coordinate system is very important. Therefore, in this embodiment, the viewpoint coordinate system is established when the vehicle is first started or when the driver's seat is adjusted.

[0059] In some embodiments, the driving state includes at least one of driving state, waiting state, and leisure state.

[0060] It should be understood that while the effective range of visual tracking is the area directly in front of and to the right of the driver, this area is also the driver's primary visual field. If it interferes with the user's vision while they are driving, it can easily disrupt the driver's driving state and cause an accident. Therefore, the effective area is dynamically determined based on the user's driving state, including driving, waiting, and leisure states.

[0061] In this context, "driving state" refers to a vehicle speed greater than zero, with the time interval between two consecutive instances of speed exceeding zero being less than 30 seconds. During driving state, visual tracking is disabled, and the holographic interactive scene is fixedly projected onto the center position in front of the driver and passenger seats, such as... Figure 4 Point A in the diagram.

[0062] Waiting mode refers to the state where, when the vehicle speed decreases to zero for more than 30 seconds, the driving state switches to waiting mode, and it remains stationary for more than 180 seconds before switching to the next state. The effective range of visual tracking in the waiting mode is as follows: Figure 4 As shown.

[0063] Figure 4 Point A represents the midpoint of the front of the car, and the white rectangle on the right represents the edge of the rearview mirror. The effective horizontal angle θ refers to the angle formed from point A to the viewer's eye, and from the viewer's eye to the front of the rearview mirror. In this state, the effective vertical angle is disabled; the holographic projection will only project onto the effective area in front of the passenger, ensuring that the rearview mirror can be viewed without obstructing the forward view. Because the vertical angle effect is eliminated, the holographic projection will not obstruct the passenger's direct line of sight.

[0064] Leisure mode refers to a driving speed of 0 for more than 180 seconds. Once a vehicle in leisure mode starts and its speed is greater than zero, it will directly switch to driving mode. Vehicles in leisure mode will fully activate the visual tracking function, maximizing the radiation range of the holographic projection.

[0065] The time thresholds for switching between the three driving states can be dynamically set, primarily because the maximum red light duration varies from city to city. The threshold for the waiting state is the average of the city's maximum and minimum red light durations, covering over 80% of scenarios. The threshold for the leisure state is the longest red light duration in the city plus 5 seconds to prevent misjudgments due to slow vehicle start-up. Figure 5 As shown.

[0066] In step S202, the working status and position of the multiple rotating cameras and / or multiple eye-tracking cameras are determined according to the driving status, and when the multiple rotating cameras are in the enabled state, the user's facial rotation angle is collected using the multiple rotating cameras.

[0067] In this embodiment, to obtain relatively accurate eye-tracking and positioning angles, the cameras are divided into two groups: one group consists of multiple rotating cameras to determine the driver's facial rotation angle, and the other group consists of multiple eye-tracking cameras to determine the driver's eye-tracking angle. Installation scheme... Figure 6 and Figure 7 As shown:

[0068] Specifically, with Figure 6 The origin of the coordinate system is the center of the seat. The triangles and dots represent infrared cameras. Multiple triangular rotating cameras are used to calculate the user's facial rotation angle. They are mounted on the roof of the vehicle. This set of cameras is used to determine the angle of the user's facial rotation when the neck is the axis, thereby determining which eye-tracking camera (black dot) is mainly used to capture the eye angle.

[0069] There are six eye-tracking cameras in total, but only five are marked in the diagram. The large dot in the upper right corner represents two eye-tracking cameras. In clockwise order, they are the first, second, third, fourth, fifth, and sixth eye-tracking cameras. There are three horizontal eye-tracking cameras (the first, second, and third) and three vertical eye-tracking cameras (the fourth, fifth, and sixth). The fourth eye-tracking camera is installed in the rearview mirror area inside the vehicle, corresponding vertically to the third. The three horizontal cameras are installed at the same level on the console directly in front of the driver. The three vertical cameras are installed in the roof, between the driver and passenger seats. During installation, the first, third, and user should ideally form an equilateral right triangle in the horizontal direction; the fourth, sixth, and user should also form an equilateral right triangle.

[0070] The data acquired by the third and fourth eye-tracking cameras serves two main purposes:

[0071] (1) When the view camera fails to acquire data, the data acquired by the third eye-tracking camera and the fourth eye-tracking camera, combined with the data from the body rotation camera, can be used to infer the driver's face orientation and further infer the visual direction, assuming that the gaze direction is directly in front of the face.

[0072] (2) When the view camera successfully acquires data, the data acquired by the third eye-tracking camera and the fourth eye-tracking camera are combined with the rotating camera to verify the driver's face in order to exclude the situation where the user is wearing a mask.

[0073] Furthermore, in some embodiments, the working status of multiple rotating cameras and / or multiple eye-tracking cameras and the position information of the holographic projection are determined according to the driving state, including: if the driving state is a driving state, then the working state of both the rotating cameras and the eye-tracking cameras is controlled to be disabled; if the driving state is a waiting state, then the working state of the rotating cameras is controlled to be disabled, and some of the eye-tracking cameras are controlled to be disabled, while the remaining ones are enabled; if the driving state is a leisure state, then the working state of both the rotating cameras and the eye-tracking cameras is controlled to be enabled.

[0074] In some embodiments, after determining the working status of multiple rotating cameras and / or multiple eye-tracking cameras and the position of the holographic projection based on the driving state, the method further includes: if all multiple rotating cameras are disabled, then determining at least one target eye-tracking camera from the multiple eye-tracking cameras based on a preset viewing angle range.

[0075] Understandably, if the vehicle is in driving mode, the swivel camera and eye-tracking camera are disabled, and the holographic image is projected onto a fixed position, such as... Figure 4 Point A in the diagram.

[0076] When the driving mode is switched to waiting mode, the body-rotating camera is disabled, and only the second, third, and fourth eye-tracking cameras are activated to collect data. The remaining cameras are disabled, and the collected data is processed in the view coordinate system to determine whether it is in the waiting state. Figure 4 If the collected data is outside the effective range of the marked viewpoint, the data is discarded.

[0077] When the driving mode is in a leisure state, the body rotation camera and eye-tracking camera are activated.

[0078] It should be noted that the effective viewing angle θ when waiting for a train is as follows: Figure 4 As shown, the area is defined by three points: the horizontal midpoint A, the leading edge of the right rearview mirror, and the user's seat. Point A and the leading edge of the rearview mirror define a horizontal range along the vehicle's horizontal direction. Combined with the width of the front dashboard, this results in a rectangular plane, which is the area where the holographic projection can be displayed in the waiting state. Since the vehicle's hardware information is unlikely to change easily, the coordinate data of the boundary points of this area can be persistently processed.

[0079] After calculating the user's eye spatial angle based on the eye-tracking camera, and combining it with the current persistent data, the user's visual focus is calculated. Then, combined with the width and height of the holographic projection image, the center of gravity of the gaze point is finally determined. If the x and y coordinates of the center of gravity are projected within the rectangular area, it means that the data collected by the eye-tracking camera is valid; if they are not within the rectangular area, it means that the data collected by the eye-tracking camera is invalid.

[0080] In step S203, at least one target eye-tracking camera is determined based on the facial rotation angle and the working status of multiple eye-tracking cameras, and camera data collected by at least one target eye-tracking camera is acquired. Holographic projection is then performed based on the position of the holographic projection and the camera data.

[0081] In some embodiments, the plurality of eye-tracking cameras includes first to sixth eye-tracking cameras. At least one target eye-tracking camera is determined based on the facial rotation angle and the operating status of the plurality of eye-tracking cameras, including: if the facial rotation angle is within a first preset angle range, then at least one target eye-tracking camera is the first eye-tracking camera; if the facial rotation angle is within a second preset angle range, then at least one target eye-tracking camera is the second eye-tracking camera; if the facial rotation angle is within a third preset angle range, then at least one target eye-tracking camera is both the third and fourth eye-tracking cameras; if the facial rotation angle is within a fourth preset angle range, then at least one target eye-tracking camera is the fifth eye-tracking camera; and if the facial rotation angle is within a fifth preset angle range, then at least one target eye-tracking camera is the sixth eye-tracking camera.

[0082] It should be understood that the embodiments of this application use multi-functional cameras, which means refining the functions of the cameras and activating only the camera with the highest data accuracy in different scenarios for calculation, thereby improving accuracy and efficiency. Specific solutions are as follows: Figure 8 As shown.

[0083] The embodiments of this application are as follows: Figure 6 As shown, the left horizontal direction is set to 0 degrees. When the user's face rotation angle is within the range of 30°-120°, the rotating camera directly in front of the user (i.e., rotating camera A and rotating camera B) determines the use of at least one target eye-tracking camera.

[0084] (1) When the facial rotation angle is between 30° and 67.5° (i.e., the first preset angle range), the first eye-tracking camera is the primary camera.

[0085] (2) When the facial rotation angle is between 67.5° and 112.5° (i.e., the second preset angle range), the second eye-tracking camera is the primary camera.

[0086] (3) When the facial rotation angle is between 112.5° and 157.5° (i.e., the third preset angle range), the third eye-tracking camera and the fourth eye-tracking camera are the main focus;

[0087] Among them, the third and fourth eye-tracking cameras have the function of calculating the user's head-up and head-down angles, and remain operational even when not used as the primary eye-tracking cameras.

[0088] When the user's facial rotation angle is within the range of 150°-240°, it is determined by the right-side rotating camera using at least one target eye-tracking camera.

[0089] (1) When the facial rotation angle is between 112.5° and 157.5°, the third eye-tracking camera and the fourth eye-tracking camera are the main focus;

[0090] (2) When the facial rotation angle is between 157.5° and 202.5° (i.e., the fourth preset angle range), the fifth eye-tracking camera is used as the primary camera.

[0091] (3) When the facial rotation angle is between 202.5° and 240° (i.e., the fifth preset angle range), the sixth eye-tracking camera will be the primary camera.

[0092] The range of 120° to 150° is the tolerance range, which aims to solve the angle error caused by the user's twisting. The rotation angle within this range is determined by taking the average of the angles measured by the two rotating cameras as the face rotation angle.

[0093] Furthermore, in some embodiments, holographic projection based on the position of the holographic projection and camera data includes: converting the camera data to a local coordinate system and obtaining the user's gaze focus in the local coordinate system; adjusting the position of the holographic projection to the optimal position according to the gaze focus, and determining the current projection scene combination based on preset business logic, wherein the current projection scene combination includes the projection content; and performing holographic projection based on the projection content and the optimal position.

[0094] Specifically, this embodiment uses a rotating camera to assign weights to each eye-tracking camera based on the user's facial rotation angle. Data from the eye-tracking camera with the highest weight is prioritized, while data from adjacent cameras is verified. The third and fourth eye-tracking cameras also measure the angle at which the user's face is lifted. This data assists the viewpoint tracking camera or functions when the viewpoint camera is not collecting data, so continuous data acquisition is necessary. Whether the holographic projection is fixed or based on eye tracking can be configured via a system switch.

[0095] After calculating the user's pitch and rotation angles, it is necessary to switch from the viewpoint coordinate system to the local coordinate system. Then, combining the user's interpupillary distance and the plane formed by various persistent coordinates in the local coordinate system, the user's gaze focus is calculated. After obtaining the gaze focus, the position of the holographic projection is rationally moved to further determine the optimal position for the final display of the holographic projection, such as... Figure 9 As shown in the figure. The pupil distance is obtained when each camera is first activated. Before each visual tracking, the pupil distance of the user is checked. If the pupil distance obtained in two consecutive measurements is less than 0.1cm, no further checks will be performed, and the data will be considered accurate until the next time the vehicle is started.

[0096] It's important to note that rationalized movement is used to address the boundary effect. When viewing an object, the image seen at the visual focus is clear, while the image within a certain range S (this value varies depending on individual visual acuity) outside the focus is blurred. These two ranges together are called the "visible range." When the calculated visual focus falls precisely near the boundary of the effective area, limited movement can be used to ensure the holographic projection is displayed within both the effective area and the user's visible range. Once the image exceeds the visual focus by a certain range S, no movement is made, and the projection is canceled.

[0097] Furthermore, in this embodiment of the application, based on preset business logic, holographic projection is recorded according to scene categories, and the relationships between scenes are as follows: Figure 10 As shown, the scene represents a series of 3D animations. The target projection needs to have specific animation effects customized according to different business needs, so this scene represents a series of scenes. The switching between different scenes can be unified with the same sci-fi motion effect, or different animation effects can be customized according to specific business needs.

[0098] Specifically, the dynamic projection materials required for holographic projection can be generated in advance, or users can be guided to record them to generate dynamic material files. When replacing them locally, the application scenario of the dynamic material needs to be specified to achieve a personalized experience. For pre-recorded dynamic materials, the following factors need to be considered:

[0099] (1) Virtual character synchronization requires switching to different holographic projection characters in real time according to the switching of virtual characters in the voice dialogue;

[0100] (2) Emotional changes: Holographic projection should be synchronized with emotional factors and expressed through different holographic facial expressions and body movements.

[0101] (3) Switch virtual images according to actual application scenarios. They are not limited to people, but can be anything. For example, they can become a microphone when singing karaoke or a book when reading e-books.

[0102] (4) Implement transitions between different states within the same scene. Refine the interactive scene and define the scene as the smallest unit for animation playback. When switching between different scenes or between different states within a single scene, preset the required transition animations and play them according to different scene changes.

[0103] Furthermore, in some embodiments, after performing holographic projection based on the location of the holographic projection and camera data, the method further includes: determining the current wake-up method of the holographic projection; if the current wake-up method is a passive wake-up method, then matching the target scene according to the urgency of the current wake-up event and switching the current scene of the holographic projection to the target scene; if the current wake-up method is an active wake-up method, then determining the target scene according to the user's current communication method and switching the current scene of the holographic projection to the target scene.

[0104] The wake-up methods for holographic projection include passive wake-up and active wake-up.

[0105] Specifically, such as Figure 10 As shown, if the holographic projection is currently awakened passively, it's difficult to know the vehicle's current status when the infotainment system is not activated. The target scenario is matched based on the urgency of the wake-up event. For urgent wake-up events, the infotainment system doesn't perform any pre-emptive actions; it directly alerts the user via voice, while the holographic projection's movements are amplified, such as jumping, to attract the user's attention. For situations where the user only needs to be informed, but the event is not urgent, a greeting-like gesture, such as waving, can be used.

[0106] If the current wake-up method for the holographic projection is active wake-up, the interaction consists of the active user and the passive vehicle system. The vehicle system is the active communicator, and the driver is the passive wake-up party. The vehicle system interrupts the driver's actions, and the holographic projection can vividly and intuitively display the current state. Using big data intelligence, it selects wake-up events with higher weight that the user may be interested in, such as weather conditions and traffic congestion, and switches the current scene of the holographic projection to the target scene.

[0107] Holographic projection scenes also include casual conversation scenes, question-and-answer scenes, character switching scenes, and exit scenes.

[0108] In the casual conversation scene, three body animations are prepared for the holographic projection character: thinking, answering, and not knowing. Transition animations are also prepared for switching between the two states: thinking-answering and thinking-not knowing. When a state switch occurs, the transition animation plays first, followed by the animation for the corresponding state. The casual conversation scene also needs to add emotional animations such as angry, happy, gentle, mischievous, and spoiled, and play different transition animations for switching between different emotions.

[0109] When the holographic projection character changes, a unified transition animation, reminiscent of transformations in mythology, can be used. When two characters with significantly different appearances (such as a microphone or a book) switch, it often indicates a change in behavior between different functions. Adding a transition animation at this point would make it difficult to control the timing and handle combined states between different applications. Therefore, a strategy of instantaneous character transformation is adopted, skipping the transition animation and directly switching to the desired state for the user.

[0110] Exiting a scenario requires determining two exit states based on user commands. The first exit state is a complete exit, hiding the holographic projection until the vehicle's infotainment system is reactivated, at which point the holographic projection will reappear. The second exit state is switching the current holographic projection scenario to a passively activated scenario. The exit animation needs to be differentiated according to different business scenarios; for example, exiting an e-book reading scenario involves closing the book, while exiting a singing scenario involves retracting the microphone.

[0111] It should be noted that the actual scenario division will be further increased or decreased as user needs evolve.

[0112] To enable those skilled in the art to further understand the vision-tracking-based holographic projection interaction method of the embodiments of this application, the following detailed description is provided in conjunction with specific embodiments, such as... Figure 11 As shown.

[0113] In step S1101, a local coordinate system is preset with the vehicle body as the reference frame.

[0114] In step S1102, when the user starts or adjusts the seat for the first time, the system will establish a view coordinate system with the seat as the origin.

[0115] In step S1103, the eye-tracking camera is enabled or disabled based on the current driving status.

[0116] In step S1104, when the driving state is driving state, the eye-tracking camera is disabled, and the holographic projection is displayed in the middle position between the front of the driver's seat and the front passenger seat in the local coordinate system.

[0117] In step S1105, when the driving state is a leisure state, the eye-tracking camera is activated, and the camera data within the effective range is acquired, and then step S1107 is executed.

[0118] In step S1106, when the driving state is the waiting state, some eye-tracking cameras are activated and camera data within the effective range is acquired. It is then determined whether the acquired camera data is within the effective range. If it is not within the effective range, the camera data is discarded. If it is within the effective range, step S1107 is executed.

[0119] In step S1107, the view coordinate system data is converted into local coordinate system data.

[0120] In step S1108, the view focus in the local coordinate system is obtained.

[0121] In step S1109, the holographic projection is displayed in the local coordinate system according to the business logic.

[0122] The beneficial effects of the embodiments of this application are as follows:

[0123] (1) The dual-coordinate system calculation method used in the embodiments of this application simplifies the calculation logic and saves calculation costs by establishing a view coordinate system and a local coordinate system;

[0124] (2) The embodiments of this application enhance the fun of human-computer interaction by combining eye tracking and holographic projection.

[0125] (3) The holographic projection in this embodiment uses dynamic material playback with the scene as the smallest dimension to enhance the entertainment attributes;

[0126] (4) According to the embodiments of this application, different visual tracking response areas and processing logics are adopted for three driving states to distinguish the degree of visual participation, so as to ensure that there is no interference while driving, limited participation while waiting for a bus, and a full-range experience while entertaining.

[0127] (5) The embodiments of this application demonstrate that due to the limitation of the projection area during holographic projection, the movement is rationalized at the critical projection point to improve the user's visual enjoyment;

[0128] (6) The multi-functional camera cooperation and division of labor used in the embodiments of this application reduces computing costs and improves data accuracy;

[0129] (7) The source of materials is extensive, and personalized content can be achieved through personal recording. Holographic projection can create unique material libraries by changing the source of the projection materials (such as pre-recorded, real-time recorded, algorithm-generated, etc.) and apply them to different scenarios. The material library has strong portability and high reusability.

[0130] The holographic projection interaction method based on vision tracking proposed in this application determines the working status and holographic projection position of multiple rotating cameras and / or multiple eye-tracking cameras according to the driving state. When the multiple rotating cameras are enabled, they are used to collect the user's facial rotation angle. Based on the facial rotation angle and the working status of the multiple eye-tracking cameras, at least one target eye-tracking camera is determined, and the camera data collected by the at least one target eye-tracking camera is acquired. Holographic projection is then performed based on the holographic projection position and the camera data. This solves the problems of eye-tracking technology in related technologies being unable to adapt to the user's multi-angle rotation and having low interactive fun. It retains the advantages of voice interaction and hands-free operation while adding visual enjoyment. Through multi-level customization, it enhances the user's interactive experience in different scenarios, ensuring that the vehicle, while primarily a means of transportation, further enhances its entertainment attributes.

[0131] Next, referring to the accompanying drawings, a vision-tracking-based holographic projection interactive device according to an embodiment of this application is described.

[0132] Figure 12 This is a block diagram of a holographic projection interactive device based on visual tracking, according to an embodiment of this application.

[0133] like Figure 12 As shown, the vision-tracking-based holographic projection interactive device 10 includes: a detection module 100, a data acquisition module 200, and a projection module 300.

[0134] The system includes a detection module 100 for detecting the current driving status of the vehicle; a data acquisition module 200 for determining the working status and holographic projection position of multiple rotating cameras and / or multiple eye-tracking cameras based on the driving status, and for acquiring the user's facial rotation angle using multiple rotating cameras when they are enabled; and a projection module 300 for determining at least one target eye-tracking camera based on the facial rotation angle and the working status of multiple eye-tracking cameras, acquiring camera data acquired by at least one target eye-tracking camera, and performing holographic projection based on the holographic projection position and the camera data.

[0135] Furthermore, in some embodiments, the driving state includes at least one of driving state, waiting state, and leisure state.

[0136] Furthermore, in some embodiments, the acquisition module 200 is also used to: if the driving state is a driving state, control the working state of both the rotating camera and the eye-tracking camera to be disabled; if the driving state is a waiting state, control the working state of the rotating camera to be disabled, and control some of the eye-tracking cameras to be disabled, while the remaining ones are enabled; if the driving state is a leisure state, control the working state of both the rotating camera and the eye-tracking camera to be enabled.

[0137] Furthermore, in some embodiments, the plurality of eye-tracking cameras includes first to sixth eye-tracking cameras. At least one target eye-tracking camera is determined based on the facial rotation angle and the operating status of the plurality of eye-tracking cameras. The projection module 300 is further configured to: if the facial rotation angle is within a first preset angle range, then at least one target eye-tracking camera is the first eye-tracking camera; if the facial rotation angle is within a second preset angle range, then at least one target eye-tracking camera is the second eye-tracking camera; if the facial rotation angle is within a third preset angle range, then at least one target eye-tracking camera is both the third and fourth eye-tracking cameras; if the facial rotation angle is within a fourth preset angle range, then at least one target eye-tracking camera is the fifth eye-tracking camera; and if the facial rotation angle is within a fifth preset angle range, then at least one target eye-tracking camera is the sixth eye-tracking camera.

[0138] Furthermore, in some embodiments, after determining the working status of multiple rotating cameras and / or multiple eye-tracking cameras and the position of the holographic projection based on the driving state, the projection module 300 is further configured to: if all multiple rotating cameras are disabled, determine at least one target eye-tracking camera from the multiple eye-tracking cameras based on a preset viewing angle range.

[0139] Furthermore, in some embodiments, the projection module 300 is also used to: convert camera data to a local coordinate system, obtain the user's gaze focus in the local coordinate system; adjust the position of the holographic projection to the optimal position according to the gaze focus, and determine the current projection scene combination based on preset business logic, wherein the current projection scene combination includes projection content; and perform holographic projection based on the projection content and the optimal position.

[0140] Furthermore, in some embodiments, after holographic projection is performed based on the location of the holographic projection and camera data, the projection module 300 is further configured to: determine the current wake-up method of the holographic projection; if the current wake-up method is a passive wake-up method, match the target scene according to the urgency of the current wake-up event, and switch the current scene of the holographic projection to the target scene; if the current wake-up method is an active wake-up method, determine the target scene according to the user's current communication method, and switch the current scene of the holographic projection to the target scene.

[0141] Furthermore, in some embodiments, before detecting the current driving state of the vehicle, the detection module 100 is also used to: initialize a local coordinate system, wherein the local coordinate system is referenced to the vehicle body; determine whether the current vehicle is starting for the first time or whether the driver's seat has been adjusted; if the current vehicle is starting for the first time or the driver's seat has been adjusted, establish a viewing coordinate system with the adjusted driver's seat as the origin; and map the coordinates of the rotating camera and the eye-tracking camera in the local coordinate system to the viewing coordinate system based on the position of the adjusted driver's seat.

[0142] It should be noted that the foregoing explanation of the embodiment of the vision tracking-based holographic projection interaction method also applies to the vision tracking-based holographic projection interaction device of this embodiment, and will not be repeated here.

[0143] The holographic projection interaction device based on vision tracking proposed in this application determines the working status and holographic projection position of multiple rotating cameras and / or multiple eye-tracking cameras according to the driving state. When the multiple rotating cameras are enabled, they collect the user's facial rotation angle. Based on the facial rotation angle and the working status of the multiple eye-tracking cameras, at least one target eye-tracking camera is determined, and the camera data collected by the at least one target eye-tracking camera is acquired. Holographic projection is then performed based on the holographic projection position and the camera data. This solves the problems of eye-tracking technology in related technologies being unable to adapt to the user's multi-angle rotation and having low interactive fun in vehicles, thus increasing the fun of vehicle-machine interaction, improving the visual interaction experience, and enhancing user comfort.

[0144] Figure 13 A schematic diagram of the structure of a vehicle provided in an embodiment of this application. The vehicle may include:

[0145] The memory 1301, the processor 1302, and the computer program stored on the memory 1301 and executable on the processor 1302.

[0146] When the processor 1302 executes the program, it implements the holographic projection interaction method based on visual tracking provided in the above embodiments.

[0147] Furthermore, the vehicle also includes:

[0148] Communication interface 1303 is used for communication between memory 1301 and processor 1302.

[0149] The memory 1301 is used to store computer programs that can run on the processor 1302.

[0150] The memory 1301 may include high-speed RAM (Random Access Memory) memory, and may also include non-volatile memory, such as at least one disk storage.

[0151] If the memory 1301, processor 1302, and communication interface 1303 are implemented independently, then the communication interface 1303, memory 1301, and processor 1302 can be interconnected via a bus to complete communication between them. The bus can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus, etc. The bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 13 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0152] Optionally, in a specific implementation, if the memory 1301, processor 1302, and communication interface 1303 are integrated on a single chip, then the memory 1301, processor 1302, and communication interface 1303 can communicate with each other through an internal interface.

[0153] The processor 1302 may be a CPU (Central Processing Unit), an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of this application.

[0154] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the above-described holographic projection interaction method based on visual tracking.

[0155] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0156] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0157] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or N executable instructions for implementing custom logic functions or processes, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of this application pertain.

[0158] It should be understood that the various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, the N steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (FPGAs), field-programmable gate arrays (FPGAs), etc.

[0159] Those skilled in the art will understand that all or part of the steps of the methods described in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it includes one or a combination of the steps of the method embodiments.

[0160] Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of this application.

Claims

1. A holographic projection interaction method based on visual tracking, characterized in that, Includes the following steps: Detect the current driving status of the vehicle; The working status and position of the holographic projection of multiple rotating cameras and / or multiple eye-tracking cameras are determined based on the driving state, and when the multiple rotating cameras are in the enabled state, the user's facial rotation angle is collected using the multiple rotating cameras; as well as At least one target eye-tracking camera is determined based on the facial rotation angle and the working status of the plurality of eye-tracking cameras, and camera data collected by the at least one target eye-tracking camera is acquired. Holographic projection is then performed based on the position of the holographic projection and the camera data.

2. The method according to claim 1, characterized in that, The driving state includes at least one of driving state, waiting state, and leisure state.

3. The method according to claim 2, characterized in that, Determining the working status of multiple rotating cameras and / or multiple eye-tracking cameras and the position information of the holographic projection based on the driving state includes: If the driving state is the vehicle driving state, then the working states of the rotating camera and the eye-tracking camera are both disabled. If the driving state is the waiting state, then the working state of the rotating camera is controlled to the disabled state, and some of the eye-tracking cameras are controlled to the disabled state, while the remaining ones are controlled to the enabled state; If the driving state is the leisure state, then the working state of the rotating camera and the eye-tracking camera is both the enabled state.

4. The method according to claim 1, characterized in that, The plurality of eye-tracking cameras includes a first to a sixth eye-tracking camera. The step of determining at least one target eye-tracking camera based on the facial rotation angle and the operating status of the plurality of eye-tracking cameras includes: If the facial rotation angle is within a first preset angle range, then the at least one target eye-tracking camera is a first eye-tracking camera; If the facial rotation angle is within a second preset angle range, then the at least one target eye-tracking camera is a second eye-tracking camera; If the facial rotation angle is within a third preset angle range, then the at least one target eye-tracking camera is a third eye-tracking camera and a fourth eye-tracking camera; If the facial rotation angle is within the fourth preset angle range, then the at least one target eye-tracking camera is the fifth eye-tracking camera; If the facial rotation angle is within the fifth preset angle range, then the at least one target eye-tracking camera is the sixth eye-tracking camera.

5. The method according to claim 3, characterized in that, After determining the operating status of multiple rotating cameras and / or multiple eye-tracking cameras and the position of the holographic projection based on the driving state, the method further includes: If all of the multiple rotating cameras are in the disabled state, then based on a preset field of view, at least one target eye-tracking camera is determined from the multiple eye-tracking cameras.

6. The method according to claim 1 or 5, characterized in that, The holographic projection based on the position of the holographic projection and the camera data includes: The camera data is converted to a local coordinate system to obtain the user's gaze focus in the local coordinate system. The position of the holographic projection is adjusted to the optimal position according to the focal point of the gaze, and the current projection scene combination is determined based on the preset business logic, wherein the current projection scene combination includes the projection content; Holographic projection is performed based on the projected content and the optimal position.

7. The method according to claim 6, characterized in that, After performing holographic projection based on the position of the holographic projection and the camera data, the method further includes: Determine the current wake-up mode of the holographic projection; If the current wake-up method is a passive wake-up method, then the target scene is matched according to the urgency of the current wake-up event, and the current scene of the holographic projection is switched to the target scene; If the current wake-up method is an active wake-up method, then the target scene is determined according to the user's current communication method, and the current scene of the holographic projection is switched to the target scene.

8. The method according to claim 6, characterized in that, Before detecting the current driving status of the vehicle, the method also includes: Initialize the local coordinate system, wherein the local coordinate system is referenced to the body of the current vehicle; Determine whether the current vehicle is starting for the first time or whether the driver's seat has been adjusted; If the current vehicle is starting for the first time or the driver's seat is adjusted, a view coordinate system is established with the adjusted driver's seat as the origin. Based on the adjusted position of the driver's seat, the coordinates of the swivel camera and the eye-tracking camera in the local coordinate system are mapped to the view coordinate system.

9. A holographic projection interactive device based on visual tracking, characterized in that, include: The detection module is used to detect the current driving status of the vehicle; The acquisition module is used to determine the working status and holographic projection position of multiple rotating cameras and / or multiple eye-tracking cameras according to the driving state, and to acquire the user's facial rotation angle using the multiple rotating cameras when the multiple rotating cameras are in the enabled state. as well as The projection module is used to determine at least one target eye-tracking camera based on the facial rotation angle and the working status of the plurality of eye-tracking cameras, acquire camera data collected by the at least one target eye-tracking camera, and perform holographic projection based on the position of the holographic projection and the camera data.

10. The apparatus according to claim 9, characterized in that, The driving state includes at least one of driving state, waiting state, and leisure state.

11. The apparatus according to claim 10, characterized in that, The acquisition module is also used for: If the driving state is the vehicle driving state, then the working states of the rotating camera and the eye-tracking camera are both disabled. If the driving state is the waiting state, then the working state of the rotating camera is controlled to the disabled state, and some of the eye-tracking cameras are controlled to the disabled state, while the remaining ones are controlled to the enabled state; If the driving state is the leisure state, then the working state of the rotating camera and the eye-tracking camera is both the enabled state.

12. The apparatus according to claim 9, characterized in that, The plurality of eye-tracking cameras includes a first to a sixth eye-tracking camera. The step of determining at least one target eye-tracking camera based on the facial rotation angle and the operating status of the plurality of eye-tracking cameras, and the projection module, are further configured to: If the facial rotation angle is within a first preset angle range, then the at least one target eye-tracking camera is a first eye-tracking camera; If the facial rotation angle is within a second preset angle range, then the at least one target eye-tracking camera is a second eye-tracking camera; If the facial rotation angle is within a third preset angle range, then the at least one target eye-tracking camera is a third eye-tracking camera and a fourth eye-tracking camera; If the facial rotation angle is within the fourth preset angle range, then the at least one target eye-tracking camera is the fifth eye-tracking camera; If the facial rotation angle is within the fifth preset angle range, then the at least one target eye-tracking camera is the sixth eye-tracking camera.

13. The apparatus according to claim 11, characterized in that, After determining the operating status of multiple rotating cameras and / or multiple eye-tracking cameras and the position of the holographic projection based on the driving state, the projection module is further configured to: If all of the multiple rotating cameras are in the disabled state, then based on a preset field of view, at least one target eye-tracking camera is determined from the multiple eye-tracking cameras.

14. The apparatus according to claim 9 or 13, characterized in that, The projection module is also used for: The camera data is converted to a local coordinate system to obtain the user's gaze focus in the local coordinate system. The position of the holographic projection is adjusted to the optimal position according to the focal point of the gaze, and the current projection scene combination is determined based on the preset business logic, wherein the current projection scene combination includes the projection content; Holographic projection is performed based on the projected content and the optimal position.

15. The apparatus according to claim 14, characterized in that, After performing holographic projection based on the position of the holographic projection and the camera data, the projection module is further configured to: Determine the current wake-up mode of the holographic projection; If the current wake-up method is a passive wake-up method, then the target scene is matched according to the urgency of the current wake-up event, and the current scene of the holographic projection is switched to the target scene; If the current wake-up method is an active wake-up method, then the target scene is determined according to the user's current communication method, and the current scene of the holographic projection is switched to the target scene.

16. The apparatus according to claim 14, characterized in that, Before detecting the current driving status of the vehicle, the detection module is further configured to: Initialize the local coordinate system, wherein the local coordinate system is referenced to the body of the current vehicle; Determine whether the current vehicle is starting for the first time or whether the driver's seat has been adjusted; If the current vehicle is starting for the first time or the driver's seat is adjusted, a view coordinate system is established with the adjusted driver's seat as the origin. Based on the adjusted position of the driver's seat, the coordinates of the swivel camera and the eye-tracking camera in the local coordinate system are mapped to the view coordinate system.

17. A vehicle, characterized in that, include: A memory, a processor, and a computer program stored in the memory and executable on the processor, the processor executing the program to implement the visual tracking-based holographic projection interaction method as described in any one of claims 1-8.

18. A computer-readable storage medium having a computer program stored thereon, characterized in that, The program is executed by the processor to implement the visual tracking-based holographic projection interaction method as described in any one of claims 1-8.

Images