An in-vehicle augmented reality information hierarchical feedback system, method and vehicle

By combining the central control unit and the head-mounted display device, information is dynamically displayed in layers, solving the problem that AR glasses cannot adaptively adjust information, realizing intelligent layered feedback of information, and improving driving safety and user experience.

CN116483202BActive Publication Date: 2026-07-10CHERY AUTOMOBILE CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
CHERY AUTOMOBILE CO LTD
Filing Date
2023-04-24
Publication Date
2026-07-10

AI Technical Summary

Technical Problem

In existing in-vehicle augmented reality systems, the information displayed by AR glasses cannot adaptively adjust to changes in the user's field of vision, resulting in excessive information that interferes with driving, reduces user experience, and increases driving risks.

Method used

By employing a central control unit and a head-mounted display device, information is dynamically layered and displayed by acquiring eye status and focus information. The information layers are hidden or displayed according to the user's field of vision and focus point, thus achieving intelligent layered feedback of information.

Benefits of technology

The system dynamically adjusts information display in different environments to avoid information overload, reduce driving risks, and improve user experience and safety.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN116483202B_ABST
    Figure CN116483202B_ABST
Patent Text Reader

Abstract

The application belongs to the technical field of augmented reality, and provides a vehicle-mounted augmented reality information layered feedback system, method and vehicle. The system comprises a central control end and a head-mounted display device. The central control end comprises a central control module and an information layered module. The information layered module is connected with the head-mounted display device. The central control module is used for acquiring target driving real-time information and generating a display interface. The information layered module is used for dividing the display interface into a plurality of data layers according to a set requirement. The central control module is further used for acquiring the state of eyeballs, thereby capturing the focus of the eyeballs and determining eyeball focus point objects. According to the focus direction of the eyeballs or the eyeball focus point objects, the corresponding data layers are controlled to be hidden or displayed, so as to send the data layers to be displayed to the head-mounted display device for display.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention belongs to the field of augmented reality technology, and particularly relates to an information layering feedback system, method and vehicle for in-vehicle augmented reality. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] Augmented reality head-up display (AR-HUD) systems are primarily pre-installed, fixed below the windshield of a car, projecting the image onto the driver's eyes through reflection from the windshield. However, pre-installed AR-HUD systems based on the car's windshield have several drawbacks: the displayed image size is relatively small; image clarity and brightness are low, and ghosting is common; the optical-mechanical structure occupies a significant amount of space inside the vehicle. AR glasses can be used to achieve the desired augmented reality display. In order to divide the display area into the optimal viewing angle of the human eye and the overall vehicle field of vision requirements, the existing technology uses AR glasses to replace the original fixed space display of AR-HUD. The entire usage scenario of AR glasses is that the user is facing forward, and the information displayed on the AR glasses is basically fixed, which is equivalent to the display in front of the AR-HUD. However, in real usage scenarios, the driver needs to turn his head to obtain information in other directions, such as looking to the sides, or even looking behind the car, on the roof, or under the car. When the user is wearing AR glasses and needs to turn his head or body to obtain information in other directions, the fixed information display module on the AR glasses in the existing technology will interfere with the actual observation.

[0004] When car users wear AR glasses to receive various signals, if the information displayed on the lenses doesn't adapt to changes in the user's field of vision, useful information becomes noise. To address this issue, existing technology provides a wearable AR head-up display system for vehicles, which displays navigation information, driving warning information, vehicle information, and communication information in separate areas on smart AR glasses. However, the information in the AR glasses is only displayed in sections, failing to address the problem of displaying corresponding information in different environments, thus avoiding information overload and a degraded user experience. Summary of the Invention

[0005] To address the technical problems existing in the background art, the present invention provides an in-vehicle augmented reality information layered feedback system, method, and vehicle, which can display preset information in different environments, avoid information overload, and highlight information that reduces driving risks in special situations, thereby improving user experience.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] The first aspect of the present invention provides an information layering feedback system for in-vehicle augmented reality.

[0008] A vehicle-mounted augmented reality information layered feedback system includes: a central control terminal and a head-mounted display device;

[0009] The central control unit includes a central control module and an information layering module; the information layering module is connected to the head-mounted display device.

[0010] The central control module is used to: acquire real-time driving information of the target vehicle and generate a display interface;

[0011] The information layering module is used to: divide the display interface into several data layers according to set requirements;

[0012] The central control module is also used for:

[0013] It acquires the state of the eyeball to capture the eyeball's focus and determine the object at the eyeball's focal point;

[0014] Based on the direction of the eye's focus or the object the eye is focusing on, the corresponding data layer is controlled to be hidden or shown, so that the data layer to be displayed is sent to the head-mounted display device for display.

[0015] As one implementation, when the coordinate set of the eye's field of vision and the coordinate set of the windshield intersect, it is determined that the eye's focal point is facing the windshield. At this time, the central control module is used to control the display of all data layers.

[0016] As one implementation, when the object of eye focus is the connecting pillar between the roof and the front cabin, the left and right turn mirrors, or the rearview mirror, the central control module is used to control the display of user demand information and warning information related data layers, and hide other data layers.

[0017] As one implementation, when the object at the eye's focus is not the connecting pillar between the roof and the front cabin, not the left and right turn mirrors, or not a rearview mirror, the central control module is used to control the hiding of all data layers.

[0018] In one implementation, the central control module is used to: determine the direction of the eye focus point or the object at the eye focus point by comparing and intersecting the user's field of vision coordinate set with the coordinate set of the windshield, the coordinate set of the connecting pillar between the roof and the front compartment, the coordinate set of the left and right turn mirrors, and the coordinate set of the rearview mirror in the vehicle's driving coordinate system.

[0019] As one implementation method, the target vehicle real-time information includes driving data, driving environment data, vehicle body data, and navigation data.

[0020] In one implementation, the head-mounted display device is AR glasses.

[0021] In one implementation, the AR glasses include optical lenses, two laser emitters, an inner camera, and an outer camera; the two laser emitters and the outer camera are located at the outer end of the head-mounted display device, and are used to determine the direction the head-mounted display device is facing and the surrounding images, respectively; the inner camera is located at the center of the inner side of the head-mounted display device, and is used to identify the direction of eye movement, angle, and focus.

[0022] As one implementation method, the information layering of the information layering module depends on the direction of the user's field of vision and the focus of the user's field of vision.

[0023] As one implementation method, information layering of the information layering module is achieved based on customized information.

[0024] A second aspect of the present invention provides a method for hierarchical feedback of information in vehicle-mounted augmented reality.

[0025] A method for hierarchical information feedback in vehicle-mounted augmented reality, comprising:

[0026] The system acquires real-time driving information of the target vehicle and generates a display interface, which consists of several pre-divided data layers.

[0027] It acquires the state of the eyeball to capture the eyeball's focus and determine the object at the eyeball's focal point;

[0028] Based on the direction of the eye's focus or the object the eye is focusing on, the corresponding data layer is controlled to be hidden or shown, so that the data layer to be displayed is sent to the head-mounted display device for display.

[0029] As one implementation method, when there is an intersection between the coordinate set of the eye's field of vision and the coordinate set of the windshield, it is determined that the eye's focal point is facing the windshield, and at this time, all data layers are controlled to be displayed.

[0030] As one implementation method, when the object that the eye is focused on is the connecting pillar between the roof and the front cabin, the left and right turn mirrors, or the rearview mirror, the data layer related to user needs and warning information is displayed, while the other data layers are hidden.

[0031] As one implementation method, when the object that the eye is focused on is not the connecting pillar between the roof and the front cabin, not the left and right turn mirrors, or not a rearview mirror, all data layers are hidden.

[0032] As one implementation method, the process of determining the object at the focal point of the eye is as follows:

[0033] In the vehicle's driving coordinate system, the user's field of vision coordinate set is compared and intersected with the coordinate sets of the windshield, the roof and the connecting pillar of the front compartment, the left and right side mirrors, and the rearview mirror, respectively, to determine the direction of the eye's focal point or the object at the eye's focal point.

[0034] A third aspect of the invention provides a vehicle.

[0035] A vehicle comprising an in-vehicle augmented reality information layered feedback system as described above.

[0036] Compared with the prior art, the beneficial effects of the present invention are:

[0037] This invention uses an information layering module to divide the display interface into several data layers according to set requirements. Then, based on the direction of the eye's focus point or the object the eye is focused on, it further determines the information displayed by the head-mounted display device, making driving information data more intelligent and user-friendly. It also changes with the driving environment, displaying preset information and hiding other information in different environments to avoid information overload. At the same time, information that reduces driving risks will be highlighted in special situations, improving the user experience.

[0038] Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0039] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0040] Figure 1 This is a schematic diagram of the information layered feedback system structure of the in-vehicle augmented reality according to an embodiment of the present invention;

[0041] Figure 2 These are the vehicle driving coordinate system and the user's field of view coordinate system in this embodiment of the invention;

[0042] Figure 3 This is a head-mounted display device according to an embodiment of the present invention.

[0043] The components include: 1. Central control unit; 2. Head-mounted display device; 3. Windshield; 4. Rearview mirror; 5. Turning rearview mirror; 6. A-pillar; 7. Real-time database; 11. Central control module; 12. Information layering module; 13. Output module; 21. Optical lens; 22. Laser emitter; 23. Internal camera; 24. External camera. Detailed Implementation

[0044] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0045] It should be noted that the following detailed description is illustrative and intended to provide further explanation of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0046] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0047] Example 1

[0048] Reference Figure 1 This embodiment provides an information layered feedback system for in-vehicle augmented reality, which includes a central control terminal 1 and a head-mounted display device 2;

[0049] The central control terminal 1 includes a central control module 11 and an information layering module 12; the information layering module 12 is connected to the head-mounted display device 2.

[0050] The central control module 11 is used to: acquire real-time driving information of the target vehicle and generate a display interface;

[0051] The information layering module 12 is used to: divide the display interface into several data layers according to set requirements;

[0052] The central control module 11 is also used for:

[0053] It acquires the state of the eyeball to capture the eyeball's focus and determine the object at the eyeball's focal point;

[0054] Based on the direction of the eye's focus or the object the eye is focusing on, the corresponding data layer is controlled to be hidden or shown, so that the data layer to be displayed is sent to the head-mounted display device for display.

[0055] In the specific implementation process, driving data (such as vehicle speed, RPM / power, energy consumption, steering, reversing, etc.), driving environment data (such as weather, visibility, wind speed, road conditions, lane departure warning, hazard warning, etc.), vehicle data (such as tire pressure monitoring, fuel / battery level, range, etc.), and navigation data (such as navigation prompts, steering, mileage, speed limit, red lights, etc.) related to the target vehicle information are acquired to form a real-time database 7 for the target vehicle. The central control terminal 1 generates a display interface from the data messages in the real-time database 7, and the central control terminal is connected to the head-mounted display device.

[0056] Communication data (calls, entertainment, text messages, etc.) can also be stored in the target vehicle's real-time database.

[0057] In one embodiment, the information layering of the information layering module 12 depends on the direction of the user's field of vision and the focal point of the user's field of vision.

[0058] In other embodiments, those skilled in the art can implement information layering of the information layering module based on customized information.

[0059] Those skilled in the art can choose to add or remove these functional modules at different data levels. Different data levels can be used for different usage scenarios, such as during high-speed driving, normal urban road use, or in congested or high-risk road sections. This can involve automatic identification of the vehicle's environment or user-selected settings. The defined mode can have two layers: first, the information layer is determined by the vehicle's environment; second, different functional modules can be added or removed within each information layer determined by the environment. These settings can be default or customized according to personal preference. The form of information layering can also be customized based on user preferences.

[0060] For example, the information layering module 12 divides the information into layers A, B, and C. Layer A presents relatively light auxiliary information, displaying simple information such as vehicle speed, distance to the vehicle in front, and time; Layer B presents medium-level auxiliary information, including the information in Layer A, plus driving direction guidance, optimal driving speed suggestions, and safety predictions of the driving trajectories of surrounding vehicles; Layer C presents advanced auxiliary information, including the information in Layers A and B, plus suggestions for vehicle driving based on changes in road conditions. These different presentation information are similar to functional modules, which can be added or removed at different data levels to suit different usage scenarios. For example, Layer A is used for high-speed driving, Layer B for normal urban road use, and Layer C for congested or high-risk road sections.

[0061] In the specific implementation process, the central control terminal 1 also includes an output module 13, which sends one or more data layers selected for display on the display interface to the head-mounted display device for display according to the set requirements.

[0062] In practical implementation, the central control module 11 is used to: determine the direction of the eye's focal point or the object at the eye's focal point by comparing and intersecting the user's field of vision coordinate set with the coordinate sets of the windshield, the roof and front cabin connecting pillar, the left and right side mirrors, and the rearview mirror in the vehicle's driving coordinate system. This achieves the goal of accurately determining the object at the eye's focal point by unifying the focal point, the roof and front cabin connecting pillar, the left and right side mirrors, and the rearview mirror in the vehicle's driving coordinate system.

[0063] When the coordinate set of the eye's field of vision intersects with the coordinate set of the windshield 3, it is determined that the eye's focal point is facing the windshield 3. At this time, the central control module 11 is used to control the display of all data layers.

[0064] For example, when the eye's focus is on the windshield 3, the central control module 11 transmits all display data layers to the head-mounted display device for display through the output module 13. At this time, the head-mounted display device performs similarly to a conventional AR-HUD.

[0065] When the object focused on by the eye is the connecting pillar between the roof and the front cabin (such as A-pillar 6), the left and right turn-around mirrors (turn-around mirror 5), or the rearview mirror 4, the central control module controls the display of user demand information and warning information related data layers, while hiding other data layers. For example, when the object focused on by the eye is the connecting pillar between the roof and the front cabin, the left and right turn-around mirrors, or the rearview mirror, the central control module only transmits user demand information and warning information related data layers to the head-mounted display device for display through the output module. This makes some hidden information more clearly displayed, avoiding the weakening of information capture by blind spots and weak areas of vision.

[0066] When the object that the eye focuses on is not the connecting pillar between the roof and the front cabin, not the left and right turn mirrors, or not a rearview mirror, the central control module is used to control the hiding of all data layers.

[0067] In some embodiments, when the object focused on by the eye is not the connecting pillar between the roof and the front cabin, not the left or right turn mirrors, or not a rearview mirror, the head-mounted display device does not display any information. Thus, when the head-mounted display device does not display any information, it returns to its normal state and does not interfere with normal communication or observation with other occupants of the vehicle.

[0068] like Figure 3As shown, in this embodiment, the head-mounted display device is AR glasses. The AR glasses include optical lenses 21, two laser emitters 22, an internal camera 23, and an external camera 24. The two laser emitters 22 and the external camera 24 are located at the outer end of the head-mounted display device and are used to determine the direction the head-mounted display device is facing and the surrounding image, respectively. The internal camera 23 is located at the center of the inner side of the head-mounted display device and is used to identify the direction, angle, and focus of the eyeballs.

[0069] Specifically, the process of determining the user's field of vision direction is as follows:

[0070] While the vehicle is in motion, the user correctly wears the AR glasses. A user's field of vision coordinate system is constructed using the centroids of two laser emitters and the head-mounted display device. This user's field of vision coordinate system is dynamically positioned within the vehicle's driving coordinate system (which changes with the user's head or body movements). The AR glasses' internal camera monitors the user's eye movement direction, angle, and pupil contraction (focus). The AR glasses' external camera, based on the eye information collected by the internal camera, synchronously adjusts its rotation direction, angle, and focus, simulating the user's eye movement at different vehicle speeds for safe operation. The simulated safe field of view range forms a real-time coordinate set of the user's field of view in the user's field of view coordinate system and is synchronized to the vehicle's driving coordinate system. In the vehicle's driving coordinate system, the real-time coordinate set of the user's field of view is compared and the intersection is calculated with the corresponding coordinate sets of the windshield, left and right A-pillars, left and right turn-around rearview mirrors and rearview mirrors in the vehicle's driving coordinate system. When the real-time coordinate set of the user's field of view intersects with the coordinate sets of various structures of the vehicle itself, it indicates where the user's field of view is focused. The central control terminal displays the display interface of the focused field of view on the optical lens of the AR glasses.

[0071] Among them, such as Figure 2 As shown, the vehicle's driving coordinate system is as follows: with the vehicle's center of gravity as its origin, the X-axis is parallel to the vehicle chassis and points forward, the Z-axis is perpendicular to the vehicle chassis and points upward, and the Y-axis points to the right side of the vehicle.

[0072] User field of view coordinate system: with the centroid of the head-mounted display device as its origin, the X-axis is the direction of the laser emitted by the laser emitter, the Y-axis points to the right side of the head-mounted display device (parallel to the line connecting the two laser emitters), and the Z-axis is perpendicular to the plane formed by the X-axis and Y-axis and points upward.

[0073] Table 1 Relationship between vehicle speed and safe visibility

[0074] Vehicle speed (km / h) Binocular vision Left vision Right vision 0 160° 80° 80° 40 100° 50° 50° 70 70° 35° 35° 100 40° 20° 20°

[0075] Determine the corresponding coordinate sets of the windshield, left and right A-pillars, left and right turn-around mirrors, and rearview mirrors in the vehicle's driving coordinate system;

[0076] Simultaneously determine the dynamic coordinate set of the user's field of view coordinate system within the vehicle's driving coordinate system;

[0077] The user's field of vision coordinate system projects the range of the safe field of vision onto the user's field of vision coordinate system according to different vehicle speeds, and calculates and determines the coordinate set of the safe field of vision in the vehicle's driving coordinate system in real time as the user's field of vision coordinate system moves.

[0078] The user's head rotation is determined by the positional relationship between the centroids of the two laser emitters and the head-mounted display device in the vehicle's driving coordinate system. This determines the user's facing direction and the position of the user's field of vision coordinate system within the vehicle's driving coordinate system. (Data processing is completed in the central control unit; three points determine a plane. In the user's field of vision coordinate system, the direction of the user's head rotation can be determined. This, combined with the direction of the eye's field of vision, determines the direction or object the user is actually observing, and calculates what information should be fed back to the user.) The internal camera monitors the rotation and focusing of the user's eyes. The external camera uses the information from the internal camera to determine the point where the user's eyes are focused. The external camera maps the focus point it captures to the vehicle's driving coordinate system and then compares and intersects the coordinates of the focus point with (the corresponding coordinate sets of the windshield, left and right A-pillars, left and right rearview mirrors, and rearview mirrors in the vehicle's driving coordinate system) to determine the actual focused object in the user's field of vision.

[0079] For example:

[0080] When the user looks at the windshield, the AR glasses display road information, navigation information, and other information similar to those displayed by a regular AR-HUD.

[0081] When a user is preparing to or turning (e.g., turning the steering wheel or activating the turn signal) and looking at the left or right A-pillar and the turn signal mirror, the AR glasses will display the blind spot image blocked by the left or right A-pillar and the image in the turn signal mirror. Especially when visibility is low (night, rain, fog), the image in the turn signal mirror may be unclear, and the AR glasses can also brighten the corresponding image (the central control terminal retrieves the corresponding display interface from the real-time database) to improve safety. In good driving conditions, users can also choose to turn off the display interface when looking at the A-pillar, the rearview mirror, and the turn signal mirror.

[0082] When a user looks at the rearview mirror, the AR glasses will display the scene that should be displayed in the rearview mirror. Similarly, when visibility is low (night, rain, fog) and the scene in the rearview mirror is not clear, the AR glasses can also display the corresponding scene (the central control terminal retrieves the corresponding display interface from the real-time database), thus improving safety.

[0083] When the user looks at the interior of the car and their field of vision is not focused on the windshield, left and right A-pillars, left and right turn mirrors, and rearview mirrors, the AR glasses will not display an interface on the optical lenses. This ensures that the user will not be disturbed by the display interface when having necessary communication and observation in the car, and the AR glasses will become normal glasses.

[0084] In other embodiments, the vehicle can use a set of coordinates instead of steering mirrors and side mirrors, and use some feature points or surfaces to replace the positions where the user originally looks at the steering mirrors and side mirrors, thus simplifying the vehicle.

[0085] This embodiment makes the information data in AR glasses more intelligent and user-friendly, changing with the driving environment. In different environments, it displays useful information, avoids information overload, and highlights information that reduces driving risks in special situations, thereby improving the user experience.

[0086] The AR glasses in this embodiment can determine what information the user wants to obtain when their field of vision changes, and thus can actively change the information displayed on the AR glasses. Based on the set information density, the AR glasses will transmit the information that the user needs to know and the information that the user should know to the user. The flexible and intelligent usage scenarios are more in line with the user's previous usage habits, making it more convenient for the user.

[0087] Example 2

[0088] This embodiment provides a method for hierarchical information feedback in vehicle-mounted augmented reality, which includes:

[0089] Step 1: Obtain real-time driving information of the target vehicle and generate a display interface; the display interface consists of several pre-divided data layers.

[0090] Step 2: Obtain the state of the eyeball to capture the focus of the eyeball and determine the object at the eyeball's focal point.

[0091] In step 2, the process of determining the object that the eye focuses on is as follows:

[0092] In the vehicle's driving coordinate system, the user's field of vision coordinate set is compared and intersected with the coordinate sets of the windshield, the roof and the connecting pillar of the front compartment, the left and right side mirrors, and the rearview mirror, respectively, to determine the direction of the eye's focal point or the object at the eye's focal point.

[0093] When the coordinate set of the eye's field of vision intersects with the coordinate set of the windshield, it is determined that the eye's focal point is facing the windshield, and at this time, all data layers are controlled to be displayed.

[0094] When the focus of the eye is on the connecting pillar between the roof and the front cabin, the left and right turn mirrors or the rearview mirror, the data layer related to user needs and warning information is displayed, and the rest of the data layer is hidden.

[0095] When the object that the eye focuses on is not the connecting pillar between the roof and the front cabin, not the left and right turn mirrors, or not a rearview mirror, control all data layers to be hidden.

[0096] Example 3

[0097] This embodiment provides a vehicle that includes the in-vehicle augmented reality information layering feedback system as described in Embodiment 1 above.

[0098] It should be noted that, apart from the in-vehicle augmented reality information layered feedback system, the other structures of the vehicle in this embodiment can be implemented using existing structures, which will not be described in detail here.

[0099] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A vehicle-mounted augmented reality information layered feedback system, characterized in that, include: Central control unit and head-mounted display device; The central control unit includes a central control module and an information layering module; the information layering module is connected to the head-mounted display device. The central control module is used to: acquire real-time driving information of the target vehicle and generate a display interface; The information layering module is used to: divide the display interface into several data layers according to set requirements; The central control module is also used for: It acquires the state of the eyeball to capture the eyeball's focus and determine the object at the eyeball's focal point; Based on the direction of the eye's focus or the object the eye is focusing on, the corresponding data layer is controlled to be hidden or shown, so that the data layer to be displayed is sent to the head-mounted display device for display; When the coordinate set of the eye's field of vision and the coordinate set of the windshield intersect, it is determined that the eye's focal point is facing the windshield. At this time, the central control module is used to control the display of all data layers. When the object focused on by the eye is the connecting pillar between the roof and the front cabin, the left and right turn mirrors or the rearview mirror, the central control module is used to control the display of user demand information and warning information related data layers, and hide other data layers. When the object that the eye focuses on is not the connecting pillar between the roof and the front cabin, not the left and right turn mirrors, or not a rearview mirror, the central control module is used to control the hiding of all data layers.

2. The in-vehicle augmented reality information layered feedback system as described in claim 1, characterized in that, The central control module is used to: determine the direction of the eye focus point or the object at the eye focus point by comparing and intersecting the user's field of vision coordinate set with the coordinate set of the windshield, the coordinate set of the connecting pillar between the roof and the front compartment, the coordinate set of the left and right turn mirrors, and the coordinate set of the rearview mirror in the vehicle's driving coordinate system.

3. The in-vehicle augmented reality information layered feedback system as described in claim 1, characterized in that, The target vehicle's real-time driving information includes driving data, driving environment data, vehicle body data, and navigation data.

4. The in-vehicle augmented reality information layered feedback system as described in claim 1, characterized in that, The head-mounted display device is AR glasses.

5. The in-vehicle augmented reality information layered feedback system as described in claim 4, characterized in that, The AR glasses include optical lenses, two laser emitters, an inner camera, and an outer camera; the two laser emitters and the outer camera are located at the outer end of the head-mounted display device, and are used to determine the direction the head-mounted display device is facing and the surrounding images, respectively; the inner camera is located at the center of the inner side of the head-mounted display device, and is used to identify the direction of eye movement, angle, and focus.

6. The in-vehicle augmented reality information layered feedback system as described in claim 1, characterized in that, The information layering module depends on the direction of the user's field of vision and the focus of the user's field of vision.

7. The in-vehicle augmented reality information layered feedback system as described in claim 1, characterized in that, The information layering module implements information layering based on customized information.

8. A method for hierarchical information feedback in vehicle-mounted augmented reality, applied to the hierarchical information feedback system for vehicle-mounted augmented reality as described in any one of claims 1-7, characterized in that, include: The system acquires real-time driving information of the target vehicle and generates a display interface, which consists of several pre-divided data layers. It acquires the state of the eyeball to capture the eyeball's focus and determine the object at the eyeball's focal point; Based on the direction of the eye's focus or the object the eye is focusing on, the corresponding data layer is controlled to be hidden or shown, so that the data layer to be displayed is sent to the head-mounted display device for display; When the coordinate set of the eye's field of vision intersects with the coordinate set of the windshield, it is determined that the eye's focal point is facing the windshield, and at this time, all data layers are controlled to be displayed. When the focus of the eye is the connecting pillar between the roof and the front cabin, the left and right turn mirrors or the rearview mirror, control the display of the data layer related to user needs and warning information, and hide the other data layers. When the object that the eye focuses on is not the connecting pillar between the roof and the front cabin, not the left and right turn mirrors, or not a rearview mirror, control all data layers to be hidden.

9. The information layering feedback method for in-vehicle augmented reality as described in claim 8, characterized in that, The process of determining the object as the focal point of the eye is as follows: In the vehicle's driving coordinate system, the user's field of vision coordinate set is compared and intersected with the coordinate sets of the windshield, the roof and the connecting pillar of the front compartment, the left and right side mirrors, and the rearview mirror, respectively, to determine the direction of the eye's focal point or the object at the eye's focal point.

10. A vehicle, characterized in that, Including the information layered feedback system for in-vehicle augmented reality as described in any one of claims 1-7.

Citation Information

Patent Citations

  • Information processing method and device

    CN111443796A

  • Driving information display method, device and system

    CN115097628A