Vehicle augmented reality navigation method, augmented reality head-up display device, and vehicle

Abstract

The application discloses a kind of vehicle augmented reality navigation method, augmented reality heads-up display device and vehicle.Therein, the method comprises: according to the lane level navigation map data obtained, the bifurcation data of the navigation route is obtained;In response to bifurcation data meet preset trigger condition, at least one reverse guide element is rendered and displayed in the field of view range of augmented reality heads-up display of vehicle, wherein, bifurcation data is based on the positioning information of vehicle and lane level navigation map data acquisition, different reverse guide elements are mapped in different no-entry lane, and no-entry lane is determined based on bifurcation data and the lane that vehicle is prohibited to travel.The lane of the present application solves the technical problem that the accuracy and significance of navigation guidance for the driver in the bifurcation driving scene in the related art are low.

Description

Technical Field

[0001] This invention relates to the field of navigation, and more specifically, to a vehicle augmented reality navigation method, an augmented reality head-up display device, and a vehicle. Background Technology

[0002] In current urban traffic and highway driving scenarios, the visualization of navigation data at intersections poses a major challenge to navigation systems, demanding higher accuracy and safety in guiding and prompting drivers when facing multi-lane choices at intersections. When a vehicle approaches an intersection, drivers need to make lane selection decisions and change lanes in advance within a short time. Current navigation methods, such as those based on in-vehicle screens or electronic dashboards, rely on static and two-dimensional arrow guidance, lacking precise guidance and prominent prompts. This makes it difficult for drivers to quickly identify the correct driving path at multi-lane intersections, increasing the likelihood of emergency lane changes and thus threatening driving safety. In other words, the accuracy and prominence of navigation guidance for drivers at intersections are relatively low.

[0003] There is currently no effective solution to the above problems. Summary of the Invention

[0004] This invention provides a vehicle augmented reality navigation method, an augmented reality head-up display device, and a vehicle, to at least solve the technical problem of low accuracy and salience in navigation guidance for drivers in intersection driving scenarios in related technologies.

[0005] According to one aspect of the present invention, a vehicle augmented reality navigation method is provided, comprising: acquiring intersection data along the navigation route based on acquired lane-level navigation map data; and rendering and displaying at least one reverse guidance element within the field of view of the vehicle's augmented reality head-up display in response to the intersection data satisfying a preset trigger condition, wherein the intersection data is acquired based on the vehicle's positioning information and lane-level navigation map data, and different reverse guidance elements are mapped within different prohibited lanes, the prohibited lanes being lanes where vehicles are prohibited from driving, determined based on the intersection data and the vehicle's perception data.

[0006] According to another aspect of the present invention, a vehicle augmented reality navigation device is also provided, comprising: an acquisition module, configured to acquire intersection data along the navigation route based on acquired lane-level navigation map data; and a display module, configured to render and display at least one reverse guidance element within the field of view of the vehicle's augmented reality head-up display in response to the intersection data meeting a preset trigger condition, wherein the intersection data is acquired based on the vehicle's positioning information and lane-level navigation map data, and different reverse guidance elements are mapped within different prohibited lanes, the prohibited lanes being lanes where vehicles are prohibited from driving, determined based on the intersection data and the vehicle's perception data.

[0007] According to another aspect of the present invention, an augmented reality head-up display device is also provided, comprising: a memory storing an executable program; and a processor for running the program, wherein the program executes the methods of various embodiments of the present invention during runtime.

[0008] According to another aspect of the present invention, a vehicle is also provided, including the above-described augmented reality head-up display device.

[0009] According to another aspect of the present invention, a computer-readable storage medium is also provided, the computer-readable storage medium including a stored executable program, wherein, when the executable program is executed, it controls the device where the computer-readable storage medium is located to perform the methods of various embodiments of the present invention.

[0010] According to another aspect of the present invention, a computer program product is also provided, including a computer program that, when executed by a processor, implements the methods of various embodiments of the present invention.

[0011] According to another aspect of the present invention, a computer program product is also provided, including a non-volatile computer-readable storage medium storing a computer program that, when executed by a processor, implements the methods of various embodiments of the present invention.

[0012] According to another aspect of the present invention, a computer program is also provided, which, when executed by a processor, implements the methods of the various embodiments of the present invention.

[0013] In this embodiment of the invention, firstly, based on the acquired lane-level navigation map data, intersection data along the navigation route is obtained. Then, in response to the intersection data meeting preset trigger conditions, at least one reverse guidance element can be rendered and displayed within the augmented reality head-up display (HUD) field of view of the vehicle. This intersection data is obtained based on the vehicle's positioning information and lane-level navigation map data. Different reverse guidance elements can be mapped into different prohibited lanes, which are lanes where vehicles are prohibited from driving, determined based on the intersection data and the vehicle's perception data. It is noteworthy that this application, through lane-level navigation map data, can more accurately identify intersection data along the vehicle's navigation route. In driving scenarios where the vehicle approaches an intersection, the reverse guidance element is displayed instantly based on the augmented reality HUD. Reverse guidance elements are marked on those prohibited lanes, providing a direct visual blocking prompt for prohibited lanes. This can more significantly and clearly remind drivers to avoid entering these lanes, effectively preventing drivers from entering prohibited lanes, avoiding traffic chaos and potential accidents caused by drivers changing lanes temporarily, and significantly improving driving safety. Meanwhile, since the reverse guidance elements are displayed directly on the driver's line of sight by the augmented reality head-up display, the number of times the driver needs to look away from the screen during driving can be reduced, making the driving process smoother and improving the overall driving experience. This advantage is even more obvious in scenarios with dense traffic flow and complex intersections, thus solving the technical problem of low accuracy and significance of navigation guidance for drivers in intersection driving scenarios in related technologies. Attached Figure Description

[0014] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0015] Figure 1 This is a flowchart of a vehicle augmented reality navigation method according to an embodiment of the present invention;

[0016] Figure 2 This is a schematic diagram of an optional rendering and display of a reverse guide element according to an embodiment of the present invention;

[0017] Figure 3 This is a schematic diagram of an optional method for determining the target rendering position according to an embodiment of the present invention;

[0018] Figure 4 This is a schematic diagram of a vehicle augmented reality navigation device according to an embodiment of the present invention;

[0019] Figure 5 This is a schematic diagram of an augmented reality head-up display device according to an embodiment of the present invention. Detailed Implementation

[0020] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0021] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0022] According to an embodiment of the present invention, an embodiment of a vehicle augmented reality navigation method is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0023] This application provides a vehicle augmented reality navigation method. This method can be used to provide augmented reality navigation functionality for preset application scenarios. These preset application scenarios can include the following scenarios in the vehicle field: driver-driven vehicle scenarios, assisted navigation driving scenarios, and intelligent navigation-guided pilot (NGP) scenarios in urban or highway areas. Furthermore, the preset application scenarios may also include, but are not limited to: augmented reality navigation scenarios for assisted navigation of trucks in the logistics and transportation field, augmented reality navigation scenarios for assisted navigation of agricultural vehicles in the agricultural machinery field, augmented reality navigation scenarios for controlling drones, and augmented reality navigation scenarios for controlling intelligent robots (such as cleaning robots, service robots, delivery robots, etc.).

[0024] When the aforementioned preset application scenario is a scenario in a field other than the vehicle field, those skilled in the art should understand that the vehicle in the above-mentioned vehicle augmented reality navigation method can be replaced with other objects (such as agricultural machinery, drones, robots, etc.), and correspondingly, the augmented reality navigation function applied to the vehicle can be replaced with augmented reality navigation functions related to other objects. Based on this, this application embodiment takes the vehicle field as an example to illustrate the specific implementation of the above-mentioned augmented reality navigation method.

[0025] The Augmented Reality Head-Up Display (ARHUD) in this application refers to a technology that projects virtual information, in the form of images or text, into the driver's line of sight, enabling the driver to more easily access and understand various information while driving. The ARHUD interface can be located in front of the vehicle's dashboard, projecting information onto the windshield or other transparent surfaces. The driver can see this virtual information without taking their eyes off the vehicle. This virtual information can include, but is not limited to, vehicle speed, navigation instructions, traffic signs, warning messages, phone calls, and music playback, and can be determined according to actual needs; no specific limitations are imposed here.

[0026] While both displaying navigation information on the vehicle's infotainment screen and displaying navigation information via an AR HUD can provide navigation guidance, displaying navigation information via an AR HUD is not simply changing the subject of the same navigation information. Rather, there are fundamental differences in the logic and effects of generating and implementing the displayed navigation information. These differences determine their respective characteristics and application scenarios.

[0027] The core of AR HUD lies in integrating the navigation guidance light carpet with the actual road, merging perceived lane lines, surrounding vehicles, non-motorized vehicles, pedestrians, and animals with the real environment, and presenting it on the windshield in front of the driver's line of sight. In effect, this alignment and integration of virtual navigation information with the real world ensures a high degree of consistency between virtual and real-world elements, providing users with an intuitive and immersive driving assistance experience.

[0028] In contrast, the navigation information displayed on the in-vehicle infotainment screen is presented as virtual information such as navigation routes (not navigation guide light carpets) and the vehicle itself and surrounding vehicles. These are rendered through animations on a fixed display screen inside the car. The navigation routes and virtual information do not need to be spatially integrated with the external environment. Their main purpose is to show the user what the surrounding environment is like, and they do not need to be as precise.

[0029] The specific differences are as follows:

[0030]

[0031] Table 1

[0032] As shown in Table 1, regarding whether virtual information aligns with actual roads, the fundamental difference between AR HUD and traditional in-vehicle screens in navigation display function lies in their ability to overlay navigation information directly within the driver's field of vision due to their augmented reality characteristics. AR HUDs can display a navigation guidance light carpet on the current road, aligning it perfectly with the actual road. This allows users to clearly understand how to drive ahead, providing a very intuitive experience. In contrast, in-vehicle screens do not require direct visual integration with the road and do not display navigation information via a light carpet. Instead, they use color-coded markers to indicate which roads are passable, letting the user know which roads are available, and ultimately, the user decides which one to take. In addition, AR HUD can display lane lines ahead within a certain range, as well as information about motor vehicles, non-motor vehicles, pedestrians, or animals in the surrounding environment. It can intuitively alert users to external factors that may affect driving operations. Especially in environments with low visibility, where users may not notice their surroundings, this information can greatly help users make decisions to avoid traffic accidents. However, the navigation on the car's screen will render some surrounding objects, but it can only give a general idea that there may be an object nearby. Users cannot intuitively perceive the specific location of the object.

[0033] Regarding whether real-time prediction of the vehicle's position is necessary, since the underlying implementation logic of these two technologies is different, for example, suppose that in low visibility conditions, an AR HUD is used to display the position element of the vehicle in front to help users identify the distance between their own vehicle and the vehicle in front. How to ensure that this position element fits the vehicle in front is very important for users. The position of the vehicle in front needs to be predicted to fit better. For example, if the vehicle in front suddenly accelerates or decelerates, in order for the animation rendering on the AR HUD to keep up with the vehicle in front, the position of the vehicle in front needs to be predicted. Specifically, multiple predicted positions of the vehicle in front need to be obtained first (for example, some predicted positions of the vehicle in front are the positions after acceleration, and some predicted positions of the vehicle in front are the positions after deceleration). Then, the positions of the vehicles in front in the current frame are averaged by a window to obtain a predicted position. This predicted position can then be used to display on the AR HUD. In this case, even if the vehicle in front suddenly accelerates, this predicted position will still fit relatively well.

[0034] The navigation screen displayed on the car's infotainment system will show an animated rendering of the car in front. However, this animation rendering does not have a strong requirement to be perfectly aligned with the car in front. Even if there is a certain distance difference between the rendered position of the car in front and its actual position, it does not matter. The car's infotainment system only needs to indicate a rough relative position of the car in front, and does not need to be particularly accurate. In other words, the navigation displayed on the car's infotainment system does not predict the speed of the car in front. Instead, it uses perception data to detect and calculate in real time whether the car will collide with the car in front at present, without needing to know whether the car will collide with the car in front in the future.

[0035] Regarding the issue of the navigation guide light carpet exceeding the screen width, since the AR HUD only displays a certain range of content in front of the vehicle, and the navigation guide light carpet must be adapted to road conditions, such as when turning right or making a U-turn, the navigation guide light carpet would exceed the screen width. However, the screen displays the passable road ahead (without showing the navigation guide light carpet), so even if a right turn or U-turn is required, the passable road ahead will be shown in the image. Therefore, the screen does not have the issue of the navigation guide light carpet exceeding the screen width.

[0036] Figure 1 This is a vehicle augmented reality navigation method according to an embodiment of the present invention, such as... Figure 1 As shown, the method includes the following steps:

[0037] Step S102: Based on the acquired lane-level navigation map data, obtain the intersection data along the navigation route.

[0038] The aforementioned lane-level navigation map data refers to high-precision lane-level map data. Lane-level navigation map data can include basic road information, such as road name, direction, and intersections, and can also include detailed descriptions of each lane, such as lane number, lane type (straight, turning, exit), lane width, and lane line attributes. Lane-level navigation map data supports lane-level navigation and driving assistance, allowing the system to identify which lane the vehicle is currently traveling in and which lanes ahead are passable and which are not. For example, when a vehicle approaches a complex intersection, lane-level navigation map data can provide detailed information on each lane, enabling the navigation system to accurately guide the driver on how to change lanes to reach their destination.

[0039] The aforementioned intersection data refers to information on intersections or forks in a vehicle's path, including precise geographical coordinates, lane configurations, traffic rules, and directional signs. By analyzing intersection data, navigation systems can predict upcoming road conditions and formulate corresponding navigation strategies, such as informing drivers in advance which lanes to enter and which to avoid. For example, at highway ramps, roundabouts in cities, or multi-exit intersections, navigation systems will use intersection data to remind drivers to prepare for lane changes in advance, preventing sudden braking or sharp turns.

[0040] As an optional implementation, lane-level navigation map data can be retrieved from a cloud server or local storage. This data can include basic road network information and detailed information about each lane, such as lane geometry, lane line attributes (solid or dashed), and lane direction (straight or turning). Alternatively, precise vehicle location information can be obtained in real time using positioning technologies such as onboard GPS receivers, inertial measurement units, and visual sensors. This information can include vehicle latitude and longitude coordinates, heading angle, speed, and other parameters, with positioning accuracy reaching the lane level, such as within 1 to 2 meters, to ensure the accuracy of subsequent operations. Next, the vehicle's location information can be matched with the lane-level information in the map data to determine the exact lane the vehicle is currently in. Then, based on the vehicle's destination, a path planning algorithm can be used to calculate a better driving route, while simultaneously identifying the data of various intersections along the route, including but not limited to intersection types such as crossroads and Y-shaped intersections, lane configurations, and traffic rules. Finally, the identified intersection data can be filtered to determine which intersections are important and will affect vehicle driving decisions. The filtering process can take into account factors such as the vehicle's real-time location, driving direction, and distance from the intersection to ensure the timeliness and relevance of the information.

[0041] In the process described above, by acquiring lane-level navigation map data, the system can more accurately identify various intersections ahead of the vehicle, such as complex multi-lane junctions. This allows the system to inform the driver in advance which lane to change lanes, which lane to follow, or which prohibited lanes to avoid. Using lane-level navigation data allows for better adaptation to various road configurations and traffic rules. For example, it can flexibly respond to highway ramps, roundabouts, or multi-exit intersections in cities, providing timely guidance and enhancing the versatility and intelligence of the navigation system.

[0042] In step S104, in response to the intersection data meeting the preset triggering conditions, at least one reverse guidance element is rendered and displayed within the augmented reality head-up display field of view of the vehicle.

[0043] The intersection data is obtained based on vehicle positioning information and lane-level navigation map data. Different reverse guidance elements are mapped into different prohibited lanes. The prohibited lanes are lanes where vehicles are prohibited from driving, determined based on intersection data and vehicle perception data.

[0044] The aforementioned preset trigger conditions can refer to a series of pre-defined rules or thresholds. When a vehicle approaches a specific intersection and the distance between the vehicle and the intersection reaches a certain threshold, the navigation system will activate the reverse guidance function. Preset trigger conditions ensure that reverse guidance elements are displayed when needed, neither too early causing information interference nor too late losing navigation value. For example, the preset trigger conditions can be set to activate when the distance between the vehicle and the intersection is less than or equal to 500 meters. Then, when the vehicle is within 500 meters of a complex highway fork in the road, the system will automatically display reverse guidance elements on the prohibited lanes, helping the driver make the correct lane selection. These preset trigger conditions can also be determined according to actual needs and are not limited here.

[0045] The aforementioned augmented reality head-up display (HUD) field of view refers to the visible area of ​​the road ahead that the driver can see through the HUD device. Augmented reality navigation information, such as arrows, text, and reverse guidance elements, can be overlaid within this area. For example, if the driver is driving along a highway, the augmented reality HUD can clearly display reverse guidance elements for lanes that are off-limits within the driver's field of view, directly above the road, without requiring the driver to move their eyes to find navigation information.

[0046] The aforementioned reverse guidance elements can refer to special visual cues displayed in augmented reality head-up displays, which can be used to indicate lane areas where vehicles cannot pass. Reverse guidance elements can be X-shaped markings, light walls, or horizontal lines, etc., so that drivers can clearly know which lanes they cannot enter. For example, at complex intersections, marking reverse guidance elements on lanes that cannot be driven can effectively prevent drivers from entering by mistake and reduce the risk of traffic accidents caused by incorrect lane selection. For another example, at a highway junction with multiple exits, the navigation system will display X-shaped markings on those unexpected exits or lanes that are prohibited from entering, clearly reminding drivers to avoid entering these lanes.

[0047] The aforementioned positioning information refers to data on the vehicle's current location, which can be provided by the Global Positioning System (GPS), Inertial Measurement Unit (INS), or other positioning technologies. This information may include the vehicle's latitude and longitude coordinates, heading angle, and speed. Positioning information helps navigation systems understand the vehicle's specific location in real time, thereby providing accurate navigation suggestions. For example, when a vehicle is driving near an intersection, the navigation system uses the positioning information to compare with lane-level navigation map data to confirm whether the vehicle is in the correct lane. If the vehicle is not in the correct lane, it can provide lane-changing suggestions or warnings.

[0048] The aforementioned perception data refers to the real-time capture of environmental changes around the vehicle by various sensors such as cameras, radar, and lidar, including the identification of road markings, traffic signals, and obstacles. This perception data can be used to monitor road conditions and the vehicle's surroundings in real time. Combined with location information and lane-level navigation map data, it can generate more refined and real-time navigation guidance, especially in complex road environments. For example, if a vehicle approaches a fork in the road with construction barriers, and the map data is not updated in time, the navigation system can adjust the display of reverse guidance elements immediately using perception data, such as construction signs captured by a camera, to prevent the driver from mistakenly entering the closed construction lane.

[0049] As an optional implementation, the intersection data can be determined to meet preset trigger conditions. These preset trigger conditions can be set according to actual needs. For example, based on the vehicle's real-time positioning information, the relative distance between the vehicle and the intersection ahead can be continuously calculated. When this relative distance shortens to within a preset threshold, the system can determine that the intersection data meets the preset trigger conditions, and then begin preparing to display the reverse guidance elements. Next, for each lane marked as infeasible, corresponding reverse guidance elements, such as X-shaped signs, can be generated using 3D rendering technology. The design of the reverse guidance elements can consider visual recognition, the driver's line of sight angle, and distance to ensure clear visibility under different lighting conditions without causing visual fatigue or distracting the driver's attention. Then, using the vehicle's positioning information and perception data, the generated reverse guidance elements can be accurately mapped to the actual location of the infeasible lanes, and the position and size of the elements can be adjusted in real time to match changes in the vehicle's motion state. For example, when the vehicle accelerates or decelerates, is in different gears, or encounters road bumps, a dynamic compensation algorithm can be used to ensure that the reverse guidance elements are always stably displayed in the driver's head-up display field of view. Under certain triggering conditions, reverse-guided elements can be projected onto the augmented reality head-up display device, creating an accurate fit with the real world.

[0050] In the aforementioned scenario, when a vehicle approaches an intersection, the driver often needs to make a lane selection decision within a short period of time. This application, by displaying reverse guidance elements in real time as the vehicle approaches the intersection, can effectively prevent the driver from entering a prohibited lane, avoiding traffic chaos and potential accidents caused by the driver's temporary lane change, and significantly improving driving safety. By displaying the reverse guidance elements directly on the driver's line of sight using an augmented reality head-up display, the number of times the driver needs to shift their gaze to the screen during driving can be reduced, making the driving process smoother and improving the overall driving experience. The intuitive display of the reverse guidance elements allows the driver to quickly understand and respond to information about prohibited lanes without having to perform complex logical reasoning, thus simplifying the driver's driving decision-making process.

[0051] In this embodiment of the invention, firstly, based on the acquired lane-level navigation map data, intersection data along the navigation route is obtained. Then, in response to the intersection data meeting preset trigger conditions, at least one reverse guidance element can be rendered and displayed within the augmented reality head-up display (HUD) field of view of the vehicle. This intersection data is obtained based on the vehicle's positioning information and lane-level navigation map data. Different reverse guidance elements can be mapped into different prohibited lanes, which are lanes where vehicles are prohibited from driving, determined based on the intersection data and the vehicle's perception data. It is noteworthy that this application, through lane-level navigation map data, can more accurately identify intersection data along the vehicle's navigation route. In driving scenarios where the vehicle approaches an intersection, the reverse guidance element is displayed instantly based on the augmented reality HUD. Reverse guidance elements are marked on those prohibited lanes, providing a direct visual blocking prompt for prohibited lanes. This can more significantly and clearly remind drivers to avoid entering these lanes, effectively preventing drivers from entering prohibited lanes, avoiding traffic chaos and potential accidents caused by drivers changing lanes temporarily, and significantly improving driving safety. Meanwhile, since the reverse guidance elements are displayed directly on the driver's line of sight by the augmented reality head-up display, the number of times the driver needs to look away from the screen during driving can be reduced, making the driving process smoother and improving the overall driving experience. This advantage is even more obvious in scenarios with dense traffic flow and complex intersections, thus solving the technical problem of low accuracy and significance of navigation guidance for drivers in intersection driving scenarios in related technologies.

[0052] Optionally, the method further includes: acquiring navigation data, wherein the navigation data includes at least: the vehicle's current lane and first light carpet control line data, the navigation data being generated based on positioning information, the vehicle's target address, and lane-level navigation map data; converting the first light carpet control line data from the map coordinate system corresponding to the navigation data to the vehicle's own coordinate system to obtain second light carpet control line data; rendering the second light carpet control line data as an augmented reality navigation light carpet, and displaying the augmented reality navigation light carpet within the field of view of the augmented reality head-up display.

[0053] The aforementioned first light carpet control line data can refer to the geometric data used in lane-level navigation maps to define the precise position and shape of the augmented reality navigation light carpet on the road. The first light carpet control line data can include a series of coordinate points to depict the boundary outline of the light carpet, as well as parameters related to the size, orientation, and shape of the light carpet.

[0054] The aforementioned vehicle coordinate system refers to a coordinate system established with the vehicle itself as the center, serving as a reference for the vehicle's orientation and motion. The origin of the vehicle coordinate system can be located at a fixed position on the vehicle, such as the center of the chassis. The X-axis can point in the direction of the vehicle's movement, the Y-axis can point to the side of the vehicle, and the Z-axis can be perpendicular to the ground. The vehicle coordinate system can dynamically change as the vehicle moves.

[0055] The aforementioned augmented reality navigation light carpet refers to a visual aid within an augmented reality navigation system. It overlays a virtual light effect, such as green or colored arrows, lines, or light strips, onto the road in front of the vehicle to visually guide the driver's direction. Setting up an augmented reality navigation light carpet provides clear and intuitive navigation guidance within the driver's natural line of sight, avoiding frequent checks of the navigation screen, thereby reducing driving risks and enhancing the driving experience.

[0056] As an optional implementation, navigation data can be obtained from lane-level navigation maps and vehicle positioning services. This navigation data may include the vehicle's current lane information and first light carpet control line data along the optimal path from the current location to the target address. The first light carpet control line data describes the geometric contour of the ideal navigation path and can consist of a series of coordinate points defined in a map coordinate system. Next, the first light carpet control line data can be converted from the map coordinate system to the vehicle coordinate system. This conversion process may include high-precision coordinate matching and real-time dynamic vehicle attitude correction. For example, quaternion interpolation compensation technology can be used to compensate for the delay in map data transmission and coordinate system deviations caused by vehicle dynamic movement, ensuring the accuracy of the light carpet data. This application's conversion of the first light carpet control line data from the map coordinate system to the vehicle coordinate system enables stable display of the navigation light carpet in the augmented reality head-up display's field of view. This conversion process eliminates deviations between different coordinate systems, ensuring that the augmented reality navigation light carpet can match the actual road scene in front of the vehicle in real time and accurately. Next, after obtaining the second light carpet control line data converted to the vehicle's coordinate system, augmented reality (AR) technology can be used to render this data into an AR navigation light carpet. This process includes 3D modeling, material setting, and transparency adjustment of the light carpet to ensure it is clearly visible in the driver's field of vision without obstructing observation of the actual road. The rendering of the light carpet also considers the relationship between the vehicle's current position and the target lane, dynamically adjusting its length, width, and curvature to adapt to the vehicle's trajectory. Finally, the AR navigation light carpet can be projected into the vehicle's AR head-up display (HUD) field of view, covering the road surface in front of the driver. For example, at a busy three-way intersection, the vehicle may need to turn from the middle straight lane to the left exit based on navigation prompts. Through the AR HUD, the driver can see a green light carpet extending from the current lane to the left exit. The width, brightness, and dynamic effects of the light carpet can adapt to different lighting conditions during the day or night, ensuring the driver can easily follow its guidance and safely and smoothly complete the turning maneuver. This display method avoids the need for the driver to frequently shift their gaze to the in-vehicle display, reducing driver distraction and improving driving safety.

[0057] In the above process, based on the lane-level map data and the vehicle coordinate system, the navigation light carpet control lines and the vehicle coordinate system achieve high-precision matching between the navigation light carpet and the actual lane, providing the driver with lane-level navigation guidance and improving navigation accuracy in complex road conditions. By displaying the navigation light carpet directly in front of the driver's line of sight through augmented reality head-up display, the driver does not need to frequently turn their head to look at the navigation screen, and can focus on the road ahead, reducing the driving risks caused by shifting gaze, especially in high-risk areas such as highway ramps and multi-lane intersections.

[0058] Optionally, the method further includes: obtaining the intersection distance in the intersection data, wherein the intersection distance is the distance between the vehicle and the intersection, and the intersection distance is a distance determined based on positioning information and lane-level navigation map data; determining that the intersection data meets the preset triggering condition in response to the intersection distance being less than or equal to a preset distance; and determining that the intersection data does not meet the preset triggering condition in response to the intersection distance being greater than the preset distance.

[0059] The intersection distance mentioned above can refer to the actual distance along the road between the vehicle's current position and the intersection ahead. In lane-level navigation systems, the intersection distance can be the straight-line distance between the vehicle's current position and the intersection ahead, or it can be the path length from the vehicle's current lane to the intersection's entrance point. The path length can take into account the road's curvature, gradient, and the vehicle's current direction of travel. The intersection distance helps determine when to start displaying augmented reality navigation light carpets or reverse guidance elements, giving the driver sufficient time to make lane selection or avoid prohibited lane decisions. By calculating the intersection distance in real time, the system can adjust the timing of augmented reality element display as needed to ensure its effectiveness.

[0060] The aforementioned preset distance refers to a threshold pre-set in the system design to determine when to activate the augmented reality navigation function. Setting the preset distance allows drivers sufficient preparation time to safely make lane selections; it also introduces information at the appropriate time, avoiding information redundancy due to premature display or impacting driver performance due to late display. The preset distance can be dynamically adjusted based on vehicle speed, driving environment (such as weather, lighting conditions, and intersection complexity), adapting to different driving scenarios and providing personalized navigation assistance. For example, in clear daytime conditions, the preset distance can be set to 500 meters, allowing drivers to see actual road signs and traffic signs earlier; while at night or in inclement weather, the preset distance can be increased to 800 meters or more, giving drivers more time to recognize and respond to the augmented reality navigation system's prompts. The specific preset distance can be determined according to actual needs and is not limited here.

[0061] As an optional implementation, the vehicle's built-in positioning system can be used to obtain the vehicle's real-time location information. Simultaneously, relevant data about the upcoming intersection can be extracted from a lane-level navigation map database, including the intersection's specific location, lane layout, and traffic rules. Then, based on the vehicle's real-time location information and lane-level navigation map data, the distance to the intersection can be determined. When the calculated intersection distance is less than or equal to a preset threshold, it can be determined that the vehicle is approaching the intersection, thereby activating the augmented reality navigation function and displaying the necessary navigation light carpet or reverse guidance elements. When the intersection distance is greater than the preset distance, the system can temporarily not display this guidance information to avoid premature display and causing driver interference.

[0062] In the above process, by obtaining the distance to the intersection and comparing it with the preset distance, the display of augmented reality navigation information is made more accurate, avoiding visual interference and information overload caused by displaying too early, and also preventing navigation invalidation caused by displaying too late. This ensures that the driver receives navigation information at the appropriate time. In driving scenarios approaching intersections, augmented reality navigation light carpets or reverse guidance elements provide the driver with intuitive and clear lane-level navigation information.

[0063] Optionally, the method further includes: transforming the first lane boundary data in the intersection data from the map coordinate system corresponding to the intersection data to the vehicle's own coordinate system to obtain transformed boundary data, wherein the first lane boundary data is used to represent the boundary of at least one prohibited lane; fusing the perception data and the transformed boundary data to obtain second lane boundary data; determining the target rendering position corresponding to at least one reverse guidance element based on the second lane boundary data; and rendering at least one reverse guidance element within the field of view of the augmented reality head-up display based on the target rendering position.

[0064] As an optional implementation, the first lane boundary data in the intersection data, i.e., the boundary point data of the prohibited lane, can be transformed into the vehicle's own coordinate system. This process can be achieved through quaternion interpolation, coordinate transformation matrices, etc., to compensate for the deviation between the map data and the vehicle's real-time position. The first lane boundary data can be adjusted according to the vehicle's instantaneous heading angle, pitch angle, and roll angle to ensure that the edge of the prohibited lane is accurately represented in the vehicle coordinate system. Then, the transformed boundary data can be fused with the vehicle's real-time perception data, such as lane lines captured by visual cameras and obstacles detected by radar, to generate second lane boundary data. The introduction of perception data ensures that the virtual signage can maintain a high degree of consistency with the actual lane lines in a dynamic driving environment, avoiding virtual signage drift or misalignment caused by vehicle movement or road surface changes. Next, based on the fused second lane boundary data, the target rendering position of each reverse guidance element within the augmented reality head-up display's field of view can be calculated. This calculation takes into account the vehicle's real-time pose, the optical characteristics of the augmented reality head-up display, and the driver's line of sight, ensuring that the virtual signs not only conform to the actual lane but also display stably and clearly within the driver's optimal line of sight. Finally, the calculated target position data can be used to render and display the reverse guidance elements in real time within the augmented reality head-up display. The visual presentation of the reverse guidance elements, such as color, size, and transparency, can be dynamically adjusted according to actual driving environment parameters, such as lighting, weather conditions, and vehicle speed, to achieve optimal readability and safety.

[0065] In the above process, by converting the boundary point data of the prohibited lane in the map coordinate system to the vehicle coordinate system and then integrating it with real-time perception data, the precise positioning and display of lane-level prohibited lane signs can be achieved, providing drivers with intuitive, safe and efficient driving assistance, and enhancing drivers' decision-making ability and driving safety.

[0066] Optionally, the first lane boundary data in the intersection data is transformed from the map coordinate system corresponding to the intersection data to the vehicle's own coordinate system to obtain transformed boundary data. This includes: acquiring first sensor data when the vehicle acquires positioning information, and second sensor data when the vehicle acquires intersection data, wherein the first sensor data includes at least a first heading angle, and the second sensor data includes at least a second heading angle; determining the vehicle's rotation angle based on the first and second heading angles; rotating the first lane boundary data based on the rotation angle to obtain first rotated boundary data; and transforming the first rotated boundary data from the map coordinate system to the vehicle's own coordinate system to obtain transformed boundary data.

[0067] As an optional implementation, first sensor data can be collected from the vehicle's positioning system, including the vehicle's position, speed, and a first heading angle, where the first heading angle can be the vehicle's direction at a certain point in time. Next, second sensor data can be obtained from the environmental perception system, including a second heading angle, i.e., the vehicle's heading angle at another point in time. Then, the first and second heading angles can be compared to calculate the vehicle's rotation angle relative to the original map data. This rotation angle reflects the vehicle's offset from the map direction during driving and can serve as the basis for subsequent coordinate transformations. Next, based on the calculated rotation angle, the first lane boundary data can be rotated accordingly to obtain first rotated boundary data. This rotation operation ensures that the lane boundary data can be accurately represented in the vehicle coordinate system, maintaining the relative position of the data even if the vehicle turns or tilts during driving. Subsequently, a coordinate transformation matrix can be used to transform the first rotated boundary data from the map coordinate system to the vehicle coordinate system, combining the data on the map with the vehicle's dynamic position and attitude, ensuring that the virtual markers can accurately fit onto lane boundaries that are not practically feasible in reality.

[0068] In the above process, by updating the angular deviation between the vehicle posture and the map data in real time, the virtual sign can be accurately matched with the non-driving lane boundary of the actual road, ensuring the stable display of the sign and the accuracy of spatial positioning.

[0069] Optionally, the second sensor data further includes: angular acceleration, which rotates the first lane boundary data based on the rotation angle to obtain first rotated boundary data, including: rotating the first lane boundary data based on the rotation angle to obtain second rotated boundary data; determining the change in heading angle based on the angular acceleration; and rotating the second rotated boundary data based on the change in heading angle to obtain the first rotated boundary data.

[0070] As an optional implementation, a first rotation operation can be performed based on the initially calculated rotation angle, i.e., the heading angle difference between the vehicle's coordinate system and the map coordinate system. This rotates the first lane boundary data in the map coordinate system to obtain second rotation boundary data, achieving preliminary compensation for the vehicle's heading angle deviation and ensuring that the virtual marker is roughly aligned with the actual lane boundary. Next, the angular acceleration of the vehicle can be continuously monitored using a gyroscope or inertial measurement unit integrated in the vehicle. Angular acceleration reflects the rate of change in the vehicle's attitude during turning, acceleration, or deceleration, capturing minute changes in vehicle attitude. Then, based on the angular acceleration data, the change in the vehicle's heading angle since the last coordinate transformation can be calculated in real time. This process involves integrating the change in angular acceleration over time to obtain minute adjustments in the vehicle's attitude, thereby estimating the new heading angle deviation. Finally, the second rotation boundary data can be combined with the newly calculated heading angle change to perform a second rotation operation, further adjusting the position of the virtual marker to obtain the first rotation boundary data. This fine-tuning process can compensate for small deviations present in the initial rotation, ensuring accurate matching between the vehicle and the actual road's prohibited lane boundaries.

[0071] During the aforementioned process, vehicles often make sharp turns, decelerate, or accelerate when approaching intersections. This application enables the virtual sign to dynamically adapt to changes in vehicle attitude by real-time measurement of angular acceleration and continuous updating of heading angle changes, maintaining an optimal display position. By combining angular acceleration data with heading angle deviation for dual correction, consistency between the virtual sign and the actual road can be maintained. The aforementioned process, through two precise rotation operations combined with real-time monitoring of angular acceleration and heading angle changes, enables accurate positioning and stable display of the virtual sign in complex intersection environments.

[0072] Optionally, the perception data and the transition boundary data are fused to obtain the second lane boundary data, including: acquiring navigation data, wherein the navigation data includes at least: the vehicle's current location in the first lane of the road, and the navigation data is generated based on positioning information, the vehicle's target location, and map data; determining the vehicle's current location in the second lane of the road based on the perception data; in response to a successful match between the first lane and the second lane, determining the transition boundary data as the second lane boundary data; and in response to a failure to match between the first lane and the second lane, correcting the transition boundary data to obtain the second lane boundary data.

[0073] As an optional implementation, navigation data can be extracted from a lane-level navigation system. This navigation data can include at least the vehicle's current lane information (i.e., the first lane), as well as other key navigation parameters such as vehicle positioning, target location, and map data. Simultaneously, onboard cameras, radar, and other sensor devices can monitor and identify the road environment around the vehicle in real time. When approaching an intersection, the vehicle's actual lane (i.e., the second lane) can be determined based on the perceived information. This lane identification process using perceived data can compensate for the lag and incompleteness of map data. Next, the first lane obtained from the navigation data can be compared and matched with the second lane determined based on the perceived data. If the first and second lanes match successfully, the accuracy of the conversion boundary data—the boundary data of prohibited lanes converted from the map coordinate system to the vehicle coordinate system—is directly confirmed, and this converted boundary data can be used directly as the boundary data for the second lane. If the first and second lane information do not match, the matching fails, indicating a deviation between the map data and the actual road conditions. At this point, the conversion boundary data can be corrected to reflect the actual perception results, thereby generating second lane boundary data that is more in line with the actual scenario. This correction process can include fine-tuning the geometry and position of the boundary to ensure that the virtual signage can accurately cover or align with the prohibited lane.

[0074] In the aforementioned process, by comparing the lane information sensed in real time with the navigation data, the system can promptly detect and correct potential errors in the map data. This dynamic correction is particularly effective during road maintenance, construction, or temporary changes, enhancing the reliability of the augmented reality navigation system. By fusing sensed data with transformation boundary data, the limitations of a single data source are overcome, enabling the augmented reality navigation system to achieve high-precision positioning and real-time correction capabilities in complex intersection environments. This provides drivers with safer, more reliable, and more intuitive lane-level navigation services.

[0075] Optionally, based on the second lane boundary data, determining the target rendering position corresponding to at least one reverse guidance element includes: determining the initial position within at least one prohibited lane based on the second lane boundary data, wherein the initial position is used to characterize the intersection point between the straight line formed by the second lane boundary data and the centerline of the corresponding lane; and determining the target rendering position based on the initial position.

[0076] As an optional implementation, the initial position within each prohibited lane can be determined based on the second lane boundary data. The initial position can be the intersection of the straight line formed by the prohibited lane boundary data and the lane's centerline. This can indicate the precise location where the sign should be placed in the road to achieve better visual effects and navigation guidance. After obtaining the initial position, the coordinates can be further adjusted to determine the target rendering position. This adjustment process can consider multiple factors such as the vehicle's real-time pose, speed, the optical characteristics of the augmented reality head-up display, and the driver's visual comfort. For example, when a vehicle approaches a prohibited lane, the sign's position can be slightly shifted into the driver's visible range to ensure the sign is always near the driver's line of sight. In practical applications, constantly changing environmental conditions and vehicle status can affect the visibility and accuracy of the sign; therefore, it can also have dynamic environmental adaptability, adjusting the sign's size, shape, and transparency in real time to ensure clear and unambiguous navigation information under various conditions.

[0077] In the above process, by accurately locating the initial position of the sign and adjusting the display position of the sign according to the lane boundary data, the system can ensure that the reverse guidance elements accurately cover the prohibited lane, greatly reducing driver misunderstanding and confusion, and improving the accuracy and effectiveness of navigation.

[0078] Optionally, the target rendering position is determined based on the initial position, including at least one of the following: using the initial position as the target rendering position; adjusting the coordinate values ​​corresponding to the first coordinate axis in the initial position to obtain the target rendering position, wherein the first coordinate axis is parallel to the vehicle's driving direction.

[0079] As an optional implementation, the initial position can be directly used. Specifically, when a vehicle approaches an intersection, if the system detects that the vehicle is running stably in a certain lane and has no obvious intention to turn or change lanes, the intersection of the vehicle's actual position and the straight line formed by the lane boundary data can be considered the relatively stable initial position. In this case, the system can directly use the initial position as the target rendering position for the reverse guidance element without additional adjustments. This processing method is simple and efficient, reduces computational resource consumption, and also ensures accurate display of the markers under stable driving conditions.

[0080] As an alternative implementation, the coordinate values ​​corresponding to the first coordinate axis can be adjusted. When the vehicle makes a significant turn or lane change, or when the vehicle is traveling at high speed, in order to compensate for the positional shift of the sign caused by vehicle movement and ensure that the sign is always within the driver's optimal line of sight, the system can adjust the coordinate values ​​corresponding to the first coordinate axis, that is, the coordinate axis parallel to the vehicle's direction of travel. Based on the vehicle's instantaneous speed, steering angle, and the optical characteristics of the augmented reality head-up display, the optimal offset of the sign along the first coordinate axis can be dynamically calculated to ensure that the sign conforms to the lane boundary and adapts to changes in the driver's line of sight, providing intuitive visual guidance.

[0081] In the above process, directly using the initial position as the target rendering position is suitable for situations where the vehicle is driving relatively smoothly. This simplifies the calculation process, improves the real-time response speed of augmented reality navigation, reduces system latency, and ensures the stability and accuracy of sign display under stable driving conditions. However, in scenarios where the vehicle experiences significant dynamic changes, fine-tuning the coordinate values ​​along the first coordinate axis ensures that the sign remains within the driver's easily perceptible field of vision, providing clear and timely lane-level navigation information.

[0082] Optionally, determining the target rendering position based on the initial position includes: obtaining the target gear position currently held by the augmented reality head-up display device; determining the viewing angle deviation between the target gear position and the preset gear position; determining the compensation height based on the intersection distance and viewing angle deviation in the intersection data; and adjusting the coordinate values ​​corresponding to the second coordinate axis in the initial position based on the compensation height to obtain the target rendering position, wherein the second coordinate axis is perpendicular to the road where the vehicle is currently located.

[0083] As an optional implementation, the current gear setting of the augmented reality head-up display (HUD) can be read. Different gear settings affect the HUD's field of view and display range, indirectly impacting the visibility and accuracy of the augmented reality signage. Next, the viewing angle deviation between the target gear and the preset gear can be compared. This deviation measurement, combined with the vehicle's distance from the intersection, allows the system to accurately calculate the required compensation height. Considering that the Z-axis projection height of the augmented reality signage differs at different gears, and the distance between the vehicle and the intersection determines the appropriate display height, the initial position's coordinates on the second coordinate axis (perpendicular to the vehicle's current road position) can be adjusted based on the calculated compensation height. This determines the accurate target rendering position of the reverse guidance element. This adjustment ensures the signage aligns horizontally with the no-entry lane boundary and vertically adapts to the vehicle's current position relative to the HUD, keeping the signage within the driver's optimal field of vision.

[0084] In the process described above, by adjusting the viewing angle caused by the head-up display (HUD) gear position, the reduced visibility range of the signs due to gear changes can be effectively compensated for. This ensures that the driver can clearly see the augmented reality signs regardless of the vehicle's gear position, thus enabling them to make the correct lane selection. The stable display of augmented reality signs on the HUD avoids the inconvenience of the driver frequently adjusting their gaze or operating the HUD, improving the driver's overall driving comfort and interactive experience.

[0085] Optionally, at least one of the following pieces of information for the reverse-guided element is dynamically adjusted based on the real environment around the vehicle: element position, element size, and element quantity.

[0086] As an optional implementation, at least one of the following pieces of information for the reverse guidance element can be dynamically adjusted based on the real-time environment surrounding the vehicle: element position, element size, and element quantity. Specifically, when a vehicle approaches an intersection, the system can adjust the position of the reverse guidance element in real time based on the vehicle's speed, direction, and distance from the intersection. For example, as the distance between the vehicle and the intersection decreases, the position of the reverse guidance element gradually moves forward to ensure that the reverse guidance element is within the driver's field of vision, providing immediate lane-level guidance. The element size can be adjusted according to the relative position and speed of the vehicle and the restricted lane. When the vehicle is far from the restricted area, the reverse guidance element can be smaller; as the vehicle approaches, the reverse guidance element can gradually enlarge. This ensures that the reverse guidance element maintains good visibility and recognizability at different distances, avoiding the problem of unclear guidance information caused by the reverse guidance element being too small or too large. In addition, in multi-lane intersection scenarios, the number of reverse guidance elements can be intelligently configured based on the lane distribution characteristics and the vehicle's expected travel path. For example, multiple reverse guidance elements can be displayed on multiple infeasible lanes to ensure that drivers are fully aware of which lanes are prohibited from entering. Furthermore, in some cases, the system may also adjust the density of reverse guidance elements to cope with changes in lane spacing and width.

[0087] In the above settings, in complex intersection environments, dynamically adjusted signs can better attract drivers' attention, reduce information omissions or misunderstandings caused by improper sign size or position, reduce the risk of emergency lane changes or accidentally entering prohibited lanes, and improve driving safety. By dynamically adjusting the position, size, and number of elements in the reverse guidance information, the guidance effect of augmented reality navigation in complex intersection scenarios can be improved, significantly enhancing the system's adaptability and safety.

[0088] Optionally, the reverse guidance elements include one of the following: a cross sign, an augmented reality light wall, and a horizontal line sign, wherein the horizontal line sign is parallel to the road where the vehicle is currently located.

[0089] As an optional implementation, the design of reverse guidance elements can include various forms such as fork-shaped signs, augmented reality light walls, and horizontal line signs. Fork-shaped signs can be used to directly indicate inaccessible lanes. Their design can consider factors such as color contrast, shape, size, and transparency to ensure clear visibility under various lighting conditions. In intersection scenarios, fork-shaped signs can intuitively convey lane prohibition information, reducing driver decision-making time and improving driving safety. Augmented reality light walls can extend along the edge of the prohibited lane, forming a virtual barrier that provides a stronger visual barrier effect, simulating a driver encountering a real physical obstacle. The height and width of the augmented reality light wall can be dynamically adjusted according to the vehicle's real-time movement and ambient lighting conditions to achieve optimal visual effects. Horizontal line signs, as an auxiliary guidance sign, are parallel to the road the vehicle is currently on and can be used to emphasize the start or end position of the prohibited lane. The display method of horizontal line signs is flexible; they can be a series of continuous short horizontal lines or long horizontal lines. Cross-shaped signs, augmented reality light walls, and horizontal line signs can also be used together to enhance the overall coherence and recognizability of the signs. Especially when driving at low speeds or in poor visibility conditions, they can provide additional visual guidance to help drivers accurately judge lane boundaries.

[0090] In the above settings, by using different visual elements to represent reverse guidance elements, the appropriate reverse guidance method can be intelligently selected according to different driving environments and conditions, effectively preventing drivers from mistakenly entering prohibited lanes, reducing the risk of traffic accidents, and providing drivers with more intuitive, safe and personalized navigation guidance in complex driving scenarios such as intersections.

[0091] The technical solution proposed in this application is described below with reference to an optional embodiment. This application proposes a real-time lane marking guidance system and method for lanes that cannot be driven at intersections based on augmented reality (AR) real-world overlay. Intersection guidance, such as complex multi-way intersections in cities and highway entrance / exit ramps, is a pain point in navigation guidance scenarios, requiring drivers to anticipate the correct lane in advance to prevent accidents caused by emergency lane changes. This application relates to the field of augmented reality navigation technology, and in particular to a method for lane-level navigation by overlaying virtual markers into real road scenes using augmented reality technology. Specifically, it can be applied to the real-time marking and warning of lanes that cannot be driven in vehicle navigation systems.

[0092] This application focuses on intersection scenarios, using full-scene lane-level augmented reality light carpets to guide drivers on which lane to take. At the same time, it draws an "X" on easily confused lanes to indicate "not this way" to help drivers make up their minds on which road to take, solving the safety problem caused by drivers' hesitation at intersections and improving the efficiency of guidance decision-making.

[0093] Specific technical solutions may include: accurate identification of intersection scenarios, where multi-source data spatiotemporal alignment can construct a map coordinate system, such as a real-time conversion engine from the WGS84 coordinate system to the vehicle coordinate system, using quaternion interpolation to compensate for the low-frequency refresh defect of map data. A dynamic compensation mechanism can solve the lateral coordinate offset caused by data transmission link delays and coordinate transformation calculation delays. After the initial calculation is completed, the results can be rotated based on the inertial measurement unit (IMU) angular velocity and the vehicle's heading angle, with the rotation angle being the difference between the heading angle at the time of the initial calculation and the heading angle at the current moment. A dynamic trigger threshold can be set; when the distance between the vehicle and the intersection is ≤ a preset threshold (D_threshold), the intersection identification system is then activated to prevent misguided intersections.

[0094] Figure 2 This is a schematic diagram of an optional rendering display of a reverse-guide element according to an embodiment of the present invention, such as... Figure 2 As shown, in a driving scenario where a vehicle approaches an intersection (not shown in the diagram), if the vehicle needs to maintain its middle lane when approaching the intersection, it can be determined that the left and right lanes are prohibited lanes. Therefore, within the vehicle's augmented reality head-up display (HUD) field of view, reverse guidance elements can be rendered and displayed in the left and right prohibited lanes. For example, an X-shaped sign is used in the diagram to clearly and intuitively remind the driver not to change lanes. The diagram also renders and displays an augmented reality navigation light carpet within the vehicle's own lane, along with other elements such as remaining range of 550km, Navigation Guided Pilot (NGP) being enabled, current speed limit of 40 km / h, current speed of 40 km / h, remaining 33 minutes to reach the destination, and estimated arrival time at 17:52.

[0095] Figure 3 This is a schematic diagram of an optional method for determining the target rendering position according to an embodiment of the present invention, such as... Figure 3 As shown in the figure, the method of determining the target rendering position in the vehicle coordinate system is illustrated. In this vehicle coordinate system, the horizontal axis displays distance scales of -40, 0, and 40, and the vertical axis displays distance scales of -50, 0, and 200. The vehicle coordinate system shows three lanes. The arrow in the figure indicates that the vehicle is at position (0,0) and is located in the middle lane. If the vehicle needs to stay in the middle lane when approaching an intersection, the left and right lanes can be identified as no-entry lanes. The points in the left and right no-entry lanes in the figure can represent the initial positions. Then, the vertical coordinate value of any one of the initial positions can be adjusted to obtain the target rendering position in the figure, i.e., the pentagram position.

[0096] This application combines map and perception data fusion with augmented reality technology to accurately guide elements in real-world non-drivable lanes. For example, the position, size, and number of X-shaped markers can be dynamically adjusted in real-time according to the actual road scene. It can dynamically anchor lane lines, establish perceived lane line topology relationships, and after transferring map-based lane line data to the vehicle's coordinate system, match it with the perceived lane lines. After matching, the X-shaped markers are positioned at the center of the corresponding two lane lines.

[0097] This application can maintain the visibility and stability of signs under complex driving conditions such as changes in vehicle position and bumps. Specifically, it can employ a Z-axis dynamic compensation algorithm, including setting a base height (Z_base), where Z_base = 0, i.e., the ground projection reference, and using a gear coupling compensation formula, which can be expressed as follows:

[0098] Z_{adjust}=x*tan(ΔeyePitch);

[0099] Where x can be the longitudinal distance between the intersection and the vehicle, and ΔeyePitch can be set as the difference in downward viewing angle between different gears and the base gear.

[0100] The proposed method for marking impassable lanes based on augmented reality head-up display determines impassable lanes through multi-source data fusion and uses 3D rendering technology to overlay dynamic X-shaped markings on the real road surface. The position, size, and number of X-shaped markings can be dynamically adjusted in real time according to the real road scene.

[0101] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation portals are provided for users to choose to authorize or refuse.

[0102] According to another aspect of the present invention, a vehicle augmented reality navigation device is also provided. This device can execute the vehicle augmented reality navigation method of the above embodiments. The specific implementation method and preferred application scenarios are the same as those of the above embodiments, and will not be described again here.

[0103] Figure 4 This is a schematic diagram of a vehicle augmented reality navigation device according to an embodiment of this application, such as... Figure 4 As shown, the device includes the following: an acquisition module 402 and a display module 404.

[0104] The acquisition module 402 is used to acquire intersection data along the navigation route based on the acquired lane-level navigation map data; the display module 404 is used to render and display at least one reverse guidance element within the augmented reality head-up display field of view of the vehicle in response to the intersection data meeting the preset trigger conditions. The intersection data is acquired based on the vehicle's positioning information and lane-level navigation map data, and different reverse guidance elements are mapped in different prohibited lanes. The prohibited lanes are lanes where vehicles are prohibited from driving, determined based on the intersection data and the vehicle's perception data.

[0105] The display module is also used to acquire navigation data, which includes at least the vehicle's current lane and the first light carpet control line data. The navigation data is generated based on the positioning information, the vehicle's target address, and lane-level navigation map data. The first light carpet control line data is converted from the map coordinate system corresponding to the navigation data to the vehicle's own coordinate system to obtain the second light carpet control line data. The second light carpet control line data is rendered as an augmented reality navigation light carpet and displayed within the augmented reality head-up display's field of view.

[0106] The display module is also used to obtain the intersection distance in the intersection data. The intersection distance is the distance between the vehicle and the intersection, which is determined based on the positioning information and lane-level navigation map data. In response to the intersection distance being less than or equal to a preset distance, it is determined that the intersection data meets the preset triggering conditions. In response to the intersection distance being greater than the preset distance, it is determined that the intersection data does not meet the preset triggering conditions.

[0107] The display module is further used to transform the first lane boundary data in the intersection data from the map coordinate system corresponding to the intersection data to the vehicle's own coordinate system to obtain transformed boundary data, wherein the first lane boundary data is used to represent the boundary of at least one prohibited lane; the perception data and the transformed boundary data are fused to obtain the second lane boundary data; based on the second lane boundary data, the target rendering position corresponding to at least one reverse guidance element is determined; based on the target rendering position, at least one reverse guidance element is rendered within the field of view of the augmented reality head-up display.

[0108] The display module is also used to acquire first sensor data when the vehicle acquires positioning information, and second sensor data when the vehicle acquires intersection data. The first sensor data includes at least a first heading angle, and the second sensor data includes at least a second heading angle. Based on the first heading angle and the second heading angle, the vehicle's rotation angle is determined. Based on the rotation angle, the first lane boundary data is rotated to obtain first rotation boundary data. The first rotation boundary data is transformed from the map coordinate system to the vehicle coordinate system to obtain transformed boundary data.

[0109] The second sensor data also includes: angular acceleration, which is displayed in the module and is also used to rotate the first lane boundary data based on the rotation angle to obtain the second rotated boundary data; to determine the change in heading angle based on the angular acceleration; and to rotate the second rotated boundary data based on the change in heading angle to obtain the first rotated boundary data.

[0110] The display module is also used to acquire navigation data, which includes at least: the vehicle's current location in the first lane of the road, and the navigation data is generated based on positioning information, the vehicle's target location, and map data; determining the vehicle's current location in the second lane of the road based on perception data; determining the transition boundary data as the second lane boundary data in response to a successful match between the first and second lanes; and correcting the transition boundary data to obtain the second lane boundary data in response to a failure to match between the first and second lanes.

[0111] The display module is also used to determine the initial position within at least one prohibited lane based on the second lane boundary data. The initial position is used to characterize the intersection of the straight line formed by the second lane boundary data and the centerline of the corresponding lane. Based on the initial position, the target rendering position is determined.

[0112] The display module is also used to perform at least one of the following: using the initial position as the target rendering position; adjusting the coordinate values ​​corresponding to the first coordinate axis in the initial position to obtain the target rendering position, wherein the first coordinate axis is parallel to the vehicle's driving direction.

[0113] The display module is also used to obtain the target gear position of the augmented reality head-up display device; determine the viewing angle deviation between the target gear position and the preset gear position; determine the compensation height based on the intersection distance and viewing angle deviation in the intersection data; and adjust the coordinate value corresponding to the second coordinate axis in the initial position based on the compensation height to obtain the target rendering position, wherein the second coordinate axis is perpendicular to the road where the vehicle is currently located.

[0114] Among them, at least one of the following pieces of information of the reverse guidance element is dynamically adjusted based on the real environment around the vehicle: element position, element size, and element quantity.

[0115] The reverse guidance elements include one of the following: a cross sign, an augmented reality light wall, and a horizontal line sign, wherein the horizontal line sign is parallel to the road where the vehicle is currently located.

[0116] According to another aspect of the present invention, an augmented reality head-up display device is also provided. This system can execute the vehicle augmented reality navigation method of the above embodiments. The specific implementation method and preferred application scenarios are the same as those of the above embodiments, and will not be described in detail here.

[0117] Figure 5This is a schematic diagram of an augmented reality head-up display device according to an embodiment of this application, such as... Figure 5 As shown, the device includes a memory 502 and a processor 504.

[0118] The system includes a memory 502 storing an executable program and a processor 504 for running the program. During program execution, the following actions are performed: acquiring intersection data along the navigation route based on the acquired lane-level navigation map data; rendering and displaying at least one reverse guidance element within the augmented reality head-up display field of view of the vehicle in response to the intersection data meeting a preset trigger condition; wherein the intersection data is acquired based on the vehicle's positioning information and lane-level navigation map data, different reverse guidance elements are mapped within different prohibited lanes, and prohibited lanes are lanes where vehicles are prohibited from driving, determined based on the intersection data and the vehicle's perception data; and the methods described in various embodiments of the present invention.

[0119] The aforementioned memory can refer to devices inside a computer used to store data and programs, including RAM, hard disks, etc. RAM can be used to temporarily store running programs and data, while hard disks can be used to store programs and data long-term. Memory enables the computer to read and write data and execute programs. The aforementioned processor is responsible for executing instructions in computer programs and performing data processing. It can also be responsible for controlling and executing various operations, including arithmetic operations, logical operations, and data transmission.

[0120] According to another aspect of the present invention, a vehicle is also provided, the vehicle including the above-described augmented reality head-up display device.

[0121] Embodiments of this application also provide a computer-readable storage medium including a stored executable program, wherein, when the executable program is running, it controls the device where the computer-readable storage medium is located to perform the methods of various embodiments of the present invention.

[0122] The aforementioned computer storage media can refer to the media used in computer memory to store certain discontinuous physical quantities. Computer storage media mainly include semiconductors, magnetic cores, magnetic drums, magnetic tapes, laser discs, etc. Computer-readable storage media include stored programs, which can be a set of instructions that a computer can recognize and execute, running on an electronic computer to meet certain information needs.

[0123] Embodiments of this application also provide a computer program product, including a computer program that, when executed by a processor, implements the methods of various embodiments of the present invention.

[0124] The aforementioned computer program products can refer to software programs that have been written, tested, and released, and can run on computers or other devices. Computer program products can include application programs, operating systems, utility software, etc., used to achieve specific functions or solve specific problems.

[0125] Embodiments of this application also provide a computer program product, including a non-volatile computer-readable storage medium for storing a computer program that, when executed by a processor, implements the methods in various embodiments of the present invention.

[0126] The aforementioned non-volatile computer-readable storage medium can refer to a medium for storing data. Non-volatile computer-readable storage media can retain data without loss when power is off and can be used to store long-term data, such as operating systems, applications, and user files. Non-volatile storage media can include hard disk drives, solid-state drives, optical disks, and flash memory storage devices, etc.

[0127] Embodiments of this application also provide a computer program that, when executed by a processor, implements the methods described in the various embodiments of the present invention.

[0128] The aforementioned computer program can refer to a set of instructions used to tell the computer to perform specific tasks or operations. Computer programs can be written by programmers using specific programming languages ​​and can include algorithms, data structures, logic, and control flow. Computer programs can be used for a variety of purposes, including application software, operating systems, etc.

[0129] In the above embodiments of the present invention, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0130] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For instance, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0131] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0132] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0133] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0134] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A vehicle augmented reality navigation method, characterized in that, include: Based on the acquired lane-level navigation map data, the intersection data of the navigation route is obtained, wherein the intersection data is used to represent the intersection or fork point information in the vehicle's driving path. In response to the intersection data meeting a preset trigger condition, at least one reverse guidance element is rendered and displayed within the augmented reality head-up display (HUD) field of view of the vehicle. The intersection data is obtained based on the vehicle's positioning information and the lane-level navigation map data. Different reverse guidance elements are mapped within different prohibited lanes. The prohibited lanes are lanes where the vehicle is prohibited from driving, determined based on the intersection data and the vehicle's perception data. The preset trigger condition represents a pre-defined condition that triggers the display of the reverse guidance element when the distance between the vehicle and the preset intersection is less than or equal to a threshold. The reverse guidance element represents a visual cue displayed in the augmented reality HUD, indicating to the driver a lane area inconsistent with the navigation route.

2. The method according to claim 1, characterized in that, The method further includes: Acquire navigation data, wherein the navigation data includes at least: the lane where the vehicle is currently located, and the first light carpet control line data, and the navigation data is generated based on the positioning information, the vehicle's target address, and the lane-level navigation map data; The first light carpet control line data is converted from the map coordinate system corresponding to the navigation data to the vehicle's own coordinate system to obtain the second light carpet control line data; The second light carpet control line data is rendered into an augmented reality navigation light carpet, and the augmented reality navigation light carpet is displayed within the field of view of the augmented reality head-up display.

3. The method according to claim 1 or 2, characterized in that, The method further includes: Obtain the intersection distance from the intersection data, wherein the intersection distance is the distance between the vehicle and the intersection, and the intersection distance is the distance determined based on the positioning information and the lane-level navigation map data; In response to the intersection distance being less than or equal to a preset distance, it is determined that the intersection data meets the preset triggering condition; In response to the fact that the distance to the intersection is greater than the preset distance, it is determined that the intersection data does not meet the preset triggering condition.

4. The method according to claim 1 or 2, characterized in that, The method further includes: The first lane boundary data in the intersection data is transformed from the map coordinate system corresponding to the intersection data to the vehicle's own coordinate system to obtain transformed boundary data, wherein the first lane boundary data is used to represent the boundary of at least one prohibited lane. The sensed data and the conversion boundary data are fused to obtain the second lane boundary data; Based on the second lane boundary data, determine the target rendering position corresponding to the at least one reverse guidance element; Based on the target rendering position, the at least one reverse guide element is rendered into the augmented reality head-up display field of view.

5. The method according to claim 4, characterized in that, The step of transforming the first lane boundary data in the intersection data from the map coordinate system corresponding to the intersection data to the vehicle's own coordinate system to obtain transformed boundary data includes: The system acquires first sensor data when the vehicle acquires the positioning information, and second sensor data when the vehicle acquires the intersection data, wherein the first sensor data includes at least a first heading angle, and the second sensor data includes at least a second heading angle. The rotation angle of the vehicle is determined based on the first heading angle and the second heading angle; The first lane boundary data is rotated based on the rotation angle to obtain the first rotated boundary data. The first rotation boundary data is transformed from the map coordinate system to the vehicle coordinate system to obtain the transformed boundary data.

6. The method according to claim 5, characterized in that, The second sensor data also includes angular acceleration, wherein rotating the first lane boundary data based on the rotation angle to obtain the first rotated boundary data includes: The first lane boundary data is rotated based on the rotation angle to obtain the second rotated boundary data. Based on the angular acceleration, determine the change in heading angle; The second rotation boundary data is rotated based on the change in heading angle to obtain the first rotation boundary data.

7. The method according to claim 4, characterized in that, The process of fusing the perceived data and the conversion boundary data to obtain the second lane boundary data includes: Obtain navigation data, wherein the navigation data includes at least: the vehicle is currently located in the first lane of the road, and the navigation data is generated based on the positioning information, the vehicle's target location, and the map data; Based on the perception data, it is determined that the vehicle is currently located in the second lane of the road; In response to a successful match between the first lane and the second lane, the transition boundary data is determined to be the boundary data of the second lane; In response to the failure of the first lane to match the second lane, the conversion boundary data is corrected to obtain the second lane boundary data.

8. The method according to claim 4, characterized in that, Determining the target rendering position corresponding to the at least one reverse guidance element based on the second lane boundary data includes: Based on the second lane boundary data, an initial position is determined within at least one prohibited lane, wherein the initial position is used to characterize the intersection point between the straight line formed by the second lane boundary data and the centerline of the corresponding lane; Based on the initial position, the target rendering position is determined.

9. The method according to claim 8, characterized in that, Determining the target rendering position based on the initial position includes at least one of the following: The initial position is used as the target rendering position; The coordinate values ​​corresponding to the first coordinate axis in the initial position are adjusted to obtain the target rendering position, wherein the first coordinate axis is parallel to the vehicle's driving direction.

10. The method according to claim 8, characterized in that, Determining the target rendering position based on the initial position includes: Obtain the current target gear level of the augmented reality head-up display device; Determine the viewing angle deviation between the target gear and the preset gear; The compensation height is determined based on the intersection distance and the viewing angle deviation in the intersection data; The coordinate values ​​corresponding to the second coordinate axis in the initial position are adjusted based on the compensation height to obtain the target rendering position, wherein the second coordinate axis is perpendicular to the road where the vehicle is currently located.

11. The method according to claim 1 or 2, characterized in that, The following information of the reverse guidance element is dynamically adjusted based on the real environment around the vehicle: element position, element size, and element quantity.

12. The method according to claim 1 or 2, characterized in that, The reverse guidance element includes one of the following: a cross-shaped sign, an augmented reality light wall, and a horizontal line sign, wherein the horizontal line sign is parallel to the road where the vehicle is currently located.

13. An augmented reality head-up display device, characterized in that, include: Memory, which stores executable programs; A processor for running the program, wherein the program, when running, performs the method according to any one of claims 1 to 12.

14. A vehicle, characterized in that, include: The augmented reality head-up display device according to claim 13.

Images