Augmented reality navigation method for vehicle, augmented reality head-up display device and vehicle

Abstract

The invention discloses a vehicle augmented reality navigation method, an augmented reality head-up display device and a vehicle. The method comprises the following steps: rendering and displaying a first navigation light blanket on an augmented reality head-up display interface; a lane change signal is obtained in response to path planning of the vehicle, a lane change line segment in the first navigation light blanket is redrawn, a second navigation local light blanket is obtained, the terminal end of the second navigation local light blanket is located in the middle of the target lane, and the starting end of the second navigation local light blanket is located at the perigee position; switching the lane change line segment of the first navigation light blanket into a second navigation local light blanket in the augmented reality head-up display interface for rendering display; in the process that the vehicle runs to the target lane according to the second navigation local light blanket, the terminal end of the second navigation local light blanket is always located in the middle of the target lane. According to the invention, the technical problem of poor guidance and stability of a navigation light blanket in a lane changing or intersection lane dislocation scene in the prior art is solved.

Description

Technical Field

[0001] The present invention relates to the fields of vehicles, augmented reality, and navigation technology, and in particular to a vehicle augmented reality navigation method, an augmented reality head-up display device, and a vehicle. Background Art

[0002] With the rapid development of intelligent driving and augmented reality technology, AR HUD (Augmented Reality Head-Up Display) has become an indispensable part of modern cars. AR HUD can directly project key driving information within the driver's field of view, reducing the driver's gaze shift and improving driving safety. Figure 1a In the lane change scenario shown, the navigation light carpet on the AR HUD is derived from the curve between the starting lane and the target lane. However, due to the field of view limitation of the augmented reality device, the driver only sees the navigation light carpet starting from a certain distance in front of the vehicle (for example, 20 meters away) through the augmented reality head-up display, and cannot accurately understand the information of lane change or lane misalignment at the intersection. Figure 1b The current technical solution is to compress the curve from the starting lane to the target lane and compress the curve into the range of the augmented reality head-up display. However, the compression solution may bring the following Figure 1c The bow phenomenon shown causes the distal end of the navigation light carpet to be unable to fit the target lane, resulting in poor guidance and stability in lane change or lane misalignment at intersections.

[0003] To address the above-mentioned problems, no effective solutions have been proposed so far. Summary of the Invention

[0004] Embodiments of the present invention provide a vehicle augmented reality navigation method, an augmented reality head-up display device, and a vehicle, to at least solve the technical problem in the related art of poor guidance and stability of the navigation light carpet in lane change or lane misalignment at intersections.

[0005] According to one aspect of an embodiment of the present invention, a vehicle augmented reality navigation method is provided, comprising: rendering and displaying a first navigation light carpet on an augmented reality head-up display interface; in response to obtaining a lane change signal according to the vehicle's path planning, redrawing the lane change line segment in the first navigation light carpet to obtain a second navigation local light carpet, wherein the lane change signal is used to indicate that the lane in front of the vehicle has changed, the end end of the second navigation local light carpet is located in the middle of a target lane, and the target lane is the lane after the lane in front of the vehicle has changed according to the path planning, and the starting end of the second navigation local light carpet is located in the middle of the target lane on the augmented reality head-up display interface. the perigee position; in the augmented reality head-up display interface, the lane change line segment of the first navigation light carpet is switched to the second navigation local light carpet for rendering and display; wherein, in the process of the vehicle driving to the target lane according to the second navigation local light carpet, the end point of the second navigation local light carpet is always located in the middle of the target lane, and the second navigation local light carpet includes a starting segment, a first curved segment and a second curved segment, one end of the starting segment is the starting end of the second navigation local light carpet, the other end of the starting segment is connected to one end of the first curved segment, the other end of the first curved segment is connected to one end of the second curved segment, and the other end of the second curved segment is the end point of the second navigation local light carpet.

[0006] Furthermore, the lane change signal includes: a lane change starting position and a lane change end position; the lane change line segment in the first navigation light carpet is redrawn to obtain a second navigation local light carpet, including: determining a first control point position based on the perigee position and the lane change starting position, wherein the first control point position is used to control the line segment boundary range and curvature of the first curved segment, and the coordinate values ​​of the first control point position and the perigee position in a preset direction in the vehicle's own vehicle coordinate system are consistent, the preset direction is perpendicular to the longitudinal symmetry plane of the vehicle, and the lane change line segment is a line segment between the lane change starting position and the lane change end position; determining a second control point position based on the lane change end position, wherein the second control point position is used to control the line segment boundary range and curvature of the second curved segment, and the coordinate values ​​of the second control point position and the lane change end position in the preset direction in the own vehicle coordinate system are consistent; generating a second navigation local light carpet based on the perigee position, the lane change end position, the first control point position, and the second control point position.

[0007] Further, based on the perigee position and the lane change starting position, the position of the first control point is determined, including: determining a first distance between the perigee position and the current position of the vehicle in the current orientation of the vehicle head, and a second distance between the lane change starting position and the current position of the vehicle in the current orientation of the vehicle head, wherein the current orientation of the vehicle head is parallel to the longitudinal symmetry plane of the vehicle; in response to the first distance being greater than the second distance, the position of the first control point is determined based on a first preset distance value between the perigee position and the current orientation of the vehicle head; in response to the first distance being less than or equal to the second distance, the position of the first control point is determined based on the perigee position, the distance between the perigee position and the lane change starting position in the current orientation of the vehicle head, and the first preset distance value.

[0008] Furthermore, based on the lane change end position, the second control point position is determined, including: determining the second control point position based on the lane change end position and a second preset distance value on the current direction of the vehicle head, wherein the current direction of the vehicle head is parallel to the longitudinal symmetry plane of the vehicle.

[0009] Furthermore, after redrawing the lane change line segments in the first navigation light carpet to obtain the second navigation local light carpet, the above method also includes: splicing the second navigation local light carpet with the non-lane change line segments in the first navigation light carpet to obtain a spliced ​​navigation light carpet, wherein the non-lane change line segments are the line segments in the first navigation light carpet other than the lane change line segments.

[0010] Furthermore, the above method also includes: sending the vehicle's positioning information to the server; receiving first lane-level navigation data returned by the server, wherein the first lane-level navigation data at least includes: first light carpet control line data, the first lane-level navigation data is generated based on the positioning information, the vehicle's target address and lane-level map data; converting the first light carpet control line data from the map coordinate system corresponding to the first lane-level navigation data to the vehicle's own vehicle coordinate system to obtain first converted control line data; rendering the first converted control line data as a first navigation light carpet.

[0011] Furthermore, the first lane-level navigation data also includes: preset position information of the current lane, the current lane is the lane in which the vehicle is currently traveling, the current lane is determined based on the positioning information, and the first lane-level navigation data is generated based on the preset position information, the target address and the lane-level map data; rendering the first conversion control line data into a first navigation light carpet, including: in response to obtaining a lane change signal, based on the positioning information and the preset position information, correcting the first conversion control line data to obtain corrected control line data; rendering the corrected control line data into the first navigation light carpet.

[0012] Furthermore, based on the positioning information and the preset position information, the first conversion control line data is corrected to obtain corrected control line data, including: converting the positioning information from the geographic coordinate system to the vehicle coordinate system to obtain the first vehicle position, and converting the preset position information from the map coordinate system corresponding to the lane-level map data to the vehicle coordinate system to obtain the second vehicle position; based on the deviation between the first vehicle position and the second vehicle position, the first conversion control line data is corrected to obtain corrected control line data.

[0013] Furthermore, the above method also includes: in response to detecting that the vehicle drives into the target lane, or obtaining a lane change end signal according to the vehicle's path planning, re-rendering the first conversion control line data into a first navigation light carpet, and displaying the first navigation light carpet in the augmented reality head-up display interface.

[0014] Furthermore, the above method also includes: converting the near-point position from the vehicle coordinate system to the map coordinate system corresponding to the lane-level map data to obtain a converted position; sending the converted position to the server, and receiving second lane-level navigation data returned by the server, wherein the second lane-level navigation data at least includes: second light carpet control line data, wherein the second light carpet control line data is generated based on the converted position, the target address of the vehicle and the lane-level map data; converting the second light carpet control line data from the map coordinate system to the vehicle coordinate system to obtain second converted control line data; rendering the second converted control line data as a third navigation light carpet, and displaying the third navigation light carpet in the augmented reality head-up display interface.

[0015] According to another aspect of an embodiment of the present invention, an augmented reality head-up display device is provided, including: a memory storing an executable program; and a processor for running the program, wherein the program executes any one of the methods in the above embodiments when running.

[0016] According to another aspect of an embodiment of the present invention, a vehicle is provided, comprising: the augmented reality head-up display device in the above embodiment.

[0017] According to another aspect of an embodiment of the present invention, an electronic device is provided, including: a memory storing an executable program; and a processor for running the program, wherein the program executes the methods of various embodiments of the present invention when running.

[0018] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is provided. The computer-readable storage medium includes a stored executable program, wherein when the executable program is running, the device where the computer-readable storage medium is located is controlled to execute the methods in various embodiments of the present invention.

[0019] According to another aspect of an embodiment of the present invention, a computer program product is provided, including a computer program. When the computer program is executed by a processor, the method in each embodiment of the present invention is implemented.

[0020] According to another aspect of an embodiment of the present invention, a computer program product is provided, including a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method in each embodiment of the present invention is implemented.

[0021] According to another aspect of the embodiments of the present invention, a computer program is provided. When the computer program is executed by a processor, the methods in various embodiments of the present invention are implemented.

[0022] In an embodiment of the present invention, a first navigation light carpet is rendered and displayed on an augmented reality head-up display interface; in response to a lane change signal obtained according to the vehicle's path planning, the lane change line segments in the first navigation light carpet are redrawn to obtain a second navigation local light carpet; and the lane change line segments of the first navigation light carpet are switched to the second navigation local light carpet for rendering and display within the augmented reality head-up display interface, thereby showing the driver a lane change trend, such as a lane change trend or a lane misalignment trend at an intersection, by displaying the second navigation local light carpet. It is easy to notice that the second navigation local light carpet is obtained by redrawing the lane change line segment in the first navigation light carpet, so that the starting end of the second navigation local light carpet is located at the perigee position of the augmented reality head-up display interface, and the end end of the second navigation local light carpet is located in the middle of the target lane. Moreover, in the process of the vehicle driving to the target lane according to the second navigation local light carpet, the end end of the second navigation local light carpet is always located in the middle of the target lane, avoiding the back bow phenomenon of the second navigation local light carpet, and ensuring that the end end of the second navigation local light carpet fits the target lane, thereby improving the guidance and stability of the navigation light carpet in lane changing or lane misalignment at intersections, and solving the problem of poor guidance and stability of the navigation light carpet in lane changing or lane misalignment at intersections caused by the use of a compression scheme. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0024] Figure 1a is a schematic diagram of a lane change scenario according to the related art;

[0025] Figure 1b A schematic diagram of expressing a lane change trend through a navigation light carpet in a lane change scenario according to related art;

[0026] Figure 1c This is a schematic diagram of a bow phenomenon caused by compression processing of a navigation light carpet in a lane change scenario according to related art;

[0027] Figure 2 is a flow chart of a vehicle augmented reality navigation method according to an embodiment of the present invention;

[0028] Figure 3a is a schematic diagram of a control point position and a second navigation local light carpet before a vehicle changes lanes in a lane change scenario according to an embodiment of the present invention;

[0029] Figure 3b is a schematic diagram of a control point position and a second navigation local light carpet during a vehicle starting to change lanes in a lane change scenario according to an embodiment of the present invention;

[0030] Figure 3c is a schematic diagram of a first navigation light carpet after a vehicle completes lane change in a lane change scenario according to an embodiment of the present invention;

[0031] Figure 4a is a schematic diagram of a control point position and a second navigation local light blanket before a vehicle changes lanes in a lane misalignment scenario at an intersection according to an embodiment of the present invention;

[0032] Figure 4b is a schematic diagram of a control point position and a second navigation local light blanket during a vehicle lane change at an intersection with lane misalignment according to an embodiment of the present invention;

[0033] Figure 4c 2 is a schematic diagram of a first navigation light carpet after a vehicle completes lane change in a lane misalignment at an intersection according to an embodiment of the present invention;

[0034] Figure 5 is a schematic diagram of determining the position of a control point in a vehicle coordinate system according to an embodiment of the present invention;

[0035] Figure 6a is a schematic diagram of first lane-level navigation data when a server binds a vehicle to a preset position of a lane according to an embodiment of the present invention;

[0036] Figure 6b is a schematic diagram of rendering a first navigation light carpet by an augmented reality head-up display device according to an embodiment of the present invention;

[0037] Figure 7 3 is a schematic diagram of the swinging changes of a navigation light carpet according to an embodiment of the present invention. DETAILED DESCRIPTION

[0038] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0039] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0040] According to an embodiment of the present invention, a vehicle augmented reality navigation method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0041] An embodiment of the present application provides a vehicle augmented reality navigation method. The method can be used to provide augmented reality navigation functions for preset application scenarios. The above-mentioned preset application scenarios may include the following scenarios in the vehicle field: commuting manual or automatic driving scenarios, artificial intelligence (AI) assisted or automatic driving scenarios for family cars, and intelligent navigation assistance (Navigation Guided Pilot, NGP) scenarios in urban areas or high-speed areas. In addition, the above-mentioned preset application scenarios may also include but are not limited to: lane changing or lane dislocation scenarios of manually driven trucks, intelligent driving trucks or unmanned trucks in the field of logistics and transportation, lane changing or lane dislocation scenarios of manually driven agricultural vehicles or automatic driving agricultural vehicles in the field of agricultural machinery, and lane changing or lane dislocation scenarios of intelligent robots (such as cleaning robots, service robots, delivery robots, etc.).

[0042] When the above-mentioned preset application scenarios are scenarios in fields other than the vehicle field, those skilled in the art should be able to understand that the vehicle in the above-mentioned vehicle augmented reality navigation method can be replaced with other objects (such as agricultural machinery, drones, robots, etc.), and accordingly, the vehicle augmented reality navigation method can be replaced with augmented reality navigation methods related to other objects. On this basis, in the embodiments of this application, the specific implementation methods of the above-mentioned vehicle augmented reality navigation method are exemplified by taking the augmented reality navigation field as an example.

[0043] It should be noted that, since the vehicle augmented reality navigation method in this application is mainly used to assist the driver in driving the vehicle, the above-mentioned vehicle augmented reality navigation method is mainly based on manual driving scenarios.

[0044] Figure 2 is a flow chart of a vehicle augmented reality navigation method according to an embodiment of the present invention. Figure 2 As shown, the method includes the following steps:

[0045] Step S202: Render and display a first navigation light carpet on the augmented reality head-up display interface.

[0046] The aforementioned augmented reality head-up display (AR-HUD) refers to the visual range of virtual information visible to the driver through an ARHUD (Augmented Reality Head-Up Display, or AR-HUD). It can typically be viewed as a virtual screen projected by the ARHUD. Generally, an AR HUD can render and display some driver-assistive information within the HUD's field of view, such as a navigation light carpet to assist the driver in navigating the vehicle. Therefore, accurately presenting driver-assistive information within the AR HUD plays a crucial role in ensuring driver safety and stability while driving.

[0047] The above-mentioned first navigation light carpet can be rendered and displayed within the augmented reality head-up display interface. It is an element that needs to be combined with the real scene to be accurately observed. The first navigation light carpet fits the real scene and can be presented to the driver in a perspective manner. In order to facilitate the driver to accurately know the driving path, the lines, colors and brightness of the first navigation light carpet will change in real time according to the real scene. For example, when the vehicle is driving straight, the first navigation light carpet will maintain a straight extension to help the driver maintain the center position of the lane and provide lane keeping assistance; when the vehicle is about to change lanes, the light carpet will bend and fit the changed lane to provide the driver with clear lane change guidance. The presentation of the first navigation light carpet may include but is not limited to: virtual light carpet, dynamic arrows, etc.

[0048] In an optional solution of this embodiment, during the driving process of the vehicle, based on the information of the vehicle driving environment perceived by the vehicle's sensors (such as cameras, radars, etc.), and combined with the vehicle's destination address (i.e., the address of the vehicle's destination), the vehicle can be planned in real time to obtain first lane-level navigation data. That is, the first lane-level navigation data may include but is not limited to: lane line data, lane light carpet control line data, lane road type data, vehicle lane change data, etc., wherein the light carpet control line data may refer to a series of coordinate points used to describe the shape and position of the road the vehicle is currently traveling. Further, based on the light carpet control line data, a first navigation light carpet can be rendered on the augmented reality head-up display interface, and the first navigation light carpet can be displayed in the augmented reality head-up display interface.

[0049] In another optional solution of this embodiment, while the vehicle is traveling, the vehicle can quickly determine its current location through a positioning system, such as GPS (Global Positioning System), Beidou Navigation System, etc., that is, obtain the vehicle's positioning information. The server generates first lane-level navigation data based on the positioning information, the vehicle's destination address, and lane-level map data. After receiving the first lane-level navigation data sent by the server, the augmented reality head-up display device can render it as a first navigation light carpet on the augmented reality head-up display interface and display the first navigation light carpet within the augmented reality head-up display interface. The above-mentioned lane-level map data can be map data of the road currently traveled by the vehicle provided by the server, and can be high-precision map data or low-precision map data, as long as it contains lane line data, lane road type data, etc. Different map data can be provided by different manufacturers, and the server can communicate with different manufacturers to obtain map data provided by these manufacturers.

[0050] It should be noted that considering that GPS may be affected by signal interference, resulting in positioning errors, in order to ensure the accuracy of the determined positioning information, various sensors on the vehicle, such as cameras, radars, lidars and other sensors, can also be used to perceive the environmental information near the vehicle in real time, such as road features, lane lines, traffic signs, the positions of surrounding vehicles and pedestrians, etc., and then use the perceived environmental information to adjust the vehicle position located by GPS, so as to obtain the accurate current position of the vehicle and construct accurate positioning information.

[0051] Step S204, in response to obtaining a lane change signal according to the path planning of the vehicle, the lane change line segment in the first navigation light carpet is redrawn to obtain a second navigation local light carpet, wherein the lane change signal is used to represent a change in the lane in front of the vehicle, and the end end of the second navigation local light carpet is located in the middle of the target lane, and the target lane is the lane after the lane in front of the vehicle is changed according to the path planning, and the starting end of the second navigation local light carpet is located at the perigee position of the augmented reality head-up display interface.

[0052] The above-mentioned path planning can be a driving path planned by the vehicle based on the sensed data, or it can be a driving path planned by the server based on the vehicle's positioning information. The above-mentioned lane change signal can be a signal that determines that the lane in front of the vehicle has changed based on the vehicle's path planning. The lane change here can refer to a change in the vehicle's driving lane, that is, the vehicle needs to change lanes (for example, the vehicle changes from the current Mth lane to the Nth lane), or the lane change can also refer to a lane misalignment at an intersection in front of the vehicle (for example, the misaligned lane is located to the right of the current lane, and the vehicle is equivalent to needing to drive to the right front, similar to a vehicle changing lanes). The above-mentioned target lane can be the lane that the vehicle needs to drive in after the lane in front of the vehicle changes. For example, if the vehicle changes from the current Mth lane to the Nth lane, the target lane can be the Nth lane. If there is a lane misalignment at an intersection in front of the vehicle, and the misaligned lane is located to the right of the current lane, the target lane can be the right front lane of the intersection, but is not limited to this.

[0053] To facilitate the use of the navigation light carpet to guide drivers through lane changes or maneuvering, lane change signals can include a lane change starting point and a lane change ending point. These two locations can be determined based on a specific strategy and the actual environment. Typically, the lane change starting point is located in the middle of the vehicle's current lane, and the lane change ending point is located in the middle of the target lane. For example, if the target lane is a long solid line, the distance between the two locations parallel to the vehicle's longitudinal plane of symmetry is first fixed (e.g., 60 meters). The lane change guide line between the two locations must then ensure that it does not intersect the long solid line, thereby determining the two locations. It should be noted that the longitudinal plane of symmetry can be a plane passing through the vehicle's centerline and perpendicular to the ground, thereby dividing the vehicle into two symmetrical parts. For another example, if the target lane is a turning lane, the lane change ending point is first determined based on the target lane to ensure that the vehicle can turn into the target lane normally while complying with traffic regulations. The lane change starting point can then be determined based on the distance between the two locations parallel to the vehicle's longitudinal plane of symmetry.

[0054] The lane change line segment mentioned above can be the line segment between the lane change starting point and the lane change end point in the first navigation light carpet. In an embodiment of the present application, the lane change line segment can be redrawn to generate the second navigation local light carpet, rather than compressed. This redrawing ensures that the second navigation local light carpet can both express the trend of lane change or lane misalignment and overlap with the target lane. In other words, the second navigation local light carpet has higher guidance and stability.

[0055] The aforementioned perigee position can be the starting point of the navigation light carpet visible to the driver in the augmented reality head-up display (AR HUD). The perigee position can include two scenarios: the first refers to the midpoint of the lower edge of the AR HUD in the vehicle coordinate system; the second refers to the point on the line segment where the vehicle's longitudinal symmetry plane intersects the AR HUD, located at the lower edge of the AR HUD, in the vehicle coordinate system. To ensure that the driver can see the starting point of the lane change in the AR HUD, the starting point of the redrawn second navigation partial light carpet must be located at the perigee position. Furthermore, to ensure that the navigation light carpet seen by the driver aligns with the target lane, the ending point of the redrawn second navigation partial light carpet must be located in the middle of the target lane. The starting point can be the end closest to the vehicle visible to the driver, and the ending point can be the end farthest from the vehicle visible to the driver. Since the navigation light carpet typically has a certain width, the starting and ending points of the second navigation partial light carpet can be a line segment. It should be noted that due to the limitations of the augmented reality head-up display interface, the driver may not be able to see the end point even if he can see the starting point. Sometimes it takes a while of driving until the augmented reality head-up display interface moves to the end point of the lane change before the driver can see the end point in the augmented reality head-up display interface.

[0056] Optionally, considering that the lane ahead of the vehicle changes, it can usually be considered as an S-shaped curve, such as Figure 1a and Figure 1bAs shown, the second navigation local light carpet can include two curved segments to form an S-shaped curve. In order to avoid the occurrence of reverse bowing, the curvature and line segment boundary range of the two curved segments can be controlled based on the perigee position and the lane change end position, thereby ensuring that the connection between the two curved segments is smooth and there is no reverse bowing. In other words, the driver will not experience sudden changes in the direction of the vehicle's head or even U-turns during driving, so that the vehicle can travel smoothly and stably from the first curved segment to the second curved segment. Furthermore, considering that there is usually a distance between the current position of the vehicle and the starting position of the lane change, in order to reflect this trend in the augmented reality head-up display interface, that is, it is necessary to drive forward for a while before starting to change lanes, the second navigation local light carpet can also include a starting segment, the starting end of which can be a straight line segment parallel to the longitudinal symmetry plane of the vehicle. Therefore, the second navigation local light carpet finally redrawn may include a starting segment, a first curved segment and a second curved segment, one end of the starting segment is the starting end of the second navigation local light carpet, the other end of the starting segment is connected to one end of the first curved segment, the other end of the first curved segment is connected to one end of the second curved segment, and the other end of the second curved segment is the end end of the second navigation local light carpet. It should be noted that, considering that only the lane change line segment in the first navigation light carpet is redrawn, and there are other line segments after the lane change line segment, in order to ensure that the second navigation local light carpet and other line segments in the first navigation light carpet appear to be smoothly connected visually, the second navigation local light carpet may also include an ending segment, which may also be a straight line segment, parallel to the longitudinal symmetry plane of the vehicle after the lane change. At this time, the second navigation local light carpet may include a starting segment, a first curved segment, a second curved segment and an ending segment. One end of the starting segment is the starting end of the second navigation local light carpet, and the other end of the starting segment is connected to one end of the first curved segment, the other end of the first curved segment is connected to one end of the second curved segment, and the other end of the second curved segment is connected to one end of the ending segment. The other end of the ending segment is the end end of the second navigation local light carpet.

[0057] In an optional solution of this embodiment, while the driver is driving the vehicle according to the first navigation light carpet, he can determine whether the lane in front of the vehicle has changed based on the vehicle's real-time positioning information and pre-planned path planning. If it is determined that the lane in front of the vehicle has changed, a lane change signal can be generated at this time to inform the augmented reality head-up display device that the lane change line segment in the first navigation light carpet needs to be redrawn.

[0058] In another optional solution of this embodiment, in order to save the vehicle's computing resources, the vehicle only needs to upload real-time positioning information to the server, and the server determines whether the lane in front of the vehicle has changed. If it is determined that the lane in front of the vehicle has changed, the server can send a lane change signal to the vehicle, so that the augmented reality head-up display device can determine that the lane change line segment in the first navigation light carpet needs to be redrawn after receiving the lane change signal.

[0059] Furthermore, since only part of the line segments in the first navigation light carpet are redrawn, the light carpet control line data will not change. Therefore, the redrawing here is to redraw the light carpet, and there is no step of rendering the light carpet control line data into a navigation light carpet. In the process of redrawing the lane change line segments, a suitable drawing method can be used as needed, for example, a spline curve, a Bezier curve, a Catmull-Röhm curve and other curve construction methods can be used to obtain a second navigation local light carpet, but it is not limited to this. Any drawing method that can obtain a second navigation local light carpet that meets the above requirements can be used, that is, as long as the redrawn second navigation local light carpet includes a starting segment, a first curved segment and a second curved segment, and one end of the starting segment is located at the perigee position, the other end of the starting segment is connected to the first curved end, the other end of the first curved segment is connected to the second curved segment, and the other end of the second curved segment is located in the middle of the target lane.

[0060] Optionally, considering that the third-order Bezier curve is similar to an S-shaped curve, in an embodiment of the present application, a third-order Bezier curve construction method is used to obtain the second navigation local light carpet, that is, in addition to the perigee position and the lane change end position, two control point positions need to be determined, so that a Bezier curve is generated based on the perigee position, the lane change end position and the two control point positions as the second navigation local light carpet. Considering that the Bezier curve is a complete curve, in order to reflect the starting segment and the ending segment, a fixed length can be added when determining the two control point positions, so that the beginning and end of the Bezier curve appear as a straight line. In addition, considering that there is a certain distance between the perigee position and the lane change starting position, one control point position can be determined based on the perigee position and the lane change starting position, and another control point position can be determined based on the lane change end position.

[0061] Step S206 : Switching the lane change line segment of the first navigation light carpet to the second navigation partial light carpet for rendering and display in the augmented reality head-up display interface.

[0062] In an optional scheme of this embodiment, after obtaining the lane change signal, the lane change line segment of the first navigation light carpet can be switched to the second navigation local light carpet, that is, the lane change line segment is not displayed in the augmented reality head-up display interface, but the second navigation local light carpet is displayed. At this time, for the driver, what is rendered and displayed in the augmented reality head-up display interface is the second navigation local light carpet, as well as the line segment after the lane change end position in the first navigation light carpet, that is, the non-lane change line segment in the first navigation light carpet, so that the driver can clearly see the trend of lane change or lane misalignment at the intersection, and clearly understand the specific position of the target lane.

[0063] Furthermore, in order to ensure that the navigation light carpet seen by the driver is a complete light carpet, the second navigation partial light carpet can be spliced ​​with the non-lane change line segment in the first navigation light carpet to obtain a complete spliced ​​navigation light carpet, and then the first navigation light carpet can be switched to the spliced ​​navigation light carpet in the augmented reality head-up display interface, that is, the spliced ​​navigation light carpet can be directly displayed in the augmented reality head-up display interface.

[0064] Optionally, during the process of the vehicle driving to the target lane according to the second navigation partial light carpet, the end point of the second navigation partial light carpet is always located in the middle of the target lane.

[0065] In an alternative embodiment of this embodiment, as the vehicle moves toward the target lane, its position changes perpendicular to the vehicle's longitudinal plane of symmetry, causing the perigee position to also change perpendicular to the vehicle's longitudinal plane of symmetry. Therefore, the second navigation local light carpet can be redrawn based on the changed perigee position, ensuring that the endpoint of the second navigation local light carpet remains in the center of the target lane.

[0066] It should be noted that the aforementioned vehicle augmented reality navigation method is applicable to scenarios where the driver is actively driving the vehicle, rather than scenarios where the vehicle is autonomously driving. In contrast, in this application, the first navigation light carpet and the second navigation local light carpet generated by the augmented reality head-up display device are only used as auxiliary prompts. The user does not actually need to strictly follow the first navigation light carpet or the second navigation local light carpet to drive the vehicle. The corresponding augmented reality head-up display device can adjust the second navigation local light carpet in real time according to the real-time position of the vehicle. Based on this, the vehicle augmented reality navigation method provided by the embodiment of the present application is different from the autonomous driving technology mentioned in the periodic acquisition of navigation data and perception data according to a fixed route, and the two cannot be generalized.

[0067] It should also be noted that this application uses an augmented reality head-up display (AR-HUD) rather than a vehicle screen for information presentation. Although both displaying navigation information on a vehicle screen and displaying it via an ARHUD can provide navigation guidance, it should be noted that displaying navigation information via an AR HUD is not simply replacing the same navigation information with a different display element. Instead, the two differ in their fundamental logic and effects in generating and implementing the displayed navigation information. These differences determine their respective characteristics and application scenarios.

[0068] The core of AR HUD lies in the integration of the navigation light carpet with the actual road. This integration combines the perceived lane markings ahead, surrounding vehicles, non-motorized vehicles, pedestrians, and animals, and other necessary information with the real environment, presenting it on the windshield in front of the driver's line of sight. Effectively, this alignment of virtual information with the real world maintains a high degree of consistency, providing users with an intuitive and immersive driving assistance experience.

[0069] In contrast, the navigation information displayed on the car screen includes the navigation route (not the navigation light carpet), the vehicle itself, and surrounding vehicles, and other virtual information rendered through animation on a fixed display screen inside the car. The navigation route and virtual information do not need to be spatially integrated with the external environment. Their main purpose is to provide users with a general understanding of the surrounding environment, and there is no high requirement for accuracy.

[0070] Table 1

[0071] Information display location AR HUD Car screen Does the virtual information match the actual road? yes no Is it necessary to predict the position of the vehicle in front in real time? yes no Is there a problem with the navigation light carpet being out of frame? yes no

[0072] Table 1 shows the difference between navigation information displayed by an AR HUD while driving and navigation information displayed on the vehicle's onboard screen. As shown in Table 1, the fundamental difference between AR HUD and traditional onboard screen navigation display functions is whether the virtual information aligns with the actual road. Due to its augmented reality capabilities, AR HUD can overlay navigation information directly within the driver's field of view. This displays a navigation light carpet on the road the vehicle is currently on, aligning it with the actual road. This allows the user to clearly understand how to navigate the road ahead, making it very intuitive. However, navigation on the onboard screen does not require direct visual integration with the road and does not display navigation information via a light carpet. Instead, it color-codes the available roads ahead, indicating to the user which roads are passable. This allows the user to know which roads are available, leaving the user to decide which road to take. In addition, AR HUD can display information such as the lane lines ahead within a certain range ahead, motor vehicles, non-motor vehicles, pedestrians or animals in the surrounding environment, and intuitively prompt users of external factors that may affect driving operations. Especially in environments with low visibility, users may not notice what the surrounding environment is like. This information can greatly help users make decisions to avoid traffic accidents. However, the navigation on the car screen will render some surrounding objects, but it can only roughly understand that there may be an object nearby, and users cannot intuitively feel the specific location of the object.

[0073] Regarding whether it is necessary to predict the position of the vehicle in front in real time, since the underlying implementation logic of the two technologies is different, for example, assuming that in low visibility, the AR HUD is used to display the position element of the vehicle in front to help users identify the distance between the vehicle and the vehicle in front. How to ensure that this position element is consistent with the vehicle in front is very important for users. It is necessary to predict the position of the vehicle in front to be more consistent. For example, if the vehicle in front suddenly accelerates or decelerates, in order to make the animation rendering on the AR HUD keep up with the vehicle in front, it is necessary to predict the position of the vehicle in front. Specifically, it is necessary to first obtain multiple predicted positions of the vehicle in front (for example, some predicted positions of the vehicle in front are positions after acceleration, and some predicted positions of the vehicle in front are positions after deceleration). Then, the current frame position of the vehicle in front is windowed and averaged to obtain a predicted position. At this time, this predicted position can be displayed on the AR HUD. At this time, even if the vehicle in front suddenly accelerates, the predicted position is relatively consistent.

[0074] The navigation screen displayed on the car screen will have an animated rendering of the vehicle in front, but this animation rendering does not have such a strong requirement for being able to fit the vehicle in front. It does not matter even if there is a certain distance difference between the rendered position of the vehicle in front and the actual position. The car screen only needs to indicate an approximate relative position of the vehicle in front, and does not require a particularly accurate position of the vehicle in front. In other words, the navigation displayed on the car screen does not predict the speed of the vehicle in front, but instead uses perception data to detect and calculate in real time whether the vehicle will collide with the vehicle in front at the moment, without knowing whether the vehicle will collide with the vehicle in front in the future.

[0075] Regarding whether there is a problem of navigation light carpet exceeding the frame, since the display field of view of the AR HUD is only the content within a certain range in front of the vehicle, and the navigation light carpet must be aligned with the road conditions, for example, if you want to turn right or make a U-turn ahead, in this case, the navigation light carpet will exceed the frame. However, the picture displayed on the car screen includes the passable road ahead (and the navigation light carpet is not displayed). Even if you want to turn right or make a U-turn ahead, the passable road ahead will be displayed in the picture, so the car screen does not have a problem of navigation light carpet exceeding the frame.

[0076] The following is Figure 3a The lane changing scenario shown in FIG. Figure 3a As shown, the two vertical solid lines represent the road the vehicle is currently traveling on. The left vertical solid line and the middle vertical dotted line constitute the left lane, and the right vertical solid line and the middle vertical dotted line constitute the right lane. The rectangular box in the figure represents the augmented reality head-up display interface, point A represents the current position of the vehicle, point B represents the point in the middle of the lower edge of the augmented reality head-up display interface, point C represents the starting point of the lane change, point D represents the end point of the lane change, and E and F represent two control points. When the vehicle is driving in the left lane, the augmented reality head-up display device can display the following in the augmented reality head-up display interface. Figure 3a The first navigation light carpet shown by the dotted arrow in the figure, the vehicle can upload the real-time positioning information to the server. The server can determine whether the lane in front of the vehicle has changed based on the positioning information reported by the vehicle, and in the case of a lane change in front of the vehicle, that is, the vehicle changes lanes from the left lane to the right lane, send a lane change signal to the vehicle, which includes the lane change starting position (such as the lane change starting position). Figure 3a Point C shown in the figure) and the lane change end point position (as shown in the figure) Figure 3a Point C is usually located in the middle of the left lane, and point D is usually located in the middle of the right lane. After receiving the lane change signal, the augmented reality head-up display device can change the lane based on the perigee position of the augmented reality head-up display interface (such as Figure 3aThe lane change line segment between point C and point D in the first navigation light carpet is redrawn. Specifically, the position of the first control point (such as point B) can be determined based on the position of point B and point C. Figure 3a The position of point E shown in FIG3 is used as the reference point, and based on the position of point D, the position of the second control point is determined (as shown in FIG3 is used as the reference point). Figure 3a Then, based on the positions of points B, E, F, and D, a Bezier curve is drawn to obtain the following: Figure 3a The second navigation local light carpet is shown as the solid line with an arrow in the figure. The augmented reality head-up display device can display the second navigation local light carpet instead of the lane change line segment in the augmented reality head-up display interface. At this time, the augmented reality head-up display interface also displays the non-lane change line segment in the first navigation light carpet, that is, the line segment after point D. Figure 3a As shown, the starting point of the second navigation local light carpet is located at point B, and the ending point of the second navigation local light carpet is located in the middle of the right lane. The second navigation local light carpet consists of a straight segment extending forward (i.e., the starting segment) and two curved segments with opposite curvatures. The straight segment and the two curved segments are smoothly connected, and the last curved segment is smoothly connected to the non-lane change segment. Figure 3b As shown, when the vehicle is moving towards the right lane, the position of point B will be close to the middle lane line, and the vehicle will gradually approach the position of point D. Then, the redrawn second navigation local light carpet will gradually tend to be a straight line, but the starting end of the second navigation local light carpet will always be at point B, and the end end will always be in the middle of the right lane.

[0077] The following is Figure 4a The intersection misalignment scenario shown in the figure is used as an example to illustrate. Figure 4a As shown in the figure, the vertical solid line represents the lane, the blank area in the middle indicates the intersection, and the same lane is misaligned on both sides of the intersection. The rectangular box in the figure represents the augmented reality head-up display interface, point A represents the current position of the vehicle, point B represents the point in the middle of the lower edge of the augmented reality head-up display interface, point C represents the starting point of the lane change, point D represents the end point of the lane change, and E and F represent two control points. During the driving process of the vehicle, the augmented reality head-up display device can display the following in the augmented reality head-up display interface: Figure 4a The first navigation light carpet shown by the dotted arrow in the figure, the vehicle can upload the real-time positioning information to the server. The server can determine whether the lane in front of the vehicle has changed based on the positioning information reported by the vehicle, and in the case of a lane change in front of the vehicle, that is, the vehicle needs to enter the lane after the misalignment at the intersection, send a lane change signal to the vehicle, which includes the lane change starting position (such as the lane change starting position). Figure 4a Point C shown in the figure) and the lane change end point position (as shown in the figure) Figure 4aPoint C is usually located in the middle of the vehicle's current lane, and point D is usually located in the middle of the lane after the lane change. After receiving the lane change signal, the augmented reality head-up display device can adjust the lane based on the perigee position of the augmented reality head-up display interface (such as the perigee position of the augmented reality head-up display interface). Figure 4a The lane change line segment between point C and point D in the first navigation light carpet is redrawn. Specifically, the position of the first control point (such as point B) can be determined based on the position of point B and point C. Figure 4a The position of point E shown in FIG3 is used as the reference point, and based on the position of point D, the position of the second control point is determined (as shown in FIG3 is used as the reference point). Figure 4a Then, based on the positions of points B, E, F, and D, a Bezier curve is drawn to obtain the following: Figure 4a The second navigation local light carpet is shown as the solid line with an arrow in the figure. The augmented reality head-up display device can display the second navigation local light carpet instead of the lane change line segment in the augmented reality head-up display interface. At this time, the augmented reality head-up display interface also displays the non-lane change line segment in the first navigation light carpet, that is, the line segment after point D. Figure 4a As shown, the starting point of the second navigation local light carpet is located at point B, and the ending point of the second navigation local light carpet is located in the middle of the lane after the displacement. The second navigation local light carpet consists of a straight segment extending forward (i.e., the starting segment) and two curved segments with opposite bending directions. The straight segment and the two curved segments are smoothly connected, and the last curved segment is smoothly connected to the non-lane change segment. Figure 4b As shown, during the vehicle's driving at the intersection, point B will approach the dislocated lane, and the vehicle will gradually approach point D. The redrawn second navigation local light carpet will gradually tend to a straight line, but the starting end of the second navigation local light carpet will always be at point B, and the end end will always be in the middle of the right lane.

[0078] In an embodiment of the present invention, a first navigation light carpet is rendered and displayed on an augmented reality head-up display interface; in response to a lane change signal obtained according to the vehicle's path planning, the lane change line segments in the first navigation light carpet are redrawn to obtain a second navigation local light carpet; and the lane change line segments of the first navigation light carpet are switched to the second navigation local light carpet for rendering and display within the augmented reality head-up display interface, thereby showing the driver a lane change trend, such as a lane change trend or a lane misalignment trend at an intersection, by displaying the second navigation local light carpet. It is easy to notice that the second navigation local light carpet is obtained by redrawing the lane change line segment in the first navigation light carpet, so that the starting end of the second navigation local light carpet is located at the perigee position of the augmented reality head-up display interface, and the end end of the second navigation local light carpet is located in the middle of the target lane. Moreover, in the process of the vehicle driving to the target lane according to the second navigation local light carpet, the end end of the second navigation local light carpet is always located in the middle of the target lane, avoiding the back bow phenomenon of the second navigation local light carpet, and ensuring that the end end of the second navigation local light carpet fits the target lane, thereby improving the guidance and stability of the navigation light carpet in lane changing or lane misalignment at intersections, and solving the problem of poor guidance and stability of the navigation light carpet in lane changing or lane misalignment at intersections caused by the use of a compression scheme.

[0079] As an optional embodiment, the lane change line segments in the first navigation light carpet are redrawn to obtain a second navigation local light carpet, including: determining a first control point position based on a perigee position and a lane change starting position, wherein the first control point position is used to control the line segment boundary range and curvature of the first curved segment, and the coordinate values ​​of the first control point position and the perigee position in a preset direction in the vehicle's own vehicle coordinate system are consistent, and the preset direction is perpendicular to the longitudinal symmetry plane of the vehicle; determining a second control point position based on a lane change end position, wherein the second control point position is used to control the line segment boundary range and curvature of the second curved segment, and the coordinate values ​​of the second control point position and the lane change end position in the preset direction in the own vehicle coordinate system are consistent; generating a second navigation local light carpet based on the lane change starting position, the lane change end position, the first control point position and the second control point position.

[0080] The above-mentioned vehicle coordinate system can be a coordinate system built with the vehicle as the center, such as Figure 5 As shown, the X-axis of the coordinate system is parallel to the longitudinal symmetry plane of the vehicle and points to the current orientation of the vehicle's front. The Y-axis of the coordinate system is perpendicular to the longitudinal symmetry plane. The position of the ego vehicle coordinate system changes as the vehicle's current position changes.

[0081] It should be noted that if the lane change signal is sent from the server to the vehicle, the two locations contained in the lane change signal are in the map coordinate system. Therefore, coordinate transformation is required to convert these two locations to the vehicle coordinate system to obtain the lane change start and end locations. Since the vehicle's position changes in real time, the lane change start and end locations also change in real time.

[0082] In an optional solution of this embodiment, a third-order Bezier curve construction method is preferentially employed to generate the second local navigation light blanket. For this third-order Bezier curve, in addition to the lane change starting and ending positions, two control points must be determined: a first control point and a second control point. Considering the distance between the vehicle's current position and the perigee, as well as the distance between the vehicle's current position and the lane change starting position, to reflect this trend within the augmented reality head-up display, the first control point is determined based on the perigee and lane change starting positions, while the second control point is determined based on the lane change ending positions.

[0083] Optionally, the Y-axis coordinate value of the first control point position is the same as the Y-axis coordinate value of the perigee position, so that the proximal portion of the second navigation local light carpet is displayed as a straight line in the augmented reality head-up display interface. The X-axis coordinate value of the first control point position affects the curvature and line segment boundary range of the first curved segment in the second navigation local light carpet. For example, the larger the X-axis coordinate value of the first control point position, the greater the curvature of the first curved segment and the larger the line segment boundary range. Similarly, the Y-axis coordinate value of the second control point position is the same as the Y-axis coordinate value of the lane change endpoint position, so that the distal portion of the second navigation local light carpet is displayed as a straight line in the augmented reality head-up display interface. The X-axis coordinate value of the second control point position affects the curvature and line segment boundary range of the second curved segment in the second navigation local light carpet. For example, the smaller the X-axis coordinate value of the second control point position, the greater the curvature of the second curved segment and the larger the line segment boundary range. The relative position between the first control point and the second control point determines the smoothness of the transition between the first curved segment and the second curved segment, as well as the overall shape of the second navigation local light carpet. In this application, the positions of the two control points can be determined through experience or experiments.

[0084] Considering the high similarity between the shape of the third-order Bezier curve and the desired second navigation local light carpet, a third-order Bezier curve generation method can be used to generate the second navigation local light carpet. This not only simplifies the generation of the second navigation local light carpet but also ensures that the generated second navigation local light carpet meets the desired requirements. The lane change starting point is taken into account during the generation of the first control point to ensure that the second navigation local light carpet includes a nearly straight starting segment. This not only reflects the trend of lane changes or lane misalignment at intersections, but also informs the driver that there is still some distance between the vehicle's current position and the lane change starting point, and that an immediate lane change is not necessary.

[0085] Optionally, the position of the first control point is determined based on the perigee position and the lane change starting position, including: determining a first distance between the perigee position and the current position of the vehicle in the current orientation of the vehicle head, and a second distance between the lane change starting position and the current position of the vehicle in the current orientation of the vehicle head, wherein the current orientation of the vehicle head is parallel to the longitudinal symmetry plane of the vehicle; in response to the first distance being greater than the second distance, the position of the first control point is determined based on a first preset distance value between the perigee position and the current orientation of the vehicle head; in response to the first distance being less than or equal to the second distance, the position of the first control point is determined based on the perigee position, the distance between the perigee position and the lane change starting position in the current orientation of the vehicle head, and the first preset distance value.

[0086] The above-mentioned first preset distance can be a length predetermined through experience or experiments, which can ensure that the proximal part of the second navigation local light carpet appears as a straight line. For example, the first preset distance can be 15 meters, 20 meters, 25 meters, etc., but not limited to this. The above-mentioned current orientation of the vehicle head can refer to the orientation parallel to the longitudinal symmetry plane among the different orientations of the vehicle head, that is, the X-axis direction in the vehicle coordinate system. The above-mentioned first distance can be a distance calculated based on the perigee position and the current position of the vehicle. Since the augmented reality head-up display interface is projected at a fixed position on the vehicle windshield, the first distance can also be a predetermined fixed distance, for example, 20 meters, but not limited to this. The above-mentioned second distance can be a distance calculated based on the lane change starting position and the current position of the vehicle.

[0087] In an optional solution to this embodiment, since there is a certain distance between the perigee position and the vehicle's current position, and a certain distance between the vehicle's current position and the lane change starting point, in order to reflect this trend through the second navigation local light carpet, it is necessary to consider the influence of the lane change starting point when determining the position of the first control point. Considering that the vehicle's current position is the origin in the ego-vehicle coordinate system, the first distance can be a pre-stored fixed distance or a distance determined based on the X-axis coordinate value of the perigee position in the ego-vehicle coordinate system, and the second distance can be a distance determined based on the X-axis coordinate value of the lane change starting point in the ego-vehicle coordinate system. If the first distance is greater than the second distance, it can be determined that the lane change starting point position no longer needs to be displayed in the augmented reality head-up display interface, and the driver needs to change lanes as soon as possible. Therefore, at this time, the lane change starting point position can be ignored, and the first preset distance can be directly added to the X-axis coordinate value of the perigee position in the ego vehicle coordinate system as the X-axis coordinate value of the first control point position, and the Y-axis coordinate value of the first control point position is the same as the Y-axis coordinate value of the perigee position in the ego vehicle coordinate system. The Z-axis coordinate value of the first control point position can be a fixed value, or can be the same as the Z-axis coordinate value of the perigee position in the ego vehicle coordinate system. If the first distance is less than or equal to the second distance, it can be determined that the lane change starting point location still needs to be displayed in the augmented reality head-up display interface, and the driver can wait before changing lanes. Therefore, the lane change starting point location needs to be considered at this time. The first preset distance can be added to the X-axis coordinate value of the perigee location in the ego vehicle coordinate system, and the target distance is added as the X-axis coordinate value of the first control point location. The Y-axis coordinate value of the first control point location is the same as the Y-axis coordinate value of the perigee location in the ego vehicle coordinate system. The Z-axis coordinate value of the first control point location can be a fixed value or the same as the Z-axis coordinate value of the perigee location in the ego vehicle coordinate system. The target distance here refers to the distance on the X-axis between the perigee location and the lane change starting point location. That is, the target distance can be the difference between the X-axis coordinate value of the lane change starting point location in the ego vehicle coordinate system and the X-axis coordinate value of the perigee location in the ego vehicle coordinate system. Therefore, the X-axis coordinate value of the first control point location can be considered to be the X-axis coordinate value of the lane change starting point location plus the first preset distance.

[0088] For example, still Figure 5 Taking the lane change scenario shown as an example, the positions of points E and B are both on the same Y-axis coordinate value, and if the distance between AC is less than the distance between AB, the X-axis coordinate value of point E can be increased by 15 meters by default on the X-axis coordinate value of point B; if the distance between AC is greater than or equal to the distance between AB, the X-axis coordinate value of point E can be increased by the distance between BC and 15 meters by default on the X-axis coordinate value of point B.

[0089] By considering the influence of the lane change starting point in the process of determining the position of the first control point, the proximal part of the second navigation local light blanket can reflect the distance between the vehicle and the lane change starting point, so that the driver can accurately grasp the timing of lane change and avoid missing the opportunity to change lanes.

[0090] Optionally, determining the position of the second control point based on the lane change end position includes: determining the position of the second control point based on the lane change end position and a second preset distance value on the current direction of the vehicle head, wherein the current direction of the vehicle head is parallel to the longitudinal symmetry plane of the vehicle.

[0091] The second preset distance can be a length predetermined through experience or experiment, which can ensure that the distal portion of the second navigation local light blanket appears as a straight line. For example, the second preset distance can be 15 meters, 20 meters, 25 meters, etc., but is not limited thereto.

[0092] In an optional solution of this embodiment, to ensure a smooth connection between the second navigation local light carpet and the non-lane change line segments in the first navigation light carpet, the distal portion of the second navigation local light carpet needs to appear as a straight line. To reflect this trend through the second navigation local light carpet, the X-axis coordinate value of the lane change endpoint position in the ego vehicle coordinate system is directly subtracted from the first preset distance to form the X-axis coordinate value of the second control point position, while the Y-axis coordinate value of the second control point position is the same as the Y-axis coordinate value of the lane change endpoint position in the ego vehicle coordinate system. The Z-axis coordinate value of the second control point position can be a fixed value or can be the same as the Z-axis coordinate value of the lane change endpoint position in the ego vehicle coordinate system.

[0093] For example, still Figure 5 Taking the lane change scenario shown as an example, the positions of point F and point D are both at the same Y-axis coordinate value, and the X-axis coordinate value of point F can be reduced by 15 meters by default based on the X-axis coordinate value of point D.

[0094] The position of the second control point is determined by the end point of the lane change, ensuring that the far end of the second navigation local light carpet can reflect the vehicle's tendency to travel in a straight line. The driver can then accurately grasp the direction of the vehicle after the lane change, and achieve a smooth transition between the second navigation local light carpet and the non-lane change line segment in the first navigation light carpet.

[0095] As an optional implementation, after redrawing the lane change line segments in the first navigation light carpet to obtain the second navigation local light carpet, the above method also includes: splicing the second navigation local light carpet with the non-lane change line segments in the first navigation light carpet to obtain a spliced ​​navigation light carpet, wherein the non-lane change line segments are the line segments in the first navigation light carpet other than the lane change line segments.

[0096] In an optional solution of this embodiment, in order to ensure that the navigation light carpet seen by the driver is a complete light carpet, the second navigation local light carpet can be spliced ​​with the non-lane change line segment in the first navigation light carpet to obtain a complete spliced ​​navigation light carpet, and then the first navigation light carpet can be switched to the spliced ​​navigation light carpet in the augmented reality head-up display interface, that is, the spliced ​​navigation light carpet can be directly displayed in the augmented reality head-up display interface.

[0097] By splicing the second navigation local light carpet with the non-lane change line segments in the first navigation light carpet, the driver is ensured to see a complete light carpet in the augmented reality head-up display interface, improving the driver's experience.

[0098] As an optional implementation, the above method also includes: sending the vehicle's positioning information to a server; receiving first lane-level navigation data returned by the server, wherein the first lane-level navigation data at least includes: first light carpet control line data, the first lane-level navigation data is generated based on the positioning information, the vehicle's target address and lane-level map data; converting the first light carpet control line data from the map coordinate system corresponding to the first lane-level navigation data to the vehicle's own coordinate system to obtain first converted control line data; rendering the first converted control line data as a first navigation light carpet.

[0099] The above-mentioned positioning information can be used to reflect the current position of the vehicle, and can be obtained in real time to ensure the timeliness of the rendered first navigation light carpet. The above-mentioned target address can be selected or input by the user on the vehicle side, and then uploaded to the server by the vehicle side. It can be an address obtained before the vehicle is driven, or it can be an address obtained by the user modifying the pre-entered target address during the vehicle's driving. The above-mentioned server can refer to a server that can provide map data of the road the vehicle is currently traveling on, such as lane-level map data, and navigation data for assisting users in driving the vehicle. The above-mentioned first lane-level navigation data can refer to navigation data provided by the server for guiding the vehicle from the current position corresponding to the positioning information to the target address. It is lane-level data and can include but is not limited to: lane line data, first light carpet control line data, lane road type data, vehicle lane change data, etc.

[0100] In an optional solution of this embodiment, in order to accurately generate a guiding light carpet to assist the user in driving the vehicle, the vehicle can obtain real-time positioning information through a positioning system such as the GPS (Global Positioning System) and the Beidou Navigation System. Considering that the positioning system may be affected by signal interference, resulting in positioning errors, in order to ensure the accuracy of the determined positioning information, various sensors on the vehicle, such as cameras, radars, lidars, etc., can also be used to perceive the environmental information near the vehicle in real time, such as road features, lane lines, traffic signs, and the positions of surrounding vehicles and pedestrians. The perceived environmental information is then used to adjust the vehicle position located by the positioning system, thereby obtaining the accurate current position of the vehicle and constructing accurate positioning information.

[0101] The positioning information is then sent to the aforementioned server, allowing the server to determine the vehicle's current location and corresponding lane-level map data based on the received positioning information. The server then generates first lane-level navigation data for assisting vehicle driving based on the positioning information and lane-level map data, combined with the vehicle's target address. After generating the first navigation data, the server can then send the first lane-level navigation data to the vehicle. After receiving the first lane-level navigation data returned by the server, the augmented reality head-up display device can convert the first light carpet control line data from the map coordinate system to the vehicle coordinate system, as the first lane-level navigation data is data in the map coordinate system corresponding to the lane-level map data, while rendering the navigation light carpet requires data in the vehicle coordinate system. This converts the control line data to obtain the aforementioned first converted control line data, and can then render the first converted control line data as the first navigation light carpet.

[0102] By generating first lane-level navigation data through a server, waste of vehicle computing resources is avoided. Coordinate conversion is used to achieve seamless integration of lane-level map data and the vehicle's real-time environmental information, thereby improving the reliability of vehicle navigation. This allows the augmented reality head-up display device to construct an accurate first navigation light carpet based on the first light carpet control line data, thereby improving the user's experience of driving the vehicle based on the navigation light carpet.

[0103] Optionally, the first lane-level navigation data also includes: preset position information of the current lane, the current lane is the lane in which the vehicle is currently traveling, the current lane is determined based on the positioning information, and the first lane-level navigation data is generated based on the preset position information, the target address and the lane-level map data; rendering the first conversion control line data as a first navigation light carpet, including: in response to obtaining a lane change signal, based on the positioning information and the preset position information, correcting the first conversion control line data to obtain corrected control line data; rendering the corrected control line data as the first navigation light carpet.

[0104] It should be noted that the first navigation light carpet serves only as an auxiliary guide during driving and does not determine how the driver steers. This means that the vehicle's position may deviate while the driver is driving, meaning that the vehicle's real-time positioning information may deviate perpendicular to the longitudinal plane of symmetry. If the server generates the first lane-level navigation data directly based on the positioning information uploaded by the vehicle, this may cause the first navigation light carpet displayed in the augmented reality head-up display to vibrate.

[0105] In order to avoid the above situation, when the server generates the first lane-level navigation data, it will default the vehicle's position in the lane to the lane's preset position, for example, the middle position. The above preset position information may refer to the middle position information of the current lane, but is not limited to this. For example, Figure 6a As shown in the figure, the two vertical solid lines represent the road the vehicle is currently traveling on. The left vertical solid line and the middle vertical dotted line constitute the left lane, and the right vertical solid line and the middle vertical dotted line constitute the right lane. When the vehicle is traveling in the left lane, assuming that the vehicle travels from position M to position N, since the lane the vehicle is in does not change, the server still uses position M instead of position N when generating the first lane-level navigation data, resulting in the first navigation light carpet rendered in the map perspective not being as shown. Figure 6a The curve shown by the dashed arrow in Figure 6a However, in the vehicle perspective, if the augmented reality head-up display device renders the first navigation light carpet based on the first lane-level navigation data, the rendered first navigation light carpet will be as follows: Figure 6b The curve shown by the dashed arrow in the figure is not Figure 6b The curve indicated by the arrow in the figure does not end in the middle of the right lane, that is, the end of the first navigation light carpet does not fit in with the target lane. Figure 7 The swing effect shown in FIG. 1 is that before the vehicle moves to the right lane, the first navigation light carpet is a Figure 7 The first navigation light carpet is a curve shown by the dotted line with an arrow in the figure. The end point of the first navigation light carpet is close to the right lane line of the right lane. After the vehicle changes lanes, the first navigation light carpet suddenly changes into a line like Figure 7 The straight line shown by the solid arrow in .

[0106] In an optional solution of this embodiment, in order to ensure that the end point of the first navigation light carpet is aligned with the target lane line, a real-time deviation can be determined based on the positioning information and the preset position information, and then the first conversion control line data can be corrected based on the real-time deviation, such as Figure 6bAs shown, the first conversion control line data can be corrected based on the real-time deviation to obtain the corrected control line data, and then rendering is performed based on the corrected control line data to obtain the following: Figure 6b The first navigation light carpet is shown in the solid line with arrows.

[0107] By correcting the control line data in the first lane-level navigation data, it is possible to avoid errors between the positioning information and the preset position information, which may affect the rendering position of the first navigation light carpet, avoid the first navigation light carpet not fitting into the target lane and cause trouble to the driver, and avoid a decrease in the driving experience caused by swinging.

[0108] Optionally, based on the positioning information and the preset position information, the first conversion control line data is corrected to obtain corrected control line data, including: converting the positioning information from the geographic coordinate system to the vehicle coordinate system to obtain the first vehicle position, and converting the preset position information from the map coordinate system corresponding to the lane-level map data to the vehicle coordinate system to obtain the second vehicle position; based on the deviation between the first vehicle position and the second vehicle position, the first conversion control line data is corrected to obtain corrected control line data.

[0109] In an optional solution of this embodiment, since the positioning information is typically longitude and latitude information in a geographic coordinate system, while the preset position information is position information in a map coordinate system, the two are inherently incomparable. Therefore, both the positioning information and the preset position information can be converted to the vehicle coordinate system to obtain two vehicle positions. The deviation between the two vehicle positions can then be compared to determine the real-time deviation between the positioning information and the preset position information. Based on this real-time deviation, the first converted control line data can be corrected to obtain corrected control line data. The specific correction process is the same as that described above and will not be repeated here.

[0110] Through the coordinate system conversion, the first vehicle position and the second vehicle position are comparable, so that the real-time deviation between the positioning information and the preset position information can be accurately determined.

[0111] Optionally, the above method also includes: in response to detecting that the vehicle drives into the target lane, or obtaining a lane change end signal according to the vehicle's path planning, re-rendering the first conversion control line data into a first navigation light carpet, and displaying the first navigation light carpet in the augmented reality head-up display interface.

[0112] The above-mentioned lane change end signal can be a signal generated by the server based on the vehicle's real-time positioning information and path planning to determine that the vehicle is traveling into the target lane, or it can be a signal generated based on the vehicle's real-time positioning information and path planning to determine that the vehicle cannot perform the lane change operation.

[0113] In an optional solution of this embodiment, after the vehicle determines that it has traveled into the target lane through perception data, it can be determined that the second navigation local light carpet is no longer needed to be displayed in the augmented reality head-up display interface. Therefore, the first conversion control line data can be re-rendered as the first navigation light carpet, and the first navigation light carpet can be displayed in the augmented reality head-up display interface, such as Figure 3c or Figure 4c As shown, the first navigation light carpet may be a straight line.

[0114] In another optional scheme of this embodiment, after the vehicle receives the lane change end signal, it can be determined that the second navigation local light carpet is no longer needed to be displayed in the augmented reality head-up display interface. Therefore, the first conversion control line data can be re-rendered as the first navigation light carpet, and the first navigation light carpet can be displayed in the augmented reality head-up display interface.

[0115] It should be noted that since the rendering process of the first navigation light carpet ensures that the starting end of the first navigation light carpet starts from the current position of the vehicle, that is, even if there is a deviation in the first conversion control line data, a first navigation light carpet starting from the current position of the vehicle can be rendered. Therefore, in the process of re-rendering the first navigation light carpet, there is no need to correct the first conversion control line data.

[0116] By re-rendering and displaying the first navigation light carpet, the first navigation light carpet can be displayed normally after the driver does not need to perform a lane change operation, avoiding the display of the second navigation partial light carpet causing trouble to the driver and reducing driving safety.

[0117] As an optional implementation, the above method also includes: converting the near-point position from the vehicle coordinate system to the map coordinate system corresponding to the lane-level map data to obtain a converted position; sending the converted position to the server, and receiving second lane-level navigation data returned by the server, wherein the second lane-level navigation data at least includes: second light carpet control line data, wherein the second light carpet control line data is generated based on the converted position, the target address of the vehicle and the lane-level map data; converting the second light carpet control line data from the map coordinate system to the vehicle coordinate system to obtain second converted control line data; rendering the second converted control line data as a third navigation light carpet, and displaying the third navigation light carpet in the augmented reality head-up display interface.

[0118] In order to avoid the need for the augmented reality head-up display device to redraw the lane change line segments in the first navigation light carpet after obtaining the first control line data and rendering the first navigation light carpet, in an optional solution of this embodiment, the vehicle can directly send the perigee position to the server, so that the server can generate a lane-level navigation light carpet starting from the perigee position. However, since the perigee position is the position in the vehicle coordinate system, it is necessary to obtain the converted position in the map coordinate system through coordinate conversion. The vehicle can then send the converted position to the server, so that the server can generate the second lane-level navigation data based on the converted position. Since the generation process of the second lane-level navigation data is similar to the generation process of the first lane-level navigation data, the coordinate conversion process of the second light carpet control line data is similar to the coordinate conversion process of the first light carpet control line data, and the rendering process of the third navigation light carpet is similar to the rendering process of the first navigation light carpet, they will not be repeated here.

[0119] By directly providing the perigee position to the server, the server can directly generate second lane-level navigation data that meets the expected requirements, and then directly render a navigation light carpet starting from the perigee position, further simplifying vehicle operation.

[0120] It should be noted that due to the different configuration information of the augmented reality head-up display devices on different vehicles, adjusting the lane-level navigation data generation process in the server is difficult and will result in significant delays. Therefore, in this embodiment of the application, the second navigation local light carpet is preferably generated by redrawing.

[0121] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0122] According to an embodiment of the present invention, an augmented reality head-up display device is provided, comprising: a memory storing an executable program; and a processor for running the program, wherein the program executes the methods of various embodiments of the present invention when running.

[0123] An embodiment of the present application further provides a vehicle, comprising: the augmented reality head-up display device in the above embodiment of the present invention.

[0124] An embodiment of the present application further provides a computer-readable storage medium, which includes a stored executable program, wherein when the executable program is running, the device where the computer-readable storage medium is located is controlled to execute the methods in various embodiments of the present invention.

[0125] An embodiment of the present application further provides a computer program product, including a computer program, which implements the methods in various embodiments of the present invention when executed by a processor.

[0126] An embodiment of the present application further provides a computer program product, including a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium is used to store a computer program, and when the computer program is executed by a processor, the method in each embodiment of the present invention is implemented.

[0127] The embodiments of the present application further provide a computer program, which implements the methods in the above-mentioned embodiments of the present invention when executed by a processor.

[0128] In the above embodiments of the present invention, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0129] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only exemplary. For example, the division of the units can be a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0130] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0131] In addition, the functional units in the various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0132] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk, etc. Various media that can store program codes.

[0133] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as within the scope of protection of the present invention.

Claims

1. A vehicle augmented reality navigation method, characterized in that: include: Rendering of the first navigation light carpet in the augmented reality head-up display interface; In response to a lane change signal acquired according to the vehicle's path planning, the lane change line segments in the first navigation light carpet are redrawn to obtain a second navigation partial light carpet, wherein the lane change signal is used to indicate a lane change ahead of the vehicle, the end point of the second navigation partial light carpet is located in the middle of a target lane, which is the lane ahead of the vehicle after the lane has changed according to the path planning, and the start point of the second navigation partial light carpet is located at the perigee position of the augmented reality head-up display interface; Switching the lane change line segment of the first navigation light carpet to the second navigation partial light carpet for rendering and display in the augmented reality head-up display interface; In which, in the process of the vehicle driving to the target lane according to the second navigation local light carpet, the end point of the second navigation local light carpet is always located in the middle of the target lane, and the second navigation local light carpet includes a starting section, a first curved section and a second curved section. One end of the starting section is the starting end of the second navigation local light carpet, and the other end of the starting section is connected to one end of the first curved section, and the other end of the first curved section is connected to one end of the second curved section, and the other end of the second curved section is the end point of the second navigation local light carpet.

2. The method according to claim 1, characterized in that The lane change signal includes: a lane change starting position and a lane change end position; and redrawing the lane change line segment in the first navigation light carpet to obtain a second navigation partial light carpet includes: Determining a first control point position based on the perigee position and the lane change starting position, wherein the first control point position is used to control a line segment boundary range and curvature of the first curved segment, the first control point position and the perigee position have consistent coordinate values ​​in a preset direction in the vehicle coordinate system, the preset direction being perpendicular to a longitudinal symmetry plane of the vehicle, and the lane change line segment being a line segment between the lane change starting position and the lane change ending position; Determining a second control point position based on the lane change endpoint position, wherein the second control point position is used to control a line segment boundary range and a curvature of the second curved segment, and the coordinate values ​​of the second control point position and the lane change endpoint position in the preset direction in the vehicle coordinate system are consistent; The second navigation local light carpet is generated based on the perigee position, the lane change end position, the first control point position, and the second control point position.

3. The method according to claim 2, characterized in that The determining the position of the first control point based on the perigee position and the lane change starting position includes: Determining a first distance between the perigee position and the current position of the vehicle in the current heading of the vehicle head, and a second distance between the lane change starting position and the current position of the vehicle in the current heading of the vehicle head; In response to the first distance being greater than the second distance, determining the first control point position based on the perigee position and a first preset distance value on the current orientation of the vehicle head, wherein the current orientation of the vehicle head is parallel to a longitudinal symmetry plane of the vehicle; In response to the first distance being less than or equal to the second distance, the first control point position is determined based on the perigee position, the distance between the perigee position and the lane change starting point position in the current heading of the vehicle head, and the first preset distance value.

4. The method according to claim 2, characterized in that The determining the position of the second control point based on the lane change end position includes: The second control point position is determined based on the lane change end position and a second preset distance value on the current orientation of the vehicle head, wherein the current orientation of the vehicle head is parallel to the longitudinal symmetry plane of the vehicle.

5. The method according to any one of claims 1 to 4, characterized in that After redrawing the lane change line segments in the first navigation light carpet to obtain a second navigation partial light carpet, the method further includes: The second navigation local light carpet is spliced ​​with the non-lane change line segments in the first navigation light carpet to obtain a spliced ​​navigation light carpet, wherein the non-lane change line segments are line segments in the first navigation light carpet other than the lane change line segments.

6. The method according to any one of claims 1 to 4, characterized in that The method further comprises: Send the vehicle's positioning information to the server; receiving first lane-level navigation data returned by the server, wherein the first lane-level navigation data includes at least first light carpet control line data, and the first lane-level navigation data is generated based on the positioning information, the target address of the vehicle, and lane-level map data; Converting the first light carpet control line data from a map coordinate system corresponding to the first lane-level navigation data to a vehicle coordinate system of the vehicle to obtain first converted control line data; The first converted control line data is rendered as the first navigation light carpet.

7. The method according to claim 6, characterized in that The first lane-level navigation data further includes: preset position information of a current lane, the current lane being the lane in which the vehicle is currently traveling, the current lane being determined based on the positioning information, and the first lane-level navigation data being generated based on the preset position information, the target address, and the lane-level map data; and rendering the first conversion control line data into a first navigation light carpet includes: In response to acquiring the lane change signal, based on the positioning information and the preset position information, correcting the first conversion control line data to obtain corrected control line data; The modified control line data is rendered as the first navigation light carpet.

8. The method according to claim 7, characterized in that The step of correcting the first conversion control line data based on the positioning information and the preset position information to obtain corrected control line data includes: Converting the positioning information from a geographic coordinate system to a vehicle coordinate system to obtain a first vehicle position, and converting the preset position information from a map coordinate system corresponding to the lane-level map data to a vehicle coordinate system to obtain a second vehicle position; Based on the deviation between the first vehicle position and the second vehicle position, the first conversion control line data is corrected to obtain the corrected control line data.

9. The method according to claim 7 or 8, characterized in that The method further comprises: In response to detecting that the vehicle has entered the target lane, or obtaining a lane change end signal according to the vehicle's path planning, the first conversion control line data is re-rendered as the first navigation light carpet, and the first navigation light carpet is displayed in the augmented reality head-up display interface.

10. The method according to claim 1, characterized in that The method further comprises: Converting the perigee position from the vehicle coordinate system to a map coordinate system corresponding to the lane-level map data to obtain a converted position; sending the conversion position to a server, and receiving second lane-level navigation data returned by the server, wherein the second lane-level navigation data at least includes: second light carpet control line data, wherein the second light carpet control line data is generated based on the conversion position, the target address of the vehicle, and the lane-level map data; converting the second light carpet control line data from the map coordinate system to the vehicle coordinate system to obtain second converted control line data; The second converted control line data is rendered into a third navigation light carpet, and the third navigation light carpet is displayed in the augmented reality head-up display interface.

11. An augmented reality head-up display device, characterized in that: include: a memory storing an executable program; A processor, configured to run the program, wherein the program executes the method according to any one of claims 1 to 10 when running.

12. A vehicle, characterized in that: include: The augmented reality head-up display device according to claim 11.

Images