Navigation arrow display method, vehicle-mounted device, readable storage medium and chip

Abstract

The present application relates to the technical field of vehicle navigation. Provided are a navigation arrow display method, a vehicle-mounted device, a readable storage medium and a chip. The method is applied to a vehicle-mounted device in a vehicle, which is provided with a head-up display (HUD). The method comprises: detecting that there is an obstacle on a first lane where a vehicle is currently located; determining the position of the obstacle in a field-of-view image; determining the display position of a navigation arrow on the basis of the position of the obstacle in the field-of-view image; and on the basis of the display position of the navigation arrow, using an HUD to display the navigation arrow. In the technical solution provided by the present application, a navigation arrow has a higher degree of integration with actual road conditions, thereby providing a better user experience.

Description

Navigation arrow display method, in-vehicle equipment, readable storage medium and chip

[0001] This application claims priority to Chinese patent application filed on November 22, 2023, with application number 202311573540.4 and entitled "Navigation Arrow Display Method, Vehicle-Mounted Device, Readable Storage Medium and Chip", the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of vehicle navigation technology, and in particular to a navigation arrow display method, in-vehicle device, readable storage medium, and chip. Background Technology

[0003] With the development of computer technology, people have gradually combined augmented reality (AR) technology with head-up displays (HUDs) in vehicles, forming a more powerful AR-HUD. Vehicles can display AR navigation arrows via the AR-HUD while driving. Currently, vehicles typically use navigation data and lane-related information to determine the position of the AR navigation arrows, and then display them at that location using the AR-HUD. However, this display method suffers from weak robustness in the AR navigation arrow display, meaning the integration of the AR navigation arrows with the actual road conditions is not high, resulting in a poor display effect. Summary of the Invention

[0004] This application provides a navigation arrow display method, an in-vehicle device, a readable storage medium, and a chip to solve the problem of poor integration between AR navigation arrows and actual road conditions and poor display effects in the prior art.

[0005] To achieve the above objectives, this application adopts the following technical solution:

[0006] In a first aspect, embodiments of this application provide a navigation arrow display method, which is applied to an in-vehicle device in a vehicle equipped with a head-up display (HUD). The method includes: detecting an obstacle in a first lane where the vehicle is currently located; determining the position of the obstacle in a field of view image; and displaying a navigation arrow using the HUD based on the position of the obstacle in the field of view image.

[0007] The head-up display (HUD) can be a traditional head-up display or a virtual reality head-up display (AR-HUD), and this embodiment does not impose any restrictions on it.

[0008] The method provided in this embodiment enables the vehicle-mounted device to determine a suitable display position for the navigation arrow based on the position of obstacles such as fences and water barriers in the actual road conditions, and to display the navigation arrow at that position using a HUD, so that the navigation arrow has a high degree of integration with the actual road conditions, thereby improving the user's viewing experience.

[0009] In some embodiments, detecting an obstacle in the first lane where the vehicle is currently located includes: acquiring a view image of the vehicle; and detecting the obstacle in the first lane based on the view image. In other words, in this embodiment, the on-board device determines whether an obstacle exists in the first lane through image recognition.

[0010] Of course, in some other embodiments, the on-board equipment can also use radar and field-of-view images to determine whether there are obstacles in the first lane. For example, radar can be used to detect whether there are obstacles near the vehicle. If there are obstacles near the vehicle, a field-of-view image can be acquired and identified to determine whether there are obstacles in the first lane. The radar can be lidar, millimeter-wave radar, etc., and this embodiment does not limit this.

[0011] In some embodiments, displaying navigation arrows using a HUD based on the location of the obstacle in the field of view image includes:

[0012] When the obstacle in the first lane does not impede passage, the HUD displays a navigation arrow indicating passage through the first lane, based on the obstacle's position in the field of view.

[0013] When an obstacle in the first lane affects traffic, but the second lane in the same direction as the first lane is passable, the HUD displays a navigation arrow indicating whether to enter the second lane from the first lane, based on the obstacle's position in the field of view.

[0014] Using the method provided in this embodiment, the vehicle-mounted device, after identifying the presence of an obstacle in the first lane, can further determine the impact of the obstacle on the traffic conditions of the first lane, and display different navigation arrows according to different impact results, making the display process quite intelligent.

[0015] In addition, when there are obstacles preventing passage in the first lane and other lanes in the same direction as the first lane, the on-board equipment can also display a warning message indicating that the road is impassable to alert the user.

[0016] In some embodiments, displaying a navigation arrow using a HUD based on the position of the obstacle in the field of view image includes: determining the display position of the navigation arrow based on the position of the obstacle in the field of view image; and displaying the navigation arrow using a HUD based on the display position of the navigation arrow.

[0017] In some embodiments, when the navigation arrow is a forward arrow, determining the display position of the navigation arrow based on the position of the obstacle in the field of view image, and displaying the navigation arrow using a HUD based on the display position of the navigation arrow, includes: determining a traffic guide line for the navigation arrow based on the position of the obstacle in the field of view image; determining the position on the traffic guide line at a preset distance from the vehicle as the display position of the navigation arrow; and using a HUD to display the navigation arrow along the traffic guide line based on the display position of the navigation arrow.

[0018] In this embodiment, the vehicle can determine the appropriate display position and traffic guidance line for the navigation arrow based on the position of the obstacle, and guide the vehicle to pass safely along the traffic guidance line. This method helps the vehicle drive safely.

[0019] In some embodiments, determining the navigation arrow's guide line based on the location of the obstacle in the field of view image includes: when the first lane in the field of view image is a straight lane, determining a passable area of ​​the first lane based on the location of the obstacle, wherein the passable area does not include the obstacle; and determining the center line of the passable area as the guide line of the first lane.

[0020] Using the method provided in this embodiment, when the first lane is a straight lane, the on-board equipment determines the center line of the passable area of ​​the first lane as the traffic guide line, so that the navigation arrow can be displayed in the center of the passable area to guide the vehicle safely through the obstacle and avoid hitting the obstacle.

[0021] In some embodiments, determining the traffic guidance line of the navigation arrow based on the position of the obstacle in the field of view image includes: when the first lane in the field of view image is a curved lane, determining the center line of the boundary lines on both sides of the passable area of ​​the first lane based on the position of the obstacle in the field of view image, wherein each point on the center line is equidistant from the boundary lines on both sides of the passable area; and determining the center line as the traffic guidance line of the navigation arrow.

[0022] The method provided in this embodiment enables vehicles to avoid obstacles when the first lane is a curved lane. The traffic guidance line determined by the on-board equipment can not only guide vehicles to avoid obstacles, but also closely match the curvature of the curved lane. This not only helps to improve driving safety, but also helps to improve the display effect of the navigation arrow.

[0023] In some embodiments, when the navigation arrow is a left-turn arrow or a right-turn arrow, the display position of the navigation arrow is determined according to the position of the obstacle in the field of view image, and the navigation arrow is displayed using a HUD according to the display position of the navigation arrow, including: determining the traffic guide line of the navigation arrow according to the position of the obstacle; determining the trajectory tangent line of the vehicle when it is driving stably after making a left or right turn according to navigation information and vehicle driving information; determining the intersection of the traffic guide line and the trajectory tangent line as the display position of the navigation arrow; and using a HUD, displaying the navigation arrow along the direction of the vehicle's trajectory tangent line according to the display position of the navigation arrow.

[0024] Using the method provided in this embodiment, under the influence of traffic guidance lines, the vehicle-mounted equipment can display left-turn and right-turn arrows in front of the vehicle's actual traffic position, providing a better visual experience.

[0025] In some embodiments, when the obstacle is a fork in the road, the display position of the navigation arrow is determined based on the position of the obstacle in the field of view image, and the navigation arrow is displayed using a HUD based on the display position of the navigation arrow, including: determining the location of the fork as the display position of the navigation arrow; and using the HUD to display the navigation arrow along the direction of the fork at the location of the fork.

[0026] Compared to displaying navigation arrows in the middle of a fork in the road, displaying navigation arrows starting from the fork point and following the direction of the fork is easier for drivers to see and provides better guidance.

[0027] In some embodiments, determining the display position of the navigation arrow based on the position of the obstacle in the field of view image includes: determining the initial display position of the navigation arrow based on the vehicle's navigation information and location information; correcting the initial display position based on the position of the obstacle in the field of view image to obtain the corrected display position; wherein the corrected display position is the final display position of the navigation arrow.

[0028] In some embodiments, the obstacle includes at least one of the following: fence, water-filled barriers, road signs, forks in a road, road construction barriers, pedestrians, and other vehicles.

[0029] In some embodiments, the method further includes: identifying whether the vehicle is driving in the wrong direction based on the field-of-view image within a preset time period; if the vehicle is driving in the wrong direction, displaying a prompt message indicating that the vehicle is driving in the wrong direction to ensure driving safety.

[0030] Secondly, embodiments of this application provide an in-vehicle device configured to perform the method shown in the first aspect above.

[0031] Thirdly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method shown in the first aspect above.

[0032] Fourthly, embodiments of this application provide a chip including a processor and a memory, the memory storing a computer program that, when executed by the processor, implements the method shown in the first aspect above.

[0033] Fifthly, embodiments of this application provide a computer program product storing a computer program that, when executed by a processor, can implement the method shown in the first aspect above.

[0034] Sixthly, embodiments of this application also provide a vehicle that includes in-vehicle equipment as shown in the second aspect above, and a head-up display (HUD) that displays navigation arrows.

[0035] It is understood that the beneficial effects of the second to sixth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here. Attached Figure Description

[0036] Figure 1 is a schematic diagram of the working principle of the HUD provided in the embodiment of this application;

[0037] Figure 2 is a schematic diagram of an AR navigation arrow provided in an embodiment of this application;

[0038] Figure 3 is a schematic diagram of a hazard warning message provided in an embodiment of this application;

[0039] Figure 4 is a schematic diagram of a lane and lane lines provided in an embodiment of this application;

[0040] Figure 5 is a schematic diagram of a vehicle coordinate system provided in an embodiment of this application;

[0041] Figure 6 is a schematic diagram of a pixel coordinate system provided in an embodiment of this application;

[0042] Figure 7A is a partial structural schematic diagram of the vehicle to which the navigation arrow display method provided in the embodiments of this application is applicable;

[0043] Figure 7B is a schematic diagram of the arrow determination module provided in an embodiment of this application;

[0044] Figure 8 is a schematic diagram of the steering angle provided in an embodiment of this application;

[0045] Figure 9 is a schematic flowchart of the navigation arrow display method provided in an embodiment of this application;

[0046] Figure 10 is a schematic diagram of the display style of the AR navigation arrow provided in the embodiment of this application;

[0047] Figures 11A to 11C are schematic diagrams of different road traffic conditions provided in the embodiments of this application;

[0048] Figure 12 is a schematic flowchart of the AR navigation arrow display method in the forward-moving scenario provided in the embodiments of this application;

[0049] Figure 13 is a schematic diagram of an embodiment of this application providing a method for determining the display position of the AR forward arrow;

[0050] Figure 14 is a schematic diagram of another embodiment of this application providing a method for determining the display position of the AR forward arrow;

[0051] Figure 15 is a schematic flowchart of the AR navigation arrow display method in a turning scenario provided in an embodiment of this application;

[0052] Figure 16 is a schematic diagram of the display effect of the AR navigation arrow in a left-turn scenario provided in an embodiment of this application;

[0053] Figure 17 is a schematic flowchart of the AR navigation arrow display method in the road crossing scenario provided in the embodiments of this application;

[0054] Figure 18 is a schematic diagram of the display effect of AR navigation arrows in a road crossing scenario provided in an embodiment of this application;

[0055] Figures 19A and 19B are schematic diagrams showing the prompt information provided in an embodiment of this application;

[0056] Figure 20 is a schematic diagram of the display of prompt information provided in another embodiment of this application;

[0057] Figure 21 is a flowchart of an AR navigation arrow display method provided in another embodiment of this application;

[0058] Figure 22 is a schematic diagram of the correction of the display position of the AR navigation arrow provided in the embodiment of this application;

[0059] Figure 23A is a schematic flowchart of the lane recognition method provided in an embodiment of this application;

[0060] Figure 23B is a flowchart of a method for determining vehicle reversing according to an embodiment of this application;

[0061] Figure 24 is a schematic diagram of the longitudinal distance between the target vehicle and the self-vehicle provided in an embodiment of this application;

[0062] Figure 25 is a schematic diagram of the chip structure provided in an embodiment of this application. Detailed Implementation

[0063] The technical solutions provided in the embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0064] It should be understood that in the description of the embodiments of this application, unless otherwise stated, " / " means "or". For example, A / B can mean A or B. The "and / or" in this document is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist simultaneously, and B exists alone.

[0065] In this embodiment, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this embodiment, unless otherwise stated, "a plurality of" means two or more.

[0066] With the development of computer technology, people have gradually combined augmented reality (AR) technology with head-up displays (HUDs) in vehicles, forming a more powerful AR-HUD. AR-HUDs can improve the user's driving experience and have a wide range of applications in vehicles.

[0067] The display principle of HUD will be explained by example below.

[0068] Figure 1 is a schematic diagram illustrating the working principle of a HUD provided in an embodiment of this application. The HUD primarily operates based on the principle of optical reflection. For example, referring to Figure 1, the HUD is positioned below the windshield in the vehicle's cockpit and can project light onto the windshield using a light source. This projected light carries relevant information to be displayed. After being reflected by the windshield, the reflected light enters the driver's eyes along the direction from the forward field of vision. Since the human eye perceives light as traveling in a straight line, the driver perceives the reflected light as originating from the forward field of vision, thus seeing the relevant information projected by the HUD in their forward field of vision.

[0069] Typically, HUDs project information including vehicle instrument panel data (such as speed, fuel level, and gear position), driving alerts (such as collision warnings and lane departure warnings), and navigation information (navigation arrows, route congestion information, remaining driving time, and arrival time). As opposed to traditional technologies that display information on a car screen, HUDs directly display information in the driver's field of vision. This allows drivers to quickly access this information by simply looking ahead, avoiding the need to frequently look down at the car screen and thus improving driving safety.

[0070] AR-HUD adds AR display functionality to the basic HUD. Specifically, AR-HUD can display AR images that are integrated with actual road conditions in the driver's field of vision, enhancing the driver's access to visual information and providing a better user experience. Therefore, more and more vehicles are starting to use AR-HUD functionality.

[0071] In some implementations, vehicles can combine navigation information, advanced driver assistance system (ADAS) information, and driver visibility information (such as field-of-view images) to use AR-HUD to achieve functions such as AR navigation and AR warnings. For example, as shown in Figure 2, the vehicle can use AR-HUD to display AR navigation arrows that are fused with the actual road conditions in the forward field of view based on navigation information. Or, as shown in Figure 3, the vehicle can use AR-HUD to display hazard warning information at the location of a pedestrian after detecting a pedestrian near the lane.

[0072] Currently, vehicles equipped with AR-HUDs typically determine the type and location of AR navigation arrows using navigation data, lane recognition information, and lane line detection information, and then display the AR navigation arrows at those locations. Ideally, the AR navigation arrows should appear at appropriate times, visually blend seamlessly with the actual road conditions, and be compatible with various complex real-world road situations. However, the current display method does not consider obstacles in the driver's field of vision, such as fences, water barriers, and forks in intersections. Therefore, the robustness of the AR navigation arrow display is weak, the integration of the display with actual road conditions is low, and the display effect is unsatisfactory.

[0073] Therefore, this application provides a navigation arrow display method. Through this method, vehicles equipped with AR-HUD can display AR navigation arrows based on the positions of obstacles in the actual road conditions, so that the AR navigation arrows have a high degree of integration with the actual road conditions, thereby improving the user experience.

[0074] The following section will first explain some of the terms used in the embodiments of this application.

[0075] (1) Lanes and lane lines

[0076] A lane is the area where vehicles travel, typically located between two lane lines. For ease of description, referring to Figure 4, this embodiment refers to the lane the vehicle is currently in as the first lane, other lanes traveling in the same direction as the first lane as the second lane, third lane, etc., and lanes traveling in the opposite direction to the first lane as oncoming lanes. It should be noted that in this embodiment, the terms first, second, and third are used to distinguish different lanes; this embodiment does not restrict the relative positions of the first, second, and third lanes.

[0077] In this embodiment, lane line attributes typically include line type and color. Lane line types typically include dashed and solid lines, and lane line colors typically include yellow and white. Each lane line typically contains one or two lines. For example, as shown in Figure 4, the two yellow lines in the middle of the road are central double yellow lines, used to separate lanes traveling in different directions. The white dashed lines in the road are used to separate different lanes traveling in the same direction.

[0078] (2) Vehicle coordinate system

[0079] The vehicle coordinate system is a special coordinate system used to describe the motion of a vehicle. For example, as shown in Figure 5, the origin of the vehicle coordinate system is usually the vehicle's center of mass, the vehicle's direction of travel is the Z-axis, the vertical direction is the Y-axis, and the direction from the left to the right of the vehicle is the X-axis. During vehicle movement, we are more concerned with the relative positions of objects around the vehicle than their absolute positions. Therefore, the vehicle coordinate system is typically used to determine the positional information of various objects (such as lanes, lane lines, obstacles, etc.) in the field of view image.

[0080] (3) World Coordinate System

[0081] The world coordinate system (WCS), also known as the measurement coordinate system, is a three-dimensional Cartesian coordinate system used as a reference to describe the spatial position of the camera and the object being measured. The position of the world coordinate system can be freely determined according to the actual situation.

[0082] (4) Pixel coordinate system

[0083] The pixel coordinate system is an image coordinate system using pixels as the unit, used to describe the position of each pixel in an image. For example, as shown in Figure 6, the origin of the pixel coordinate system is usually the top-left corner of the image plane, and the X-axis and Y-axis are parallel to the horizontal and vertical edges of the image, respectively. The position of a pixel in the image pixel coordinate system can be represented by (u, v). It should be noted that the vehicle coordinate system, world coordinate system, and pixel coordinate system can be converted to each other.

[0084] The navigation arrow display method provided in the embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0085] Figure 7A is a partial structural schematic diagram of a vehicle to which the navigation arrow display method provided in this application embodiment is applicable. As shown in Figure 7A, the vehicle includes a field of view image acquisition module 701, a field of view image element recognition module 702, an information inference module 703, a navigation module 704, an in-vehicle monitoring module 705, an arrow determination module 706, an alarm prompting module 707, and an AR-HUD 708.

[0086] The field-of-view image acquisition module 701 can be a camera used to acquire field-of-view images. For example, the field-of-view image acquisition module 701 includes at least a camera disposed in front of the vehicle, and the field-of-view image includes at least the field-of-view image in front of the vehicle.

[0087] It should be noted that, in order to acquire images of the vehicle's exterior, the camera can be located at an appropriate position outside the vehicle. For example, in this embodiment, the camera can be located near the windshield inside the vehicle, or positioned around the front bumper or radiator grille to capture images of the area in front of the vehicle. Alternatively, to acquire images of the vehicle's sides, the camera can be positioned near the interior side windows. This embodiment does not limit the specific location of the camera.

[0088] The field-of-view image element recognition module 702 is used to identify key information in the acquired field-of-view image using image recognition algorithms. For example, it can segment the field-of-view image using algorithms such as the bilateral segmentation network (BiSeNet) to obtain information about obstacles such as vehicles, pedestrians, sidewalks, fences, water-filled barriers, and road forks. Alternatively, it can identify the location and extent of traffic lights and road signs in the field-of-view image using the "You Only Look Once" (YOLO) algorithm. Or, it can further identify text on road signs using optical character recognition (OCR) algorithms. This embodiment does not limit the image recognition algorithms used by the field-of-view image element recognition module 702.

[0089] The information inference module 703 is used to further determine actual vehicle driving information, road condition information, etc., based on the key information determined by the visual field image element recognition module 702. For example, it infers whether the vehicle is driving in the wrong direction based on the position of vehicles in each lane at different times. Or, it determines whether there are obstacles in each lane based on the visual field image, and if there is an obstacle in the first lane, it determines whether the obstacle affects vehicle passage. Or, it infers whether there is an intersection in the current field of vision based on information such as the text on the road sign, traffic lights, zebra crossings, and the direction of travel of vehicles ahead. Or, it infers whether there is construction ahead based on the text on the road sign.

[0090] The navigation module 704 provides navigation information for the vehicle, including but not limited to navigation points, intersection nodes, and intersection types. Navigation points refer to the future driving trajectory planned by the navigation application based on the vehicle's location and destination information. Intersection nodes are used to separate navigation points, indicating and distinguishing different intersections. Intersection types include, but are not limited to, intersections leading to the road, left-turn intersections, right-turn intersections, U-turn intersections, and forks in the road.

[0091] The vehicle monitoring module 705 is used to determine lane line information and monitor the vehicle's pose information. In this embodiment, lane line information includes lane line attributes and the three-dimensional coordinates of the lane line in the vehicle coordinate system. Lane line attributes can include color, line type, etc. The vehicle's pose information includes position information and steering angle. Position information can be latitude and longitude information, and the steering angle refers to the angle formed by the direction the vehicle's tires turn to the left or right and the direction when they do not turn, as shown in Figure 8.

[0092] The arrow determination module 706 is used to determine the type and position of the AR navigation arrow based on the information output by the above modules, so that the AR-HUD can display the AR navigation arrow of that type at that position.

[0093] The type of arrow can be determined based on the type of intersection, with different types of arrows having different indicative functions. For example, the arrow type for a left-turn intersection can be determined as a left-turn arrow, indicating that vehicles should turn left at the intersection ahead. Alternatively, the arrow type for a right-turn intersection can be determined as a right-turn arrow, indicating that vehicles should turn right at the intersection ahead. Or, the arrow type for a U-turn intersection can be determined as a U-turn arrow, indicating that vehicles should make a U-turn at the intersection ahead. Or, the arrow type for a fork in the road can be determined as a fork-road driving arrow, indicating that vehicles should travel towards the fork in the road. Furthermore, the arrow determination module 706 can also determine lane-change arrows to indicate lane-changing actions based on actual road conditions; this embodiment does not impose specific restrictions on the type of arrow.

[0094] In addition, the position of the arrow can be determined based on the vehicle's position, the vehicle's current position in the first lane, and the position of obstacles in the first lane, such as water-filled barriers, fences, and forks in the road.

[0095] In some implementations, the arrow determination module 706 comprehensively analyzes the information output by modules such as the field-of-view image element recognition module 702, information inference module 703, navigation module 704, and vehicle monitoring module 705 to determine the type and position of the AR navigation arrow, so that the AR-HUD can display the AR navigation arrow of that type at that position.

[0096] In some implementations, as shown in Figure 7B, the arrow determination module 706 includes an arrow determination unit 706A and an optimization correction unit 706B. The arrow determination unit 706A determines the type and initial display position of the AR navigation arrow based on information output from the navigation module 704 and the vehicle monitoring module 705. The optimization correction unit 706B first corrects the initial display position of the navigation arrow based on information output from the field-of-view image element recognition module 702 and the information inference module 703, determining the corrected display position so that the AR-HUD displays the AR navigation arrow of that type at the corrected position.

[0097] The alarm notification module 707 is used to provide alarm notifications for scenarios with potential risks or suspected errors based on information output by the field-of-view image element recognition module 702 or the information inference module 703. For example, scenarios with potential risks include road construction ahead, obstacles ahead, or vehicles driving in the wrong direction. Scenarios suspected of being incorrect include a mismatch between the detected road and the road suggested by the navigation system, such as not detecting the intersection suggested by the navigation system.

[0098] The AR-HUD 708 is used to present vehicle instrument information, driving alerts, navigation information, and AR visuals integrated with real-time road conditions, such as AR navigation arrows, in the driver's forward field of vision, enabling convenient access to information. See above for details.

[0099] It should be noted that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the vehicle. In other embodiments, the vehicle may include more components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0100] In some embodiments, the vehicle may further include radar, such as lidar, millimeter-wave radar, etc. For example, the radar can work in conjunction with the aforementioned field-of-view image acquisition module 701 and field-of-view image element recognition module 702. For instance, the radar can first detect whether there are obstacles near the vehicle. If obstacles are present, the field-of-view image acquisition module 701 acquires a field-of-view image, and the field-of-view image element recognition module 702 identifies the field-of-view image, thereby reducing the power consumption of the image acquisition and recognition process.

[0101] The following is a detailed description of the navigation arrow display method for vehicles provided in this embodiment.

[0102] Figure 9 is a schematic flowchart of the navigation arrow display method provided in an embodiment of this application. Referring to Figure 9, the method is executed by the corresponding module in the vehicle, specifically including the following steps S901 to S903.

[0103] S901, acquires a view of the area in front of the vehicle.

[0104] In this embodiment, the vehicle acquires a forward field of view image through the field of view image acquisition module 701. This field of view image includes the view in front of the driver's windshield, and may also include the view outside the left and right windows; this embodiment does not impose any limitations on this.

[0105] S902, based on the field of view image, an obstacle was detected in the first lane where the vehicle is currently located.

[0106] In this embodiment, the vehicle uses an image recognition algorithm via the field-of-view image element recognition module 702 to detect whether there are obstacles in the first lane where the vehicle is currently located. These obstacles include static objects that obstruct passage, such as fences, water-filled barriers, forks in the road, construction barriers, sand piles, earth piles, large rocks, goods scattered by other vehicles, and various parked vehicles (e.g., bicycles, motorcycles, electric vehicles). Alternatively, obstacles can be dynamic objects such as vehicles traveling outside the vehicle, pedestrians walking, or running animals. This embodiment does not limit the specific type of obstacle.

[0107] S903 uses AR-HUD to display AR navigation arrows based on the location of obstacles.

[0108] This embodiment does not limit the display style of the AR navigation arrow. It can be a solid line arrow as shown in Figure 10(a), or a fishbone arrow as shown in Figure 10(b), or other styles.

[0109] In S903, the impact of obstacles on traffic flow varies depending on their size, shape, and location within the first lane. For example, a large obstacle in the middle of the first lane typically affects vehicle passage. Conversely, an obstacle at the edge of the first lane and occupying a small area typically does not impede passage. Therefore, the timing and location of the AR navigation arrows will vary depending on the traffic conditions on each road.

[0110] To this end, the vehicle can first perform image recognition on the field of view to determine the traffic conditions of each lane, and then display AR navigation arrows based on the lane traffic conditions. See below for details.

[0111] Scenario 1, as shown in Figure 11A, involves an obstacle in the first lane where the vehicle is currently located, but it is still passable. In this case, the vehicle displays an AR navigation arrow indicating whether to proceed into the first lane, taking into account both the location of the obstacle and the location of the first lane.

[0112] Scenario 2, as shown in Figure 11B, involves an obstacle preventing passage in the first lane, but the second lane in the same direction is passable. In this case, the vehicle displays an AR navigation arrow instructing it to move from the first lane to the second lane, taking into account the location of the obstacle, the first lane, and the second lane. Alternatively, in this scenario, the vehicle displays a message indicating that the current lane is impassable and requesting the driver to move to another lane.

[0113] Scenario 3, as shown in Figure 11C, involves obstacles in the vehicle's current lane and other lanes in the same direction, all of which are impassable. In this case, the vehicle does not display AR navigation arrows but instead uses an AR-HUD to display a message indicating that the current road is impassable.

[0114] As can be seen, this embodiment can intelligently display AR navigation arrows based on the location of obstacles, making the display effect of AR navigation arrows more consistent with the actual road conditions and providing a better visual experience.

[0115] The following sections will provide specific explanations of the display of the AR navigation arrows under scenarios one through three.

[0116] Scenario 1: There is an obstacle in the first lane, but it is passable.

[0117] Vehicles may encounter various driving scenarios during operation, such as moving forward, turning left at an intersection, turning right at an intersection, making a U-turn, or heading towards a fork in the road. In different driving scenarios, the AR-HUD can combine the actual field of view image to display AR navigation arrows that match the actual road conditions in that driving scenario. The following examples illustrate the AR navigation arrow display method provided in this embodiment, using moving forward, turning left at an intersection, and heading towards a fork in the road as examples.

[0118] (1) The vehicle is moving in the first lane.

[0119] In this embodiment, the vehicle is currently traveling in the first lane, and according to navigation information, the vehicle should continue traveling in the first lane. In this embodiment, "traveling in the first lane" means that the vehicle continues to travel forward along the direction of the first lane. Since the first lane may be straight or curved, the actual path of the vehicle may be straight or curved. In other words, "traveling forward" should not be narrowly interpreted as traveling in a straight line.

[0120] In this scenario, if the vehicle detects an obstacle in the first lane that does not affect passage, such as the fence in Figure 11A, it will display an AR navigation arrow indicating forward movement, such as an AR forward arrow, in the first lane according to the location of the obstacle.

[0121] Figure 12 is a schematic flowchart of the AR navigation arrow display method in a forward-moving scenario provided in an embodiment of this application. This method is executed by the corresponding module in the vehicle and specifically includes the following steps S1201 to S1203.

[0122] S1201, Determine the traffic guide line for the first lane based on the position of the first lane and the position of the obstacle in the first lane.

[0123] In some embodiments, when the first lane is a straight line, the vehicle can determine a passable area of ​​the first lane and define the center line of that passable area as the traffic guide line for the first lane. This passable area is a passable area within the first lane that does not include obstacles.

[0124] For example, as shown in Figure 13, the first lane includes a first lane line and a second lane line. An obstacle fence occupies the first lane and is located near the second lane line. In the figure, curve AB is the baseline of the fence, A'B' is the projection line of the fence onto the second lane line, and the point on the fence baseline AB closest to the first lane line is feature point M. The baseline of the obstacle is the intersection line between the obstacle and the road surface in the field of view image. It can be understood that the area to the left of the feature point shown in the figure is the passable area of ​​the first lane. Vehicles can determine the midpoint N of the perpendicular line between feature point M and the first lane line. The straight line passing through this midpoint N and parallel to the first lane is the center line of this passable area, which vehicles can use as a traffic guide line.

[0125] In other embodiments, when the first lane is curved, the vehicle can determine the center line of the boundary lines on both sides of the passable area of ​​the first lane (i.e., the boundary center line) and use this boundary center line as a traffic guide line. Each point on this boundary center line is equidistant from the boundary lines on both sides; the boundary line includes the baselines of lane lines and obstacles.

[0126] For example, as shown in Figure 14, assuming there are no obstacles in the first lane, the boundary lines of the passable area of ​​the first lane are the lane lines on the left and right sides. Taking the lane line on the left side of the vehicle in the field of view as curve A1B1 and the lane line on the right side of the vehicle as curve A2B2 as an example, points A1 and B1 are projected onto curve A2B2 respectively, and the projection points are denoted as A′1 and B′1 respectively. Similarly, points A2 and B2 are projected onto curve A1B1 respectively, and the projection points are denoted as A′2 and B′2 respectively. The A′2B′2 segment of curve A1B1 and the A′1B′1 segment of curve A2B2 are intercepted. Traversing the points on one of the curves, a perpendicular line is drawn to the tangent of this curve and intersects the other curve. Connecting the midpoints of each perpendicular line segment yields the centerline of this curve. The projection point of a point onto the curve is the point on the curve closest to that point.

[0127] It is understandable that if there is an obstacle in the first lane, the boundary lines on one or both sides of the first lane are replaced with the baseline of the obstacle on that side. The traffic guide line for the first lane can then be determined using the method described above. For example, if there is no obstacle on the left side of the first lane but an obstacle on the right side, the traffic guide line for the first lane is determined based on the left lane line A1B1 and the baseline A2B2 of the obstacle on the right side of the first lane. Alternatively, if there is an obstacle on the left side of the first lane but no obstacle on the right side, the traffic guide line for the first lane is determined based on the baseline A1B1 of the obstacle on the left side of the first lane and the right lane line A2B2 of the first lane.

[0128] It should be noted that due to the presence of an obstacle in the first lane, the passable area for vehicles in the first lane is reduced. Therefore, the guide line determined based on this passable area deviates somewhat from the center line of the first lane. Furthermore, this guide line is more suitable for guiding vehicles to safely pass through the obstacle.

[0129] S1202, on the traffic guidance line, determine the display position of the AR forward arrow.

[0130] For example, the vehicle arrow determination module 706 can determine a point on the traffic guide line at a preset distance from the vehicle as the display position of the AR forward arrow. This preset distance can be 20 meters, 30 meters, etc., and this embodiment does not limit it.

[0131] S1203, along the traffic guide line of the first lane, display the AR forward arrow according to the display position of the AR forward arrow.

[0132] For example, the vehicle's AR-HUD displays an AR forward arrow, as shown in Figure 13, 20 meters away from the vehicle along the traffic guidance line of the first lane. Furthermore, this embodiment does not limit the display style of this AR forward arrow.

[0133] It should be noted that traditional technologies typically do not consider the impact of obstacles in the first lane and directly display the AR navigation arrow in the center of the first lane. This AR navigation arrow may cause the driver to drive in the direction of the arrow and encounter obstacles, posing a certain safety hazard. In this embodiment, however, when the vehicle is moving forward in the first lane, if there are obstacles in the first lane that do not affect passage, the vehicle can display an AR forward arrow in the center of the obstacle-free area of ​​the first lane to guide the vehicle safely. In comparison, the AR navigation arrow displayed in this application not only has a higher degree of integration with actual road conditions but also contributes to safe driving.

[0134] (2) The vehicle turns left at the intersection from the first lane.

[0135] In this embodiment, the vehicle is currently traveling in the first lane, and according to navigation information, the vehicle should turn left at the intersection from the first lane. In this scenario, if the vehicle detects an obstacle in the first lane that does not affect traffic, such as the fence in Figure 11A, it will display an AR navigation arrow indicating a left turn within the first lane based on the location of the obstacle.

[0136] Figure 15 is a schematic flowchart of the AR navigation arrow display method in a left-turn scenario provided in an embodiment of this application. This method is executed by the corresponding module in the vehicle and specifically includes the following steps S1501 to S1505.

[0137] S1501, based on the field of view image, identifies whether there is an intersection on the road ahead.

[0138] Because the navigation module 704 may contain inaccurate road information, if the navigation module 704 indicates that the vehicle is about to turn left at an intersection, the field-of-view image element recognition module 702 can first identify whether there is an intersection ahead to avoid errors in the AR navigation arrow display. For example, the field-of-view image element recognition module 702 can use image recognition algorithms such as BiSeNet, YOLO, and OCR to recognize the field-of-view image. If the field-of-view image identifies a traffic light ahead, and / or identifies a road sign indicating an intersection ahead, and / or identifies a large number of vehicles traveling laterally ahead, then it is determined that an intersection exists ahead.

[0139] When there is an intersection ahead, step S1502 is executed. Of course, the vehicle may also skip S1501, assume the navigation information is correct, and directly execute S1502; this embodiment does not impose any restrictions on this.

[0140] S1502, determine the traffic guide line for the first lane based on the position of the first lane and the position of the obstacle in the first lane.

[0141] The method for determining the traffic guidance line for the first lane is shown in S1201, and will not be repeated here.

[0142] S1503, determine the navigation point after the vehicle turns left from the first lane.

[0143] In this embodiment, the navigation point is the future driving route planned by the navigation module 704 based on the destination information and the vehicle's position. For example, as shown in Figure 16, the navigation point after the vehicle makes a left turn is the future driving route after the left turn. After making a left turn, the vehicle usually maintains a stable straight line within a certain lane; therefore, the navigation point after the left turn is basically a straight line. Thus, the vehicle can directly obtain the navigation point after making a left turn from the first lane from the navigation module 704.

[0144] S1504, determine the display position of the AR left turn arrow based on the navigation point after the vehicle turns left and the traffic guidance line of the first lane.

[0145] In this embodiment, the AR navigation arrow in a turning scenario is displayed at the intersection of the tangent line of the navigation line point after the turn and the traffic guidance line of the first lane. This turning scenario includes left turns and right turns. Therefore, referring to Figure 16, in a left turn scenario, the AR left turn arrow is displayed at the intersection of the tangent line of the navigation line point after the left turn and the traffic guidance line of the first lane, i.e., point M in Figure 16.

[0146] S1505, following the direction of the navigation point after turning left in the first lane, use AR-HUD to display the AR left turn arrow according to the display position of the AR left turn arrow.

[0147] For example, as shown in Figure 16, the vehicle's AR-HUD displays an AR left-turn arrow centered at point M, following the direction of the navigation point after turning left in the first lane. Of course, the AR-HUD module can also use point M as the starting point of the AR left-turn arrow; this embodiment does not restrict the position of point M on the arrow.

[0148] (3) The vehicle is traveling towards the intersection in the first lane.

[0149] In this embodiment, the vehicle is currently traveling in the first lane, and according to navigation information, the vehicle should proceed to the right-hand fork in the first lane. The AR navigation arrow is displayed as follows in this scenario.

[0150] Figure 17 is a schematic flowchart of the AR navigation arrow display method in a branch road driving scenario provided in an embodiment of this application. As shown in Figure 17, the method includes steps S1701 to S1702.

[0151] S1701, Identify road forks within the first lane in the field of view.

[0152] In this embodiment, the field-of-view image element recognition module 702 can identify the bifurcation point of the intersection using the BiSeNet algorithm and obtain the pixel coordinates of the bifurcation point.

[0153] S1702, Use AR-HUD to display AR navigation arrows along the direction of the fork in the road at the fork point.

[0154] For example, as shown in Figure 18, the vehicle's AR-HUD starts at the fork in the road and displays an AR navigation arrow facing the driver on the left side of the road wall, following the direction of the road wall at the fork. It should be noted that compared to displaying the AR navigation arrow in the middle of the fork, displaying the AR navigation arrow facing the user along the direction of the road wall at the fork is easier for the driver to see and provides better guidance.

[0155] Scenario 2: The first lane is blocked by an obstacle, while the second lane is passable.

[0156] In some embodiments, as shown in Figure 19A, when the first lane is blocked by an obstacle but the second lane is passable, the vehicle uses an AR-HUD to display an AR navigation arrow to guide the vehicle from the first lane to the second lane.

[0157] In other embodiments, as shown in Figure 19B, when the first lane is blocked by an obstacle but the second lane is passable, the vehicle can also use an AR-HUD or the vehicle's headlights (i.e., the vehicle's headlights) to project a strip-shaped area on the ground. This strip-shaped area extends from the first lane to the second lane and maintains a preset distance (e.g., 1 meter) from the obstacle to guide the vehicle through the area and avoid the obstacle.

[0158] Optionally, as shown in Figures 19A and 19B, the vehicle can also use an AR-HUD to display a message indicating that the first lane is impassable. For example, the message might be: "Lane ahead is blocked, please detour!"

[0159] Scenario 3: Obstacles exist in both the first lane and other lanes, making passage impossible.

[0160] In this embodiment, referring to Figure 20, when both the first lane and the second vehicle are blocked by obstacles, the vehicle does not display the AR navigation arrow, but instead uses an AR-HUD to display a warning message indicating that the road ahead is impassable. For example, this warning message could be: "Road ahead is impassable, please be careful!" or: "Obstacles exist ahead, impassable!"

[0161] It should be noted that in some embodiments, when multiple lanes in the same direction, such as the first and second lanes shown in Figure 20, are impassable, even if the opposite lane is passable, the vehicle does not use the AR-HUD to display information indicating that it should drive in the opposite direction to bypass the obstacle, in order to ensure driving safety.

[0162] During the display of the AR navigation arrows described above, the vehicle directly determines and displays the AR navigation arrows based on the information output by each module. In other embodiments, the vehicle may first determine the initial display position of the AR navigation arrows using conventional methods, and if there are obstacles in the first lane, correct the initial display position of the AR navigation arrows, and then display the AR navigation arrows based on the corrected position. Details are shown below.

[0163] Figure 21 is a flowchart of an AR navigation arrow display method provided in another embodiment of this application. Referring to Figure 21, the method specifically includes the following steps S2101 to S2103.

[0164] S2101, determine the initial display position of the AR navigation arrow based on the vehicle's location information, navigation data, and lane line information of the first lane.

[0165] In this embodiment, this step is specifically performed by the arrow determination unit 706A in the arrow determination module 706. For details on the process of determining the initial display position, please refer to relevant technologies; this embodiment will not elaborate further.

[0166] S2102, the initial position of the AR navigation arrow is corrected according to the position of the obstacle in the first lane to obtain the corrected display position.

[0167] In this embodiment, this step is specifically performed by the optimization and correction unit 706B in the arrow determination module 706. For example, the optimization and correction unit 706B can first determine the target display position of the AR navigation arrow based on the information output by other modules, and then replace the initial display position with the target display position. This target display position is the corrected display position. The specific process for determining this target display position can be found above and will not be repeated here.

[0168] S2103, based on the corrected position, use AR-HUD to display AR navigation arrows.

[0169] For S2101 to S2103, as shown in Figure 22, taking the scenario of a vehicle traveling from the first lane to a right-hand fork as an example, the vehicle can first use traditional technology to determine the initial display position of the AR navigation arrow as the intersection node of the fork, based on the vehicle's location information, navigation data, and lane line information of the first lane. Then, an image recognition algorithm is used to identify the intersection node and correct the display position of the AR navigation arrow from the intersection node to the fork point. Finally, an AR-HUD is used to display the AR navigation arrow at the fork point, following the direction of the fork in the first lane.

[0170] It should be noted that during the AR navigation arrow correction process described above, S2101 to S2102 are internal vehicle algorithms, which are imperceptible to the user. In other words, in scenarios where there is an obstacle in the first lane, the vehicle does not first display the AR navigation arrow at the initial display position and then move it to the corrected display position. Instead, it directly corrects the display position internally and then displays the AR navigation arrow directly at the corrected position.

[0171] In summary, through the method provided in this application embodiment, the electronic device can intelligently identify obstacles in the vehicle's current lane and intelligently display passage prompts based on the obstacles. For example, it can display AR navigation arrows based on the location of the obstacles, or display passage prompts based on the current lane and the passage conditions of other lanes. The display effect of the AR navigation arrows is highly integrated with the actual road conditions, resulting in a better user experience.

[0172] Vehicles may encounter unreliable scenarios while driving, such as road construction ahead, discrepancies between navigation information and actual road conditions, or vehicles traveling in the wrong direction. Based on this, the vehicle can identify the captured field-of-view images and issue warnings for these unreliable scenarios. These will be explained in detail below.

[0173] (1) Road construction ahead

[0174] The vehicle can use the field-of-view image element recognition module 702 to identify the field-of-view image captured by the vehicle using an image recognition algorithm to determine whether there is road construction ahead. If road construction is detected ahead, the corresponding prompt information is displayed using an AR-HUD. For example, the prompt information could be: "Road construction ahead, please proceed with caution."

[0175] (2) Navigation information does not match actual road conditions

[0176] Due to incomplete or outdated navigation information, there may be discrepancies between the navigation information and actual road conditions while driving. For example, the navigation app may instruct the vehicle to turn right, but there is no intersection on the right. Or, the navigation app may instruct the vehicle to proceed forward, but there is no road in the direction of travel.

[0177] In response to this situation, the vehicle can use the field-of-view image element recognition module 702 to identify information such as traffic lights, road signs, lane lines, and actual road images in the field-of-view image to determine the actual road conditions ahead. When the navigation information does not match the actual road conditions, the AR-HUD displays the corresponding warning message. For example, the warning message could be: "Road ahead is abnormal, please proceed with caution."

[0178] (3) Vehicles driving in the wrong direction

[0179] During traffic, vehicles may drift into the oncoming lane due to impassable roads or driver error, posing a safety hazard. To ensure vehicle safety, the system can identify the vehicle's position based on captured visual images. If a vehicle is detected traveling in the oncoming lane, a corresponding warning message will be displayed. See below for details.

[0180] Figure 23A is a schematic flowchart of the lane recognition method provided in an embodiment of this application. Referring to Figure 23A, the method specifically includes the following steps S2301 to S2303.

[0181] S2301, the vehicle acquires real-time field-of-view images.

[0182] Typically, the vehicle acquires field-of-view images at a preset frequency while in motion. For example, this frequency can be 30Hz, 60Hz, etc., and this embodiment does not limit it.

[0183] S2302, determine whether the vehicle is driving in the wrong direction based on the field of vision images within a preset time period.

[0184] In this embodiment, the relative driving directions between two vehicles include: traveling in the same direction and traveling in opposite directions. Traveling in the same direction means that the two vehicles are traveling in the same direction, while traveling in opposite directions means that the two vehicles are traveling in opposite directions. Vehicles can determine their relative driving direction to other vehicles in the same lane using image recognition algorithms based on visual field images within a preset time period. This embodiment does not limit the method used to determine whether a vehicle is traveling in the wrong direction.

[0185] In some embodiments, the vehicle uses an image recognition algorithm to identify visual field images within a preset time period. Within the same lane, it determines the proportion of other vehicles whose front is facing the vehicle. When this proportion exceeds a threshold, it is determined that the vehicle may be driving in the wrong direction. For example, the first threshold may be 80%, 90%, etc. Furthermore, the preset time period may be 5 seconds, 10 seconds, etc., and this embodiment does not impose any limitations on this.

[0186] In other embodiments, as shown in Figure 23B, the vehicle can determine the driving direction of each vehicle in the lane in the world coordinate system based on its own position information in the world coordinate system and in conjunction with an image recognition algorithm, thereby determining whether the vehicle is driving in the wrong direction. This method specifically includes the following steps S2302a to S2302f.

[0187] S2302a, determine the three-dimensional coordinates of the lane lines in the vehicle coordinate system in the field of view image.

[0188] As described above, the vehicle monitoring module 705 can obtain the three-dimensional coordinates of the actual lane line in the vehicle coordinate system. It can be understood that after the vehicle projects the three-dimensional coordinates into the field of view image, the individual pixels of the lane line in the field of view image can be determined.

[0189] S2302b: Based on the relative positional relationship between the target vehicle and the lane line in the field of view image, and based on the three-dimensional coordinates of the lane line in the vehicle coordinate system, determine the longitudinal distance between the target vehicle and the vehicle.

[0190] Taking the k-th frame of the field of view image as an example, the vehicle can first perform image segmentation on the field of view image to find the sub-region occupied by each target vehicle in each lane. Then, referring to Figure 24, the projection points of the wheels of each target vehicle on the lane line are determined. Finally, the longitudinal distance Z of the target vehicle relative to the vehicle is predicted by using the three-dimensional coordinates of the corresponding point on the lane line relative to the vehicle. In this embodiment, the longitudinal distance between two vehicles refers to the distance of the other vehicle in front of the vehicle along the driving direction of the vehicle.

[0191] S2302c determines the position coordinates of the target vehicle in the vehicle coordinate system based on the longitudinal distance between the vehicle and the target vehicle.

[0192] In some embodiments, the vehicle can determine the position coordinates of the target vehicle in the vehicle coordinate system based on the longitudinal distance between the vehicle and the target vehicle through a back-projection transformation. This is illustrated below.

[0193] Taking the vehicle coordinate system shown in Figure 5 as an example, let the position coordinates of the target vehicle in the vehicle coordinate system be P = (X, Y, Z), where Z is the longitudinal distance Z of the target vehicle relative to the vehicle itself, which has been determined in step S2303b. Let the pixel point of the car center in the field of view image be p = (u, v), and the intrinsic parameter matrix of the camera be known and K. Then, the parameters satisfy the following conditions: The formula can be transformed to obtain Thus, the position coordinates of the target vehicle in the vehicle coordinate system are determined as P = (X, Y, Z).

[0194] S2302d: Determine the position coordinates of the target vehicle in the world coordinate system based on the position coordinates of the target vehicle in the vehicle coordinate system.

[0195] It is understandable that since the position of the vehicle in the world coordinate system and the heading information of the vehicle can be detected, the position coordinates of the target vehicle in the vehicle coordinate system can be converted into position coordinates in the world coordinate system based on this information.

[0196] S2302e determines the driving direction of the target vehicle in the world coordinate system based on the position coordinates of the target vehicle in the world coordinate system in multiple frames of field-of-view images within a preset time period.

[0197] It is understandable that, for each frame of the field of view image, the vehicle can not only track and identify the target vehicle through image recognition algorithms (such as optical flow), but also perform the methods shown in S2302a to S2302d for each frame of the video image to determine the position information of the target vehicle in the world coordinate system in each frame of the field of view image. Therefore, based on the position information of the same target vehicle in the world coordinate system in multiple frames of images within a preset time period (e.g., within 5 seconds), the driving direction of the target vehicle can be determined.

[0198] S2302f determines whether the vehicle is traveling in the wrong direction based on its own driving direction and the driving direction of most vehicles on the same road within a preset time period.

[0199] For ease of description, this embodiment refers to the vehicle's direction of travel as the first direction and the direction opposite to the first direction as the second direction. Based on this, in some embodiments, the vehicle can count the proportion of vehicles traveling in the first direction within the same lane. If this proportion is higher than a first threshold, such as 80%, it is determined that the vehicle is not traveling in the wrong direction; if the proportion is lower than a second threshold (such as 10%), it is determined that the vehicle is traveling in the wrong direction. This embodiment does not limit the method for determining whether a vehicle is traveling in the wrong direction.

[0200] S2303 If a vehicle is traveling in the wrong direction, the AR-HUD will display the corresponding warning information.

[0201] In one example, the warning message could be: "Caution, a vehicle may be traveling in the wrong direction." Alternatively, the warning message could be: "You have entered the oncoming lane." This embodiment does not limit the specific content of the warning message.

[0202] Through the above steps S2301 to S2303, the vehicle can identify the visual field image over a period of time to determine whether the vehicle is driving in the wrong direction, and provide corresponding prompts when driving in the wrong direction to ensure driving safety.

[0203] In summary, this embodiment comprehensively utilizes key elements in the field of view image, such as vehicles, fences, water barriers, traffic lights, road signs, and intersections, when displaying AR navigation arrows and prompts. This makes the timing and location of the display of vehicle AR navigation arrows and prompts more consistent with actual road conditions, and it is compatible with various complex intersection scenarios, resulting in a superior user experience.

[0204] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0205] This application also provides an in-vehicle device configured to perform the navigation arrow display method shown in the above embodiments.

[0206] This application also provides a vehicle equipped with an AR-HUD, which is configured to perform the navigation arrow display method shown in the above embodiments.

[0207] It should be noted that the vehicles in this embodiment (including the vehicle itself and other vehicles around it) can be internal combustion engine vehicles that use an engine as a power source, hybrid vehicles that use an engine and an electric motor as power sources, electric vehicles that use an electric motor as a power source, and other vehicles with driving functions. This embodiment does not impose specific restrictions on the type of vehicle.

[0208] This application also provides a chip, as shown in FIG25, which includes a processor and a memory. The memory stores a computer program, which, when executed by the processor, implements the navigation arrow display method in the above embodiments.

[0209] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the navigation arrow display method provided in the above embodiments.

[0210] This application also provides a computer program product, which includes a computer program that, when run by an electronic device, enables the electronic device to implement the navigation arrow display method provided in the above embodiments.

[0211] It should be understood that the processor mentioned in the embodiments of this application can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0212] It should also be understood that the memory mentioned in the embodiments of this application can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous linked dynamic random access memory (SLDRAM), and direct rambus RAM (DRRAM).

[0213] In the embodiments provided in this application, the division of each framework or module is only a logical functional division. In actual implementation, there may be other division methods. For example, multiple frameworks or modules may be combined or integrated into another system, or some features may be ignored or not executed.

[0214] Furthermore, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0215] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0216] It should be noted that, in this application, the terms "exemplary" or "for example" are used to indicate that something is being described as an example, illustration, or illustration. Any embodiment or design described as "exemplary" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or design solutions. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0217] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0218] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for displaying navigation arrows, characterized in that, An in-vehicle device used in a vehicle, wherein the vehicle is equipped with a head-up display (HUD), the method includes: An obstacle was detected in the first lane where the vehicle is currently located; Determine the position of the obstacle in the field of view image; The HUD displays navigation arrows based on the location of the obstacle in the field of view image.

2. The method according to claim 1, characterized in that, The detection of an obstacle in the first lane where the vehicle is currently located includes: Acquire the field-of-view image of the vehicle; An obstacle was detected in the first lane where the vehicle is currently located based on the field of view image.

3. The method according to claim 1 or 2, characterized in that, The step of displaying navigation arrows using the HUD based on the position of the obstacle in the field of view image includes: When the obstacle in the first lane does not affect passage, the HUD displays a navigation arrow indicating passage through the first lane, based on the obstacle's position in the field of view.

4. The method according to any one of claims 1 to 3, characterized in that, The step of displaying navigation arrows using the HUD based on the position of the obstacle in the field of view image includes: When an obstacle in the first lane affects traffic, but a second lane traveling in the same direction as the first lane is passable, the HUD displays a navigation arrow indicating whether to enter the second lane from the first lane, based on the position of the obstacle in the field of view.

5. The method according to any one of claims 1 to 4, characterized in that, The step of displaying navigation arrows using the HUD based on the position of the obstacle in the field of view image includes: The display position of the navigation arrow is determined based on the position of the obstacle in the field of view image; The navigation arrow is displayed using the HUD according to its display position.

6. The method according to claim 5, characterized in that, When the navigation arrow is a forward arrow, determining the display position of the navigation arrow based on the position of the obstacle in the field of view image, and displaying the navigation arrow using the HUD based on the display position of the navigation arrow, includes: Based on the position of the obstacle in the field of view image, determine the passage guide line of the navigation arrow; The position of the navigation arrow is determined as the position of the navigation arrow at a preset distance from the vehicle on the traffic guidance line; Using the HUD, the navigation arrow is displayed along the guide line according to its display position.

7. The method according to claim 6, characterized in that, Determining the navigation arrow's guidance line based on the obstacle's position in the field of view image includes: When the first lane in the field of view is a straight lane, the passable area of ​​the first lane is determined according to the position of the obstacle, and the passable area does not include the obstacle; The center line of the passable area is determined as the traffic guide line for the first lane.

8. The method according to claim 6, characterized in that, Determining the navigation arrow's guidance line based on the obstacle's position in the field of view image includes: When the first lane in the field of view is a curved lane, the center line of the boundary line on both sides of the passable area of ​​the first lane is determined according to the position of the obstacle in the field of view. Each point on the center line is equidistant from the boundary line on both sides of the passable area. The center line is defined as the guide line for the navigation arrow.

9. The method according to claim 5, characterized in that, When the navigation arrow is a left-turn arrow or a right-turn arrow, determining the display position of the navigation arrow based on the position of the obstacle in the field of view image, and displaying the navigation arrow using the HUD based on the display position of the navigation arrow, includes: The navigation arrow's guide line is determined based on the location of the obstacle; Based on the navigation information and the vehicle's driving information, determine the trajectory tangent line of the vehicle when it is driving stably after making a left or right turn; The intersection of the guide line and the trajectory tangent is determined as the display position of the navigation arrow; Using the HUD, the navigation arrow is displayed along the direction of the tangent line of the trajectory after the vehicle makes a left or right turn, according to the display position of the navigation arrow.

10. The method according to claim 5, characterized in that, When the obstacle is a fork in the road, determining the display position of the navigation arrow based on the position of the obstacle in the field of view image, and displaying the navigation arrow using the HUD based on the display position of the navigation arrow, includes: The location of the bifurcation point is determined as the display position of the navigation arrow; Using the HUD, the navigation arrows are displayed at the fork in the road, following the direction of the fork.

11. The method according to claim 5, characterized in that, Determining the display position of the navigation arrow based on the position of the obstacle in the visual image includes: Based on the vehicle's navigation and location information, determine the initial display position of the navigation arrow; Based on the position of the obstacle in the field of view image, the initial display position is corrected to obtain the corrected display position; the corrected display position is the display position of the navigation arrow.

12. The method according to any one of claims 1 to 11, characterized in that, The obstacles include at least one of the following: fences, water-filled barriers, road signs, forks in the road, road construction barriers, pedestrians, and other vehicles.

13. The method according to any one of claims 1 to 12, characterized in that, The method further includes: Based on the field-of-view images within a preset time period, identify whether the vehicle is driving in the wrong direction; If the vehicle is traveling in the wrong direction, a message indicating that the vehicle is traveling in the wrong direction will be displayed.

14. A vehicle-mounted device, characterized in that, The vehicle-mounted device is configured to perform the method as described in any one of claims 1 to 13.

15. A vehicle, characterized in that, The vehicle includes the in-vehicle equipment as described in claim 14, and a head-up display (HUD).

16. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method performed as claimed in any one of claims 1 to 13.

17. A chip, characterized in that, It includes a processor and a memory, wherein the memory stores a computer program that, when executed by the processor, implements the method performed as claimed in any one of claims 1 to 13.

Images