Display method and device

Abstract

The embodiment of the invention provides a display method and device. The method comprises the following steps: acquiring a spatial position of a first road element and a display area of a head up display (HUD), wherein the display area of the HUD is used for displaying a virtual image; and displaying a first virtual element corresponding to the first road element in the display area when the first road element is outside the first boundary of the display area or a part of the first road element is inside the first boundary. According to the method and the device, even if a part or all of a certain road element is located outside the first boundary, the virtual element corresponding to the road element can still be displayed in the display area, so that the augmented reality effect can be prevented from being limited by the display area of the head-up display device, the waste of the performance of a sensing system can be avoided, and the user experience is improved. The expression ability of the augmented reality effect can be improved.

Description

Technical Field

[0001] This application relates to the field of intelligent vehicles, and more specifically, to a display method and apparatus. Background Technology

[0002] To enhance driving safety, head-up displays (HUDs) can be used to present essential driving information in a position visible to the driver. Augmented reality head-up displays (AR-HUDs) can combine navigation information and other data with the actual environment within the driver's field of vision, allowing for more convenient, intuitive, and efficient access to driving information, thus improving both safety and comfort.

[0003] However, due to the limited display area of ​​AR-HUD, information that matches the real scene may exceed the display range of AR-HUD while the vehicle is in motion, affecting the user experience. Summary of the Invention

[0004] This application provides a display method and apparatus that can avoid the augmented reality effect being limited by the display area of ​​the head-up display device, thereby improving the expressive power of the augmented reality effect.

[0005] Firstly, a display method is provided. This method includes: acquiring the spatial position of a first road element and the display area of ​​a HUD, the display area of ​​the HUD being used to display a virtual image; and displaying a first virtual element corresponding to the first road element in the display area when the first road element is outside a first boundary of the display area or a portion of the first road element is within the first boundary.

[0006] In this application, even if part or all of a road element is outside the first boundary, the virtual element corresponding to that road element can still be displayed in the display area. This avoids limiting the augmented reality effect to the display area of ​​the head-up display device, reduces wasted performance on the sensing system, and enhances the expressive power of the augmented reality effect.

[0007] In some possible implementations, the first virtual element may include a first form and a second form. The first form of the first virtual element can be used to display the first road element in the display area when the first road element is outside the first boundary of the display area or when a portion of the first road element is within the first boundary. The second form of the first virtual element can be used to display the first road element in the display area in a manner that fits the first road element when the first road element is within the first boundary of the display area.

[0008] In this application, when a road element is partially or entirely outside the display area, a first form of virtual element is used to display the corresponding virtual element; while when the road element is within the display area, a second form of virtual element can be attached to the road element for display. This method ensures that, on the one hand, even if the road element is outside the display range, the corresponding virtual element can still be displayed to avoid the loss of relevant information; on the other hand, when the road element is within the display area, relevant information can be provided to the user in a more intuitive way. Therefore, it can cater to the usage needs of different scenarios.

[0009] In some possible implementations, the first form may include a first portion that fits into the first boundary; the second form may include a first pattern that fits into the first road element.

[0010] In some possible implementations, when the first road element is outside the first boundary of the display area or a portion of the first road element is inside the first boundary, displaying the first virtual element corresponding to the first road element in the display area may include: when the first road element is outside the first boundary of the display area, displaying the first portion in the display area.

[0011] In this application, when a road element is outside the display area, the corresponding virtual element may include a first portion that is attached to the first boundary. Presenting the virtual element corresponding to the road element in a manner that is attached to the first boundary can avoid inappropriate display of the virtual element and thus avoid interference to the user.

[0012] In some possible implementations, the first form may also include a second part, which may be a portion of the first pattern. When the first road element is outside the first boundary of the display area or a portion of the first road element is inside the first boundary, displaying the first virtual element corresponding to the first road element in the display area may include: when a portion of the first road element is inside the first boundary and another portion is outside the first boundary, displaying the first part and the second part in the display area; the first part and the second part are adjacent.

[0013] In this application, when the first road element is outside the display area, the first virtual element may include a first portion that conforms to the first boundary; and when the first road element is within the display area, the first virtual element may include a first pattern that conforms to the first road element. When part of the first road element is outside the display area and another part is within the display area, displaying the first and second portions (i.e., a portion of the first pattern) of the virtual element in the display area facilitates a smooth transition between different forms of the virtual element.

[0014] In some possible implementations, the height of the first part near the end of the second part can be equal to the height of the second part near the end of the first part.

[0015] In some possible implementations, the first part may include a strip pattern extending along the first boundary.

[0016] In this application, by setting the first part as a strip pattern extending along the first boundary, during the process of a certain road element moving from outside the display area to inside the display area, the corresponding virtual element will be able to present the effect of the first pattern gradually unfolding from the rolled-up state of the strip until it is fully unfolded in the user's perception, thus achieving a smooth switching between different forms of the virtual element.

[0017] In some possible implementations, in the first form, the projected length of the first virtual element on the first boundary can be equal to the projected length of the first virtual element on the first boundary in the second form.

[0018] In this application, since the projection length of the first virtual element on the first boundary is the same in both different forms, when the first road element moves from outside the display area to inside the display area, it is possible to avoid sudden changes in the size of the first virtual element that could cause interference to the user.

[0019] In some possible implementations, the first virtual element can be used to display navigation information and / or prompts.

[0020] In some possible implementations, before displaying the first virtual element corresponding to the first road element in the display area, the method may further include: determining that the distance between the first road element and the vehicle is less than or equal to a first threshold.

[0021] In some possible implementations, the first road element may include a first fitting area; the first fitting area is: the area on the road element used to fit the virtual element when the road element is within the display area. If the first road element is outside the display area, the first fitting area may be outside the display area; if the first road element is within the display area, the first fitting area may be within the display area; if a portion of the first road element is within the display area, a portion of the first fitting area may be within the display area.

[0022] Secondly, an apparatus is provided. The apparatus may include an acquisition unit and a display unit. The acquisition unit is configured to: acquire the spatial position of a first road element and the display area of ​​a HUD, the display area of ​​the HUD being used to display a virtual image; the display unit is configured to: when the first road element is outside a first boundary of the display area or a portion of the first road element is within the first boundary, display a first virtual element corresponding to the first road element in the display area.

[0023] In some possible implementations, the first virtual element may include a first form and a second form. The first form of the first virtual element can be used to display the first road element in the display area when the first road element is outside the first boundary of the display area or when a portion of the first road element is within the first boundary. The second form of the first virtual element can be used to display the first road element in the display area in a manner that fits the first road element when the first road element is within the first boundary of the display area.

[0024] In some possible implementations, the first form may include a first portion that fits into the first boundary; the second form may include a first pattern that fits into the first road element.

[0025] In some possible implementations, the display unit can be used to display the first part in the display area when the first road element is outside the first boundary of the display area.

[0026] In some possible implementations, the first form may also include a second part, which may be a portion of the first pattern. The display unit may be used to: display the first part and the second part in a display area when a portion of the first road element is within the first boundary and another portion is outside the first boundary; the first part and the second part are adjacent to each other.

[0027] In some possible implementations, the height of the first part near the end of the second part can be equal to the height of the second part near the end of the first part.

[0028] In some possible implementations, the first part may include a strip pattern extending along the first boundary.

[0029] In some possible implementations, in the first form, the projected length of the first virtual element on the first boundary can be equal to the projected length of the first virtual element on the first boundary in the second form.

[0030] In some possible implementations, the first virtual element can be used to display navigation information and / or prompts.

[0031] In some possible implementations, before displaying the first virtual element corresponding to the first road element in the display area, the display unit may also be used to: determine that the distance between the first road element and the vehicle is less than or equal to a first threshold.

[0032] In some possible implementations, the first road element may include a first fitting area; the first fitting area is: the area on the road element used to fit the virtual element when the road element is within the display area. If the first road element is outside the display area, the first fitting area may be outside the display area; if the first road element is within the display area, the first fitting area may be within the display area; if a portion of the first road element is within the display area, a portion of the first fitting area may be within the display area.

[0033] Thirdly, an electronic device is provided, comprising: one or more processors; one or more memories; said one or more memories storing one or more programs that, when executed by said one or more processors, cause a display method as described in the first aspect and any possible implementation thereof to be executed.

[0034] Fourthly, a chip is provided, the chip including a processor and a communication interface, the communication interface being used to receive signals and transmit signals to the processor, the processor processing the signals such that the display method as described in the first aspect and any possible implementation thereof is executed.

[0035] Fifthly, a readable storage medium is provided, which stores instructions that, when executed on a device, cause a display method as described in the first aspect and any possible implementation thereof to be performed.

[0036] In a sixth aspect, a program product is provided, the program product including program code, which, when run on a device, causes the display method as described in the first aspect and any possible implementation thereof to be executed. Attached Figure Description

[0037] Figure 1 This is a schematic diagram of a HUD display method provided in an embodiment of this application;

[0038] Figure 2 This is a schematic diagram illustrating the display effect of an AR-HUD provided in an embodiment of this application;

[0039] Figure 3 This is a schematic diagram of a set of display interfaces provided in an embodiment of this application;

[0040] Figure 4 This is a schematic diagram of another set of display interfaces provided in an embodiment of this application;

[0041] Figure 5 These are schematic diagrams of two display interfaces provided in embodiments of this application;

[0042] Figure 6 This is a schematic diagram of another set of display interfaces provided in an embodiment of this application;

[0043] Figure 7 This is a schematic diagram of another set of display interfaces provided in an embodiment of this application;

[0044] Figure 8 These are schematic diagrams of two display interfaces provided in embodiments of this application;

[0045] Figure 9 This is a schematic diagram of another set of display interfaces provided in an embodiment of this application;

[0046] Figure 10 This is a schematic diagram of a display method provided in an embodiment of this application;

[0047] Figure 11 This is a schematic diagram of another display method provided in an embodiment of this application;

[0048] Figure 12 This is a schematic diagram of another display method provided in an embodiment of this application;

[0049] Figure 13 This is a schematic diagram of the structure of a device provided in an embodiment of this application;

[0050] Figure 14 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0051] The technical solutions in this application will now be described with reference to the accompanying drawings.

[0052] AR-HUD is a technology that combines augmented reality (AR) with a head-up display. Augmented reality uses technologies such as optoelectronic displays, interaction, and sensing to overlay and fuse computer-generated virtual information with the actual physical environment, enhancing the user's perception of the real world and achieving an enhanced effect. A head-up display displays information needed for driving in an area visible to the driver, allowing them to obtain information such as vehicle speed and navigation without having to look down at the instrument panel, center console screen, or other devices.

[0053] For example, Figure 1 This is a schematic diagram illustrating a display method for a HUD provided in an embodiment of this application.

[0054] like Figure 1 As shown, in augmented reality scenarios, the area used to present the virtual image of the AR-HUD can be referred to as the display area. In practical applications, the virtual image can be projected by the projection component in the HUD, and the virtual image seen by the human eye may be an image formed through multiple reflections or refractions.

[0055] In one application example, a HUD can project virtual information onto the windshield, such as... Figure 1 As shown. The image observed by the human eye may be a virtual image formed by reflection and / or refraction through the windshield. The user can see this virtual image through the windshield, and can also see the real environment in front of the vehicle; that is, the user can see a virtual image superimposed on the real scene.

[0056] In augmented reality scenarios, essential driving information such as navigation and warnings can be displayed as virtual images in front of the vehicle, and within the display area, they can be aligned with corresponding road elements in the surrounding environment (such as lanes, obstacles posing potential collision risks, and lanes / driving paths corresponding to the navigation information). The following combines... Figure 2 and Figure 3 This is an example illustration.

[0057] For example, Figure 2 This is a schematic diagram illustrating the display effect of the AR-HUD provided in the embodiments of this application.

[0058] like Figure 2 As shown in (a), vehicle 10 may be equipped with a HUD. This HUD can project a virtual image, which can be displayed in display area 100; correspondingly, from the driver's perspective, the virtual image superimposed on the real environment can be observed. For example, one or more pieces of information such as vehicle speed, road speed limit, and distance information between the vehicle and the next intersection can be displayed in display area 100 as virtual elements (such as virtual element 111), such as... Figure 2 As shown in (b) and (c) above. For example, the virtual element 111 can be updated in real time based on the corresponding information.

[0059] In one embodiment, it is assumed that there are road elements A and B in front of vehicle 10 that pose a potential collision risk. For example... Figure 2 As shown in (a), road element A can be a cyclist and road element B can be a pedestrian.

[0060] When road elements A and B are within display area 100, prompt patterns 112 and 113 can be displayed on display area 100; prompt patterns 112 and 113 can serve as prompts / warnings to the user, such as... Figure 2As shown in (b) in the diagram. In other words, the HUD can use augmented reality to warn users of potential risks by displaying cue patterns 112 and 113; cue patterns 112 and 113 can be understood as virtual elements corresponding to road elements A and B.

[0061] Furthermore, prompt pattern 112 can be aligned with road element A, and prompt pattern 113 can be aligned with road element B. For example, prompt patterns 112 and 113 can be aligned with the torso of the cyclist and pedestrian, respectively; when the cyclist or pedestrian moves within the display area 100, prompt patterns 112 and 113 can move along with the road elements A and B, respectively.

[0062] When part or all of the cyclist's / pedestrian's torso is outside the display area 100, the prompt patterns 112 and 113 cannot be attached to the corresponding cyclist / pedestrian. To avoid inappropriately attaching the prompt patterns and causing confusion to the user, the prompt patterns 112 and 113 may not be displayed within the display area 100; that is, the virtual elements corresponding to road elements A and B may not be displayed. For example, as... Figure 2 As shown in (c), when road element A is outside the left border of display area 100, the prompt pattern 112 cannot be aligned with road element A, and therefore, it may not be displayed within display area 100. For example, suppose the area on road element B used to align the prompt pattern 113 is the torso. When part of this area is within the right border of display area 100 while the rest is outside, the prompt pattern 113 cannot be aligned with that area; in this case, the prompt pattern 113 may not be displayed within display area 100, as... Figure 2 As shown in (c) in the figure.

[0063] However, if the vehicle 10 can detect road elements with collision risks (such as road elements A and B mentioned above) through sensors such as cameras, and cannot alert the user to the risk due to the limited display area, it will not only be detrimental to the safety of the vehicle and the user, but also result in a significant waste of the performance of the vehicle's perception function.

[0064] Therefore, embodiments of this application provide a display method and apparatus. In this technical solution, even if road elements exceed the display area, corresponding virtual elements can still be displayed within the display area, thus avoiding the limitation of augmented reality effects on the display area of ​​the head-up display device and improving the expressive power of augmented reality effects.

[0065] The following section will describe the display method and apparatus of this application embodiments in conjunction with several display interfaces.

[0066] For example, Figure 3 This is a schematic diagram of a set of display interfaces provided in an embodiment of this application. Figure 3 The displayed interface shown can be understood as the display interface corresponding to the virtual image projected by the HUD from the driver's perspective; the following... Figures 4 to 9 The displayed interface also uses the same perspective, which will be explained here and will not be repeated below. Among them, Figure 3 (a) to (d) in the text can correspond to the scene where road elements A and B move from outside the display area 100 to inside the area.

[0067] See Figure 3 (a) indicates that the display interface can be used when both road elements A and B are outside the display area 100.

[0068] At this time, virtual element 122 can be displayed in display area 100; virtual element 122 can correspond to road element A to indicate to the user that there is a pedestrian on the left (e.g., a cyclist, pedestrian, elderly person, child, etc.). Virtual element 123 can also be displayed in display area 100; virtual element 123 can correspond to road element B to indicate to the user that there is a pedestrian on the right. For example, virtual element 122 can include text information (also called text-image) such as "Caution: Pedestrians" or "Caution: Pedestrians on the Left"; similarly, virtual element 123 can include text-images such as "Caution: Pedestrians" or "Caution: Pedestrians on the Right" to remind the user that there is a pedestrian on the right. Furthermore, to prevent misleading information caused by inappropriate display position of virtual elements, virtual element 122 can be displayed in the left-side area of ​​display area 100, such as the upper left, lower left, or left edge; similarly, virtual element 123 can be displayed in the right-side area of ​​display area 100, such as the upper right, lower right, or right edge.

[0069] For example, the relative positions of road elements A and B with display area 100 may change due to the movement of road elements A and B themselves and / or the movement of vehicle 10. For instance, when road element A is outside the left boundary of display area 100 and a portion of road element B is inside the right boundary of display area 100, display area 100 may display as follows: Figure 3 The virtual image shown in (b) is as follows. For example, as road elements A and B continue to move relative to display area 100, when a portion of road element A is within the left boundary of display area 100 and a portion of road element B is within the right boundary of display area 100, display area 100 can display as shown in [example image]. Figure 3 The interface shown in (c) is as follows. For example, as road elements A and B continue to move relative to display area 100, when road elements A and B are within display area 100, display area 100 can display as shown in [image / description]. Figure 3 The interface shown in (d) is shown in the image.

[0070] It is understood that the embodiments of this application do not limit the display area 100 to entering the display area as shown in the example. Figure 3 The specific interface shown in (b) and (c) is as follows. For example, when the HUD is enabled, road elements A and B may already be in the position shown in (b) and (c). Figure 3 The position shown in (c) is shown in the diagram; in this case, display area 100 may not display the position shown in the diagram. Figure 3 The interfaces shown in (a) and (b) are shown in the figure.

[0071] For a given road element, whether it is outside the display area can be determined based on the positional relationship between the area on the road element used to attach to the virtual element and the display area. In other words, if part of the road element is outside the display area, it can be understood that a portion of the area on the road element used to attach to the virtual element (such as the torso of road element B) is outside the display area; if the road element is inside the display area, it can be understood that the entire area is inside the display area; and if the road element is outside the display area, it can be understood that the entire area is outside the display area. This is a general explanation and will not be repeated below.

[0072] See Figure 3 In (b), the display interface can be a display interface when road element A is outside the display area 100 and a portion of road element B is inside the display area 100. For example, for road elements A and B, both are located outside the display area 100. Figure 3 The position shown in scenario (b) can be compared with the two in Figure 2 The positions are the same in the scenario shown in (c).

[0073] See Figure 3 In (c), the display interface can be the display interface when part of road element A is within the left boundary of display area 100 and part of road element B is within the right boundary of display area 100.

[0074] See Figure 3 In section (d), the display interface can be the interface when road elements A and B are within the display area 100. When road elements A and B are within the display area 100, virtual elements 122 and 123 can be aligned with the corresponding road elements; in this case, it is not necessary to use text or images to prompt the user, but instead use virtual images that can be aligned with the road elements (such as...). Figure 2 (b) uses prompt patterns 112 and 113 to remind the user.

[0075] In one embodiment, road elements A and B are movable relative to the vehicle. When road elements A and B are still outside the display area 100, or only partially within the display area 100, since virtual elements 122 and 123 are difficult to align with their corresponding road elements A and B, they can be displayed as text or icons. However, when road elements A and B move into the display area 100, the prompt icons can align with their corresponding road elements, and virtual elements 122 and 123 can be displayed as prompt icons, thus more intuitively enhancing the user's perception of the real environment.

[0076] In this embodiment, for cases where road elements A and B are partially or entirely outside the display area 110, the display method of this application is exemplarily described by using text patterns to represent the virtual elements 122 and 123 as an example. In other embodiments, under the same circumstances, other methods (such as obstacle models, arrow patterns, bar patterns, etc.) can be used to present the virtual elements 122 and 123 in the display area 110. The following is combined with... Figures 4 to 6 Other ways of presenting virtual elements are illustrated by example.

[0077] For example, Figure 4 This is a schematic diagram of another set of display interfaces provided in the embodiments of this application. Wherein, Figure 4 (a) to (d) in the text can correspond to the scene where road elements A and B move from outside the display area 100 to inside the area.

[0078] When road elements A and B are outside the display area 100 or partially within the display area 100, it differs from... Figure 3 In sections (a) to (c), virtual elements 122 and 123 can be presented as obstacle models to alert the user to obstacles in the surrounding environment that pose a collision risk. For example, if the obstacle type corresponding to road elements A and B is pedestrians, the corresponding obstacle model can be displayed in display area 100, such as... Figure 4 (a) to (c) in the middle.

[0079] When road elements A and B are within the display area of ​​100, and Figure 3 Similar to (d) in the example, cue patterns can be used to display the virtual elements 122 and 123. Compared to obstacle models, cue patterns can more intuitively enhance the user's perception of the real environment.

[0080] In this embodiment of the application, taking pedestrians as an example of obstacle type, the virtual element is presented as an obstacle model in the display area 100, thereby providing an exemplary description of the display method of this application. It can be understood that when the road element corresponds to other obstacle types (such as plants, vehicles, etc.), the obstacle model corresponding to that type can be displayed in the display area 100.

[0081] In other embodiments, the method of classifying obstacle types can be the same as... Figure 4 The methods shown differ. For example, although both road element A and road element B represent pedestrians, they can correspond to different obstacle types; that is, pedestrians can be further subdivided into various types such as walkers and cyclists. The following combines... Figure 5 This is an example illustration.

[0082] For example, Figure 5 These are schematic diagrams of two display interfaces provided in embodiments of this application. Wherein, Figure 5 Images (a) and (b) show the situation when road elements A and B are outside the display area 100. For example, for road elements A and B, both are... Figure 5 The positions shown in (a) and (b) can be compared with their positions in the scenarios depicted. Figure 3 The positions are the same in the scenario shown in (a).

[0083] In one embodiment, unlike Figure 4 As shown, road element A and road element B can correspond to different obstacle types, such as cyclists and pedestrians, respectively; correspondingly, the obstacle models for these two types can be called the cyclist model and the pedestrian model, respectively. For example, as... Figure 5 As shown in (a), when both road elements A and B are outside the display area 100, the display area 100 can display virtual elements 122 and 123; where, corresponding to road element A, virtual element 122 can be presented using a cyclist model, and virtual element 123 can be presented using a pedestrian model. For example, similar to... Figure 3 In (b) and (c), when the road elements corresponding to virtual elements 122 and 123 are partially within the display area 100, virtual elements 122 and 123 can be displayed in the display area 100 in the manner of cyclist model and pedestrian model, respectively.

[0084] In another embodiment, an arrow pattern can be used to indicate the direction of movement of a road element. For example, assuming road element A moves to the right, when road element A is outside the display area 100, virtual element 122 may include a model of a cyclist and may also include a right arrow pattern; the right arrow pattern can indicate that road element A is moving to the right, such as... Figure 5 As shown in (b) above. For example, assuming road element A moves to the left, when road element B is outside the display area 100, virtual element 123 may include a model of a pedestrian and a pattern of a left arrow; the left arrow may indicate that road element B moves to the left.

[0085] In the above Figure 5 In (a), only obstacle models are used to present virtual elements 122 and 123; Figure 5 In (b), virtual elements 122 and 123 can be presented using a combination of arrow patterns and obstacle models. Among some possible implementations, [the following is related to...] Figure 5 The methods shown in (a) and (b) are different; the virtual elements 122 and 123 can be presented using only arrow patterns.

[0086] In the above Figures 3 to 5 In the proposed solution, when a road element is outside the display area 100 or part of it is within the display area 100, a method (such as text or pattern) can be used to display the virtual element corresponding to the road element within the display area 100; while when the road element is within the display area 100, another method can be used to present the virtual element corresponding to the road element.

[0087] In this implementation, when a road element moves from outside the display area 100 to inside the display area, the corresponding virtual element will switch / transition from one form to another. If the difference between the two forms is significant, the transition process may not be smooth enough and could cause user interference. Furthermore, when a road element is outside the display area 100 or partially within it, the corresponding virtual element can be displayed as a bar pattern at the corresponding boundary position of the display area 100 to achieve a smooth transition between different forms. The following combines... Figure 6 This is an example illustration.

[0088] For example, Figure 6 This is a schematic diagram of another set of display interfaces provided in an embodiment of this application. Wherein, Figure 6 (a) to (d) show the case where road elements A and B move relative to each other from outside the display area 100 to inside the display area 100.

[0089] Assuming that road elements A and B are within the display area of ​​100, the corresponding virtual elements are presented using prompt icons, and the prompt icons are aligned with the corresponding road elements, such as... Figure 3 As shown in (d) in the figure.

[0090] See Figure 6In (a), the display interface can be one where both road elements A and B are outside the display area 100. For example, since road element A is outside the left boundary of the display area 100, virtual element 122 in this interface may include a bar pattern located near or adjacent to the left boundary. Similarly, since road element B is outside the right boundary of the display area 100, virtual element 123 in this interface may include a bar pattern located near or adjacent to the right boundary.

[0091] See Figure 6 In (b), the display interface can be the display interface when road element A is outside the display area 100 and part of road element B is inside the display area 100.

[0092] See Figure 6 (c) In this context, the display interface can be the display interface when part of road element A is within the left boundary of display area 100 and part of road element B is within the right boundary of display area.

[0093] When part of road elements A and B is within the display area 100 and another part is outside the display area 100, virtual elements 122 and 123 may include bar patterns that fit the corresponding boundaries of the display area 100, and may also include a part of the corresponding prompt pattern.

[0094] In one embodiment, such as Figure 6 As shown in (b) and (c), a portion of the road element B is within the right boundary of the display area 100 while another portion is outside that boundary. The virtual element 123 may include virtual portions 1231 and 1232; virtual portion 1231 may be a bar pattern, and virtual portion 1232 may be a part of the prompt pattern 113. For example, virtual portions 1231 and 1232 may be adjacent. For another example, the height of the virtual element 123 may be equal to the height of the prompt pattern 113, or the height difference between the virtual element 123 and the prompt pattern 113 may be less than or equal to a certain threshold. For another example, virtual portion 1232 can be understood as the portion of the prompt pattern 113 within the right boundary of the display area 100, assuming that the prompt pattern 113 can be fitted to the road element B without being limited by the display area. For another example, virtual portion 1231 can be understood as the portion of the prompt pattern 113 within the right boundary of the display area 100, assuming that the prompt pattern 113 can be fitted to the road element B without being limited by the display area. Figure 6 The portion of the bar pattern 123 shown in (a) that is not covered by the virtual portion 1232.

[0095] In yet another embodiment, such as Figure 6As shown in (c), a portion of road element A is within the right boundary of display area 100 while another portion is outside that boundary. Virtual element 122 may include virtual portion 1221 and virtual portion 1222. Virtual portion 1221 may be a bar pattern, and virtual portion 1222 may be a part of prompt pattern 112. For example, virtual portions 1221 and 1222 may be adjacent. For another example, the height of virtual element 122 may be equal to the height of prompt pattern 112, or the height difference between virtual element 122 and prompt pattern 112 may be less than or equal to a certain threshold. For another example, virtual portion 1222 can be understood as the portion of prompt pattern 112 within the left boundary of display area 100, assuming that prompt pattern 112 can be attached to road element A without being limited by the display area. For another example, virtual portion 1221 can be understood as the portion of prompt pattern 112 within the left boundary of display area 100, assuming that prompt pattern 112 can be attached to road element A without being limited by the display area. Figure 6 The portion of the bar pattern 122 shown in (a) that is not covered by the virtual portion 1222.

[0096] For example, taking road element B as an example, when road element B moves from outside the right boundary of display area 100 into display area 100, within display area 100, virtual element 123 will smoothly transition from a bar pattern to a prompt pattern. Correspondingly, from the user's perspective, this can be perceived as: prompt pattern 113 changing from a rolled-up state (corresponding to...) Figure 6 The virtual element 123 shown in (a) slowly expands to the left (corresponding to...). Figure 6 The virtual element 123 shown in (b) and (c) remains in the fully expanded state until it is fully expanded (corresponding to...). Figure 6 The virtual element 123 shown in (d) moves along with the road element B. In this way, a smooth transition effect can be achieved for the virtual elements.

[0097] The above combination Figures 3 to 6 Taking pedestrians as an example, the display method of this application is illustrated by way of example. It is understood that road elements can be other types besides pedestrians. For example, road elements can include: moving and / or stationary obstacles (such as crash cones, fences, vehicles, animals, trees, etc.), road surfaces, lanes, etc.

[0098] The following example uses road elements to represent the roads / lanes corresponding to navigation information. Figures 7 to 9 The display method of this application is illustrated by way of example.

[0099] For example, Figure 7 This is a schematic diagram of another set of display interfaces provided in an embodiment of this application. Wherein, Figure 7(a) to (c) show the display interface when vehicle 10 moves from pose 1 to pose 3.

[0100] See Figure 7 In (a), the display interface can be the display interface when the vehicle 10 is in position 1. The display area 100 may include a navigation pattern 152, which can be aligned with the road corresponding to the navigation information to provide more intuitive navigation for the user.

[0101] The relative position between the road and the display area 100 may change due to the movement of the vehicle 10. During vehicle movement, there may be instances where part or all of the road is outside the display area 100, and the display area 100 may only display a portion of the navigation pattern 152, or even not display the navigation pattern at all. For example, the vehicle may travel from position 1 to position 3 via position 2. Or, for example, when the vehicle 10 is in position 3, the road corresponding to the navigation information may be outside the display area 100, and the display area 100 may display, as shown in the image below. Figure 7 The interface shown in (b) is as follows; that is, in order to avoid the navigation pattern 152 from being inappropriately matched with the real environment and causing interference to the user, the navigation pattern 152 may not be displayed in the display area 100.

[0102] However, if the navigation symbol 152 is not displayed, it will be difficult for users to obtain navigation information effectively, which is detrimental to the safety of users and vehicles.

[0103] In some possible implementations, when the road corresponding to the navigation information is outside the display area 100, the virtual elements corresponding to the road can be presented using text, patterns, or other methods.

[0104] For example, Figure 8 These are schematic diagrams of two display interfaces provided in embodiments of this application. Wherein, Figure 8 (a) and (b) show two possible display interfaces when the vehicle 10 is in pose 3.

[0105] Reference Figure 8 In (a), when the vehicle 10 is in position 3, a virtual element 162 can be displayed within the display area 100 to indicate navigation information; the virtual element 162 may include text and patterns such as "Please turn left", "Please turn into the left road", "Please turn into the second lane on the left".

[0106] Reference Figure 8 In (b), when the vehicle is at position 3, virtual element 162 can be displayed in the display area 100 in the form of an arrow pattern to indicate navigation information.

[0107] In one embodiment, when the road corresponding to the navigation information is within the display area 100, it can be adopted... Figure 7 The method shown in (a) is user navigation; however, when part or all of the road corresponding to the navigation information is outside the display area 100, an alternative method can be used. Figure 8 The method shown is user navigation. Under this technical solution, during vehicle movement, the virtual elements corresponding to the road will switch / transition between two different presentation methods; the switching / transition process is not smooth enough and may cause interference to the user. Figure 6 Similarly, when part or all of the road is outside the display area, the corresponding virtual element can be displayed as a bar pattern at the boundary position of the display area 100 to achieve a smooth transition between different forms. The following combines... Figure 9 This is an example illustration.

[0108] For example, Figure 9 This is a schematic diagram of another set of display interfaces provided in the embodiments of this application. Figure 9 Images (a) to (c) show the display interface when vehicle 10 moves from pose 1 to pose 3. Figure 9 In the diagram, navigation pattern 152 is represented by a shaded diagonal line, while virtual element 162 is represented by a gray pattern.

[0109] The display interface when the vehicle is in position 1 can be as follows: Figure 9 As shown in (a) above, the display area 100 can display virtual element 162 to indicate navigation information. This virtual element 162 can be aligned with the road corresponding to the navigation information. The road corresponding to the navigation information (or navigation pattern 152) is located above the lower boundary of the display area 100.

[0110] See Figure 9 (b) in the diagram refers to the display interface when the vehicle is in position 2.

[0111] For example, assuming the vehicle is in pose 2, part of the road corresponding to the navigation information is outside the lower boundary of display area 100, and another part of the road is inside the lower boundary of display area 100. Virtual element 162 may include virtual portion 1621 and virtual portion 1622; virtual portion 1621 may be a bar pattern, and virtual portion 1622 may be a part of navigation pattern 152. For example, virtual portions 1621 and 1622 may be adjacent. For another example, the width of virtual element 162 may be the same as the width of navigation pattern 152, or the width difference between virtual element 162 and the width of navigation pattern 151 may be less than or equal to a certain threshold. For another example, virtual portion 1622 can be understood as the portion of navigation pattern 152 within display area 100 that is aligned with the road regardless of the display area. For another example, virtual portion 1621 can be understood as the portion of navigation pattern 152 within display area 100 that is aligned with the road regardless of the display area. Figure 9 The portion of the bar pattern 162 shown in (c) that is not covered by the virtual portion 1622.

[0112] See Figure 9 (c) In this context, the display interface can be the display interface when the vehicle is in position 3. For example, the virtual element 162 can be displayed in the display area 100 in the form of a bar pattern; the bar pattern can be aligned with the lower boundary of the display area 100.

[0113] For example, as the vehicle moves from position 1 to position 3, within the display area 100, the virtual element 162 will smoothly transition from a navigation pattern to a bar pattern. Correspondingly, from the user's perspective, this will appear as the navigation pattern 152 transitioning from a fully expanded state (corresponding to...) Figure 9 The virtual element 162 shown in (a) slowly collapses downwards. Figure 9 The virtual element 162 shown in (b) is until it is completely collapsed. Figure 9 (c) shows the virtual element 162). This allows for a smooth transition effect between virtual elements.

[0114] The above combination Figure 9 Examples (b) and (c) illustrate the situation where part or all of the road / lane corresponding to the navigation information is outside the lower boundary of the display area 100.

[0115] In some embodiments, when part or all of the road / lane is outside the left / right boundary of the navigation area, since the distance between the road / lane on the left / right side of the display area and the vehicle is relatively large, the impact of the missing navigation information in this part can be considered small, and it is not necessary to use such navigation information. Figure 9The bar charts in (b) and (c) show the virtual elements corresponding to this section of the road / lane.

[0116] For example, Figure 10 This is a schematic diagram of a display method provided in an embodiment of this application. For example... Figure 10 As shown, the method 500 may include steps S510 and S520:

[0117] S510, obtain the spatial position of the first road element and the display area of ​​the head-up display device.

[0118] For example, the first road element can be any road element. Road elements can include: moving and / or stationary obstacles (such as crash cones, fences, vehicles, animals, trees, pedestrians, etc.), roads, lanes, etc. For example, see Figure 3 The first road element can be road element A or road element B. For example, see [link to example]. Figure 8 The first road element can be the road / lane corresponding to the navigation information.

[0119] The display area of ​​a head-up display can be used to display virtual images. For example, the display area can display images such as... Figures 2 to 9 The interface shown. For example, based on the driver's control information of the vehicle (such as one or more of the vehicle's steering angle, vehicle's pitch angle, etc.) and the arrangement of the head-up display in the vehicle, the position of the display area in space can be determined.

[0120] S520, when the first road element is outside the first boundary of the display area, or when a part of the first road element is inside the first boundary, the first virtual element corresponding to the first road element is displayed in the display area.

[0121] For example, refer to Figures 3 to 6 The virtual element corresponding to road element A can be virtual element 122; the virtual element corresponding to road element B can be virtual element 123. For example, refer to... Figure 9 The first road element can be the road / lane corresponding to the navigation information; the virtual element corresponding to this road element can be virtual element 162.

[0122] In this embodiment of the application, even if part or all of a road element is outside the first boundary, the virtual element corresponding to the road element can still be displayed in the display area. In this way, the augmented reality effect can be avoided from being limited by the display area of ​​the head-up display device, the performance of the perception system can be avoided, and the expressive ability of the augmented reality effect can be improved.

[0123] For example, if a road element is outside the display area, it can be understood that the first fitting area on the road element is outside the display area; if a road element is inside the display area, it can be understood that the first fitting area on the road element is inside the display area; if a part of a road element is inside the display area, it can be understood that a part of the first fitting area on the road element is inside the display area. The first fitting area can be used as: the area on the road element used to fit the virtual element when the road element is inside the display area.

[0124] For example, for Figures 3 to 6 For road elements A and B in the context of navigation, their body portion can be considered the first fitting area. Similarly, for a road / lane corresponding to navigation information, the road surface at its center can correspond to the first fitting area. Furthermore, for different road elements, the first fitting area can be determined based on specific requirements and implementation methods.

[0125] In some embodiments, before displaying the first virtual element corresponding to the first road element in the display area, the method may further include: determining that the distance between the first road element and the vehicle is less than or equal to a first threshold. The first threshold may be 4 meters, 5 meters, or other values.

[0126] For example, the first threshold can be related to the type of road element. For instance, when the road element is an obstacle, the first threshold can be 3 meters or 4 meters to reduce interference caused by obstacles that pose no risk of collision. Or, for instance, when the road element is a road / lane corresponding to navigation information, the first threshold can be 15 meters or 20 meters.

[0127] In real-world scenarios, multiple obstacles may exist in the vehicle's surrounding environment; some of these obstacles pose a collision risk, while others may not. For obstacles posing a collision risk, if they are outside or partially within the display area, displaying corresponding virtual elements within the display area can alert the user. However, for obstacles posing no collision risk, if they are outside or partially within the display area, displaying their corresponding virtual elements within the display area might actually be distracting for the user.

[0128] In this embodiment, the determination of whether to display the corresponding virtual element in the display area is based on the distance between the road element and the vehicle can reduce invalid information presented in the display area and reduce the interference to the user caused by too many invalid virtual elements in the display area.

[0129] In some possible implementations, the first virtual element may include a first form and a second form. The first form of the first virtual element is used to display the first road element in the display area when the first road element is outside the first boundary of the display area, or when a portion of the first road element is within the first boundary. The second form of the first virtual element can be used to display the first road element in the display area in a manner that fits the first road element when the first road element is within the first boundary of the display area.

[0130] For example, with Figure 3 Taking the scenario shown as an example, for road element A, the first form of virtual element 122 can be a text pattern, such as... Figure 3 As shown in (a) to (c); the second form of virtual element 122 can be a cue pattern, such as Figure 3 As shown in (d) in the figure.

[0131] For example, with Figure 6 Taking the scenario shown as an example, for road element B, the first form of virtual element 123 can be a bar pattern, such as... Figure 6 As shown in (a) to (c); the second form of virtual element 123 can be a cue pattern, such as Figure 6 As shown in (d) in the figure.

[0132] For example, with Figure 9 Taking the scenario shown as an example, for the road / lane corresponding to the navigation information, the first form of virtual element 162 can be a bar pattern, such as... Figure 9 As shown in (b) and (c) in the diagram; the second form of virtual element 162 can be a navigation pattern, such as... Figure 9 As shown in (a) in the figure.

[0133] In this embodiment, when part or all of a road element is outside the display area, the corresponding virtual element is displayed in a first form; and when the road element is within the display area, the corresponding virtual element can be displayed in a way that fits the road element. In this way, on the one hand, even if the road element is outside the display range, the corresponding virtual element can still be displayed to avoid the loss of relevant information; on the other hand, when the road element is within the display area, relevant information can be provided to the user in a more intuitive way. Therefore, it can meet the usage needs of different scenarios.

[0134] In some possible implementations, the first form may include a first portion that fits into the first boundary, and the second form may include a first pattern that fits into the first road element.

[0135] For example, the first form may include only the first part, or it may also include other parts. For example, in Figure 3In the scenario shown, for virtual element 122, the first form / first part can be text or a pattern, such as... Figure 3 As shown in (a) to (c) in the diagram; the second form may include cue patterns, such as Figure 3 As shown in (d) in the text. For example, as... Figure 9 As shown in (b) and (c), for virtual element 162, the first part may include a bar pattern; as Figure 9 As shown in (a) above, the first pattern may include a navigation pattern. For example, as... Figure 6 As shown in (b) and (c), for virtual element 123, virtual part 1231 can correspond to the first part, and virtual element 123 in the first form can also include virtual part 1232.

[0136] For example, the first part being attached to the first boundary can be understood as: the distance between the first part and the first boundary is less than or equal to a certain threshold; and, when the first road element moves relative to the first boundary, the relative distance between the first part and the first boundary remains unchanged.

[0137] For example, when virtual element 123 contains a bar pattern, the bar pattern can be aligned with the right boundary of display area 100; there may even be no gap between the bar pattern and the right boundary of display area 100, such as... Figure 6 As shown in (a) to (c). For example, when the virtual element 123 is presented as a text image, a gap may exist between the text image and the right boundary of the display area 100, but the distance between the text image and the right boundary can remain constant, such as... Figure 3 As shown in (a) to (c) in the figure.

[0138] In this embodiment of the application, when the first road element exceeds the display area, the first virtual element includes a first portion that fits the first boundary, which can avoid interference to the user caused by improper display of the first virtual element.

[0139] In some possible implementations, displaying a first virtual element corresponding to the first road element in the display area when the first road element is outside the first boundary of the display area can include: displaying a first portion in the display area when the first road element is outside the first boundary of the display area. For example, as... Figure 3 (a) Figure 4 (a) Figure 6 (a) Figure 9 As shown in (c) in the figure.

[0140] In some possible implementations, the first form may also include a second part, which may be a portion of the first pattern. When a portion of the first road element is within the first boundary of the display area, displaying a first virtual element corresponding to the first road element in the display area may include: when a portion of the first road element is within the first boundary and another portion is outside the first boundary, displaying both the first and second portions in the display area, with the first and second portions adjacent to each other.

[0141] For example, such as Figure 6 As shown in (b) and (c), for virtual element 123, virtual part 1231 can correspond to the first part, and virtual part 1232 can correspond to the second part. For example, as... Figure 9 As shown in (b), for virtual element 162, virtual part 1621 can correspond to the first part, and virtual part 1622 can correspond to the second part.

[0142] In some possible implementations, the height of the end of the first part closest to the second part can be equal to the height of the end of the second part closest to the first part. For example, refer to... Figure 6 In (b), the height between the end of virtual part 1231 near virtual part 1232 and the end of virtual part 1232 near virtual part 1231 is less than or equal to the second threshold.

[0143] In some possible implementations, the first part may include a stripe pattern extending along a first boundary. For example, as... Figure 6 The virtual parts 1221 and 1231 are shown in (c).

[0144] In some possible implementations, in the first form, the projected length of the first virtual element on the first boundary can be equal to the projected length of the first virtual element on the first boundary in the second form.

[0145] For example, in Figure 6 In the scenario shown, for virtual elements 123, as Figure 6 The length of the projection of the bar pattern shown in (a) onto the right boundary of the display area 100 can be equal to, as shown in the figure. Figure 6 The length of the projection of the cue pattern shown in (d) onto the boundary. For example, with... Figure 6 When virtual element 123 is displayed as shown in (b) and (c), the length of the projection of virtual element 123 onto the right boundary of display area 100 can be equal to the length of the corresponding prompt pattern (e.g., as shown in the diagram). Figure 6 (d) is shown as the projection onto the boundary.

[0146] In this embodiment, since the projection length of the first virtual element on the first boundary is the same in both different forms, when the first road element moves from outside the display area to inside the display area, the sudden change in the size of the first virtual element can be avoided from interfering with the user.

[0147] In some possible implementations, the first virtual element can be used to display navigation information and / or prompts. For example, such as Figures 3 to 6 As shown, virtual elements 122 and 123 can be used to display prompts. For example, as... Figure 8 and Figure 9 As shown, virtual element 162 can be used to display navigation information.

[0148] For example, Figure 11 This is a schematic flowchart illustrating a display method provided in an embodiment of this application. Figure 11 As shown, the method 600 may include the following steps:

[0149] S601, triggers AR display.

[0150] For example, based on the surrounding environment, road elements in the environment that need to trigger AR display can be identified. For instance, when the surrounding environment contains obstacles that pose a collision risk to vehicles (such as... Figures 3 to 6 When road elements A and B are displayed, an AR display can be triggered. Similarly, when using an AR-HUD to present navigation information, an AR display can be triggered.

[0151] This application does not limit the method of triggering AR display.

[0152] S602 inputs a three-dimensional (3D) point set into the rendering engine.

[0153] For example, data collected by sensors such as radar and cameras can determine the spatial location of the area on the road element used to attach the AR pattern (e.g., the spatial location of a cyclist's or pedestrian's torso). This area can be understood as a set of three-dimensional position points (referred to as a 3D point set). When displaying the AR pattern, the corresponding AR pattern needs to be anchored to a three-dimensional position point in this 3D point set to achieve the effect of attaching the AR pattern to the road element. Inputting this 3D point set into the rendering engine allows the corresponding AR pattern to be attached to the road element, enabling the AR pattern to move along with the road element.

[0154] S603 displays AR effects.

[0155] For example, such as Figure 3As shown in (d), when the road element is within the display area, the prompt icon can be displayed in the display area in a way that fits the road element A and the road element B.

[0156] S604, Obtain parameters of the head-up display device.

[0157] For example, parameters such as the placement position of the head-up display device, the projection angle, and the spatial position of the projection area on the windshield can be obtained. These parameters can be used to determine the display area of ​​the head-up display device.

[0158] S605 determines whether there are 3D location points outside the field of view.

[0159] Based on the spatial location of the display area, it can be determined whether any three-dimensional position points in the point set used to anchor the AR pattern extend beyond the display area.

[0160] S606 maps the three-dimensional location points of the super-field of view to a two-dimensional (2D) region.

[0161] S607, 2D / 3D fusion display.

[0162] The following combination Figure 6 The scenario shown uses road element B and virtual element 123 as examples to illustrate steps S603, S606, and S607.

[0163] For example, when all the three-dimensional position points in the 3D point set are within the display area, the road element can be considered to be within the display area, and the AR pattern can be attached to the road element, such as... Figure 6 As shown in (d) in the diagram. This display effect can be called a 3D display.

[0164] For example, if all the 3D position points in the 3D point set are outside the display area, the road element can be considered to be outside the display area. These 3D position points can be mapped to the vicinity of the display area's boundary to anchor the bar pattern; correspondingly, the display interface can be as follows: Figure 6 As shown in (a) above. This display effect can be called a 2D display.

[0165] For example, in the case where some 3D position points in the 3D point set are within the display area while others are outside the display area, the 3D position points that do not exceed the display area can be used to anchor the portion of the AR pattern that does not exceed the display area, and can correspond to the virtual portion 1232 in the virtual element 123, such as... Figure 6As shown in (b) and (c) in the diagram. For 3D position points that extend beyond the display area, they can be mapped to the right boundary of the display area to anchor the corresponding bar pattern (e.g., virtual portion 1231 in virtual element 123), as... Figure 6 As shown in (b) and (c) in the image. This display effect can be described as a fusion display of 2D and 3D.

[0166] For example, Figure 12 This is a schematic flowchart illustrating a display method provided in an embodiment of this application. Figure 12 As shown, the method 700 may include the following steps:

[0167] S701 acquires signals from the advanced driving assistance system (ADAS).

[0168] ADAS signals can be used to indicate the vehicle's control information by advanced driver assistance functions. For example, the autonomous driving domain controller can output this ADAS signal; the HUD can obtain this ADAS signal through the vehicle's internal communication network.

[0169] S702, obtain vehicle control information.

[0170] Vehicle control information can be used to instruct the driver on how to operate the vehicle.

[0171] S703, calculates AR space.

[0172] AR space refers to the space used to present augmented reality effects. In an AR-HUD scenario, this AR space can correspond to the display area.

[0173] For example, the vehicle's operating status can be determined based on ADAS signals and / or vehicle control information; and the real-time position of the display area can be determined by combining the parameters of the head-up display device.

[0174] S704, Obtain 3D point set.

[0175] For a description of 3D point sets, please refer to the relevant records in Method 600.

[0176] S705, out-of-frame detection.

[0177] Step S705 can determine whether there are any three-dimensional position points in the 3D point set that are outside the display area.

[0178] When all three-dimensional position points in the 3D point set are within the display area, steps S706 and S707 do not need to be executed.

[0179] When all three-dimensional position points in the 3D point set are outside the display area, steps S708 and S709 do not need to be executed.

[0180] S706, determine the part that goes outside the frame.

[0181] Step S706 allows for the filtering of 3D point locations that extend beyond the display area from the 3D point set.

[0182] S707, 2D projection.

[0183] 2D projection can be understood as rendering a pattern that needs to be presented in a 2D display mode.

[0184] For example, for three-dimensional position points that extend beyond the display area, they can be mapped to corresponding positions within the display area, such as near the boundary of the display area. These mapped position points can then be used to fit bar patterns, text patterns, etc.

[0185] For example, in Figure 6 In the scenarios shown in (b) and (c), for virtual element 123, the position of virtual part 1231 in the display area can be determined by 2D projection.

[0186] S708, confirm the part that is not outside the frame.

[0187] Step S708 allows for the selection of three-dimensional position points within the 3D point set that do not exceed the display area.

[0188] S709, 3D rendering.

[0189] 3D rendering can be understood as rendering patterns that need to be presented in 3D.

[0190] For example, in Figure 6 In the scenarios shown in (b) and (c), for virtual element 123, the position of virtual part 1232 in the display area can be determined by 3D rendering.

[0191] S710, image fusion.

[0192] For example, in Figure 6 In the scenarios shown in (b) and (c), virtual element 123 containing virtual parts 1231 and 1232 can be displayed.

[0193] The above text combines Figures 3 to 12 The methods provided in the embodiments of this application are described in detail below. Figure 13 and Figure 14The apparatus provided in the embodiments of this application is described in detail. The description of the apparatus embodiments corresponds to the description of the method embodiments; therefore, for any content not described in detail, please refer to the method embodiments above.

[0194] For example, Figure 13 A schematic block diagram of an apparatus 800 provided in an embodiment of this application is shown. The apparatus 800 may include modules or units for implementing the method embodiments described above.

[0195] For example, the device 800 may include an acquisition unit 810 and a display unit 820.

[0196] For example, the acquisition unit 810 can be used to acquire the spatial position of the first road element and the display area of ​​the HUD. The display unit 820 can be used to display a first virtual element corresponding to the first road element in the display area when the first road element is outside the first boundary of the display area or a part of the first road element is inside the first boundary.

[0197] In some possible implementations, the display unit 820 can be used to display the first part in the display area when the first road element is outside the first boundary of the display area.

[0198] In some possible implementations, the first form may also include a second part, which may be a portion of the first pattern. The display unit 820 may be used to: display the first part and the second part in the display area when a portion of the first road element is within the first boundary and another portion is outside the first boundary; the first part and the second part are adjacent to each other.

[0199] In some possible implementations, before displaying the first virtual element corresponding to the first road element in the display area, the display unit 820 may be used to: determine that the distance between the first road element and the vehicle is less than or equal to a first threshold.

[0200] For a description of the display area, the first virtual element, the first road element, the first part, and the second part, please refer to the relevant records in the above method embodiments.

[0201] Figure 14 This is a schematic block diagram of an electronic device provided in an embodiment of this application. Figure 14 As shown, the electronic device 900 may include one or more processors 910; one or more memories 920; the one or more memories 920 storing one or more instructions that, when executed by one or more processors 910, cause the display method as described in any of the possible implementations above to be executed.

[0202] This application also provides an apparatus including a processor and a communication interface. The communication interface is used to receive signals and transmit the signals to the processor. The processor processes the signals so that the display method described in any of the possible implementations above is executed.

[0203] The device can be a chip. For example, the chip can be a chip system or a standalone chip.

[0204] This application also provides a readable storage medium storing instructions that, when executed on a device, cause the device to perform the aforementioned method steps to implement the display method described above.

[0205] This application also provides a program product that, when run on a device, causes the device to perform the aforementioned steps to implement the display method described in the above embodiments.

[0206] This application also provides an apparatus including a module for implementing the display method as described in any of the preceding embodiments.

[0207] In addition, embodiments of this application also provide an apparatus, which may specifically be a chip, component or module. The apparatus may include a connected processor and a memory; wherein the memory is used to store instructions, and when the apparatus is running, the processor may execute the instructions stored in the memory to cause the apparatus to perform the display methods in the above-described method embodiments.

[0208] In this embodiment, the device, readable storage medium, program product or apparatus are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can be referred to the beneficial effects of the corresponding methods provided above, and will not be repeated here.

[0209] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0210] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0211] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0212] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0213] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0214] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0215] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A display method, characterized in that, include: The spatial position of the first road element and the display area of ​​the head-up display (HUD) are obtained, wherein the display area of ​​the HUD is used to display a virtual image; When the first road element is outside the first boundary of the display area, or when a portion of the first road element is within the first boundary, a first virtual element corresponding to the first road element is displayed in the display area.

2. The display method according to claim 1, characterized in that, The first virtual element includes a first form and a second form. The first form of the first virtual element is used to: display in the display area when the first road element is outside the first boundary of the display area, or when a part of the first road element is inside the first boundary; The second form of the first virtual element is used to: display the first road element in the display area in a manner that fits the first road element when the first road element is within the first boundary of the display area.

3. The display method according to claim 2, characterized in that, The first form includes a first portion that fits into the first boundary, and the second form includes a first pattern that fits into the first road element.

4. The display method according to claim 3, characterized in that, The step of displaying a first virtual element corresponding to the first road element in the display area when the first road element is outside the first boundary of the display area, or when a portion of the first road element is within the first boundary, includes: The first portion is displayed in the display area when the first road element is outside the first boundary of the display area.

5. The display method according to claim 3, characterized in that, The first form also includes a second part, which is a portion of the first pattern. The step of displaying a first virtual element corresponding to the first road element in the display area when the first road element is outside the first boundary of the display area, or when a portion of the first road element is within the first boundary, includes: When a portion of the first road element is within the first boundary and another portion is outside the first boundary, the first portion and the second portion are displayed in the display area, with the first portion and the second portion being adjacent to each other.

6. The display method according to claim 5, characterized in that, The height of the first part near the end of the second part is equal to the height of the second part near the end of the first part.

7. The display method according to any one of claims 3 to 6, characterized in that, The first portion includes a strip pattern extending along the first boundary.

8. The display method according to any one of claims 2 to 7, characterized in that, In the first state, the projection length of the first virtual element on the first border is equal to the projection length of the first virtual element on the first border in the second state.

9. The display method according to any one of claims 1 to 8, characterized in that, The first virtual element is used to display navigation information and / or prompts.

10. The display method according to any one of claims 1 to 9, characterized in that, Before displaying the first virtual element corresponding to the first road element in the display area, the method further includes: The distance between the first road element and the vehicle is determined to be less than or equal to a first threshold.

11. The display method according to any one of claims 1 to 10, characterized in that, The first road element includes a first fitting area, which is used to: fit the first virtual element when the first road element is within the display area; The first road element is located outside the first boundary of the display area, including: the first mating area is located outside the first boundary of the display area; A portion of the first road element is located within the first boundary of the display area, including: a portion of the first mating area is located within the first boundary of the display area.

12. A display device, characterized in that, include: The acquisition unit is used to: acquire the spatial position of the first road element and the display area of ​​the head-up display device (HUD), wherein the display area of ​​the HUD is used to display a virtual image; The display unit is configured to: display a first virtual element corresponding to the first road element in the display area when the first road element is outside the first boundary of the display area, or when a portion of the first road element is inside the first boundary.

13. The display device according to claim 12, characterized in that, The first virtual element includes a first form and a second form. The first form of the first virtual element is used to: display in the display area when the first road element is outside the first boundary of the display area, or when a part of the first road element is inside the first boundary; The second form of the first virtual element is used to: display the first road element in the display area in a manner that fits the first road element when the first road element is within the first boundary of the display area.

14. The display device according to claim 13, characterized in that, The first form includes a first portion that fits into the first boundary, and the second form includes a first pattern that fits into the first road element.

15. The display device according to claim 14, characterized in that, The display unit is used for: The first portion is displayed in the display area when the first road element is outside the first boundary of the display area.

16. The display device according to claim 14, characterized in that, The first form also includes a second part, which is a portion of the first pattern. The display unit is used for: When a portion of the first road element is within the first boundary and another portion is outside the first boundary, the first portion and the second portion are displayed in the display area, with the first portion and the second portion being adjacent to each other.

17. The display device according to claim 16, characterized in that, The height of the first part near the end of the second part is equal to the height of the second part near the end of the first part.

18. The display device according to any one of claims 14 to 17, characterized in that, The first portion includes a strip pattern extending along the first boundary.

19. The display device according to any one of claims 13 to 18, characterized in that, In the first state, the projection length of the first virtual element on the first border is equal to the projection length of the first virtual element on the first border in the second state.

20. The display device according to any one of claims 12 to 19, characterized in that, The first virtual element is used to display navigation information and / or prompts.

21. The display device according to any one of claims 12 to 20, characterized in that, Before displaying the first virtual element corresponding to the first road element in the display area, the display unit is further configured to: The distance between the first road element and the vehicle is determined to be less than or equal to a first threshold.

22. The display device according to any one of claims 12 to 21, characterized in that, The first road element includes a first fitting area, which is used to: fit the first virtual element when the first road element is within the display area; The first road element is located outside the first boundary of the display area, including: the first mating area is located outside the first boundary of the display area; A portion of the first road element is located within the first boundary of the display area, including: a portion of the first mating area is located within the first boundary of the display area.

23. An electronic device, characterized in that, include: One or more processors; One or more memories; the one or more memories storing one or more programs that, when executed by one or more processors, cause the display method as described in any one of claims 1-11 to be performed.

24. A chip, characterized in that, The chip includes a processor and a communication interface, the communication interface being used to receive signals and transmit the signals to the processor, the processor processing the signals such that the display method as described in any one of claims 1-11 is executed.

25. A readable storage medium, characterized in that, The readable storage medium stores instructions that, when executed on an electronic device, cause the display method as described in any one of claims 1-11 to be performed.

26. A program product, characterized in that, The program product includes program code that, when run on an electronic device, causes the display method as described in any one of claims 1-11 to be executed.

Images