Display method and apparatus

Abstract

Provided in the embodiments of the present application are a display method and apparatus. The method may comprise: acquiring a spatial position of a first road element and a display area of a head-up display (HUD), wherein the display area of the HUD is used for displaying a virtual image; and when the first road element is located outside a first boundary of the display area or a part of the first road element is located inside the first boundary, displaying, in the display area, a first virtual element corresponding to the first road element. In the present application, even if a certain road element is partially or completely located outside a first boundary, a virtual element corresponding to the road element can still be displayed in a display area. In this way, an augmented reality effect can be prevented from being limited to the display area of an HUD, the waste of performance of a sensing system can be avoided, and the expression capability of the augmented reality effect can be improved.

Description

Display method and device

[0001] The present application claims priority to the Chinese patent application No. 202411135038.X, filed on August 16, 2024, and entitled "Display method and device", the content of which is incorporated herein by reference in its entirety. TECHNICAL FIELD

[0002] The present application relates to the field of intelligent vehicles, and more particularly, to a display method and device. BACKGROUND

[0003] In order to improve driving safety, a head up display (HUD) device can be used to present information required for driving at a position visible to the driver. Among them, an augmented reality head up display (AR-HUD) can display information such as navigation information in combination with the actual environment in the driver's field of view, so that the driver can more conveniently, conveniently and intuitively obtain the information required for driving, which is beneficial to improve the safety and comfort of driving.

[0004] However, due to the limited display area of the AR-HUD, the information fitted with the real scene may exceed the display range of the AR-HUD during vehicle driving, affecting the user's experience. SUMMARY

[0005] The present application provides a display method and device, which can avoid the limitation of augmented reality effect on the display area of the head up display device, and can improve the expression ability of augmented reality effect.

[0006] In a first aspect, a display method is provided. The method comprises: obtaining a spatial position of a first road element and a display area of a HUD, the display area of the HUD being used to display a virtual image; and when the first road element is outside a first boundary of the display area or part of the first road element is inside the first boundary, displaying a first virtual element corresponding to the first road element in the display area.

[0007] In the present application, even if part or all of a road element is outside the first boundary, a virtual element corresponding to the road element can still be displayed in the display area. In this way, the limitation of augmented reality effect on the display area of the head up display device can be avoided, and the waste of the performance of the perception system can be reduced, and the expression ability of augmented reality effect can be improved.

[0008] In some possible implementation manners, the first virtual element can include a first form and a second form. The first form of the first virtual element can be used to be displayed in the display region when the first road element is outside the first boundary of the display region or a part of the first road element is inside the first boundary. The second form of the first virtual element can be used to be displayed in the display region in a manner of fitting the first road element when the first road element is inside the first boundary of the display region.

[0009] In the present application, when a part or all of a road element is outside the display region, a corresponding virtual element is displayed in a first form; and when the road element is inside the display region, a virtual element in a second form can be displayed in a manner of fitting the road element. In this way, on the one hand, even if the road element exceeds the display range, the corresponding virtual element can still be displayed to avoid the loss of relevant information; on the other hand, when the road element is inside the display region, relevant information can be provided to the user in a more intuitive manner. Thus, the use requirements of different use scenarios can be taken into account.

[0010] In some possible implementation manners, the first form can include a first part fitting the first boundary; and the second form can include a first pattern fitting the first road element.

[0011] In some possible implementation manners, when the first road element is outside the first boundary of the display region or a part of the first road element is inside the first boundary, displaying, in the display region, a first virtual element corresponding to the first road element can include: when the first road element is outside the first boundary of the display region, displaying, in the display region, the first part.

[0012] In the present application, when a road element is outside the display region, the corresponding virtual element can include a first part fitting the first boundary. Presenting the virtual element corresponding to the road element in a manner of fitting the first boundary can avoid the interference of inappropriate display of the virtual element on the user.

[0013] In some possible implementation manners, the first form can further include a second part, and the second part can be a part of the first pattern. When the first road element is outside the first boundary of the display region or a part of the first road element is inside the first boundary, displaying, in the display region, a first virtual element corresponding to the first road element can include: when a part of the first road element is inside the first boundary and another part is outside the first boundary, displaying, in the display region, the first part and the second part; and the first part and the second part are adjacent.

[0014] In the present application, when the first road element is outside the display region, the first virtual element can include a first part that fits the first boundary; and when the first road element is inside the display region, the first virtual element can include a first pattern that fits the first road element. When part of the first road element is outside the display region and another part is inside the display region, by displaying the first part and the second part (i.e. part of the first pattern) of the virtual element in the display region, a smooth transition between different forms of the virtual element is facilitated.

[0015] In some possible implementation manners, the height of the first part near one end of the second part can be equal to the height of the second part near one end of the first part.

[0016] In some possible implementation manners, the first part can include a strip pattern extending along the first boundary.

[0017] In the present application, by setting the first part as a strip pattern extending along the first boundary, in the process of relatively moving a certain road element from outside the display region to inside the display region, the corresponding virtual element will be able to present, in the user's perception, the effect that the first pattern gradually unfolds from a rolled-up state of a strip to a completely unfolded state, and a smooth switching between different forms of the virtual element can be achieved.

[0018] In some possible implementation manners, in the first form, the projection length of the first virtual element on the first boundary can be equal to the projection length of the first virtual element on the first boundary in the second form.

[0019] In the present application, since the projection lengths of the first virtual element on the first boundary in the two different forms are the same, when the first road element is relatively moved from outside the display region to inside the display region, the size of the first virtual element can be prevented from changing suddenly and causing interference to the user.

[0020] In some possible implementation manners, the first virtual element can be used to display navigation information and / or prompt information.

[0021] In some possible implementation manners, before displaying the first virtual element corresponding to the first road element in the display region, the method can further include: determining that the distance between the first road element and the vehicle is less than or equal to a first threshold.

[0022] In some possible implementations, the first road element can include a first fitting region; the first fitting region is a region on the first road element for fitting the virtual element when the first road element is within the display region. The first road element being outside the display region can include that the first fitting region is outside the display region; the first road element being within the display region can include that the first fitting region is within the display region; the part of the first road element being within the display region can include that the part of the first fitting region is within the display region.

[0023] In a second aspect, an apparatus is provided. The apparatus can include an obtaining unit and a display unit. The obtaining unit can be configured to obtain a spatial position of a first road element and a display region of a HUD, the display region of the HUD being configured to display a virtual image; and the display unit can be configured to display a first virtual element corresponding to the first road element in the display region when the first road element is outside a first boundary of the display region or a part of the first road element is within the first boundary.

[0024] In some possible implementations, the first virtual element can include a first form and a second form. The first form of the first virtual element can be configured to be displayed in the display region when the first road element is outside the first boundary of the display region or the part of the first road element is within the first boundary. The second form of the first virtual element can be configured to be displayed in the display region in a manner of fitting the first road element when the first road element is within the first boundary of the display region.

[0025] In some possible implementations, the first form can include a first part fitting the first boundary; and the second form can include a first pattern fitting the first road element.

[0026] In some possible implementations, the display unit can be configured to display the first part in the display region when the first road element is outside the first boundary of the display region.

[0027] In some possible implementations, the first form can further include a second part, and the second part can be a part of the first pattern. The display unit can be configured to display the first part and the second part in the display region when the part of the first road element is within the first boundary and another part of the first road element is outside the first boundary, and the first part and the second part are adjacent.

[0028] In some possible implementations, a height of the first part near one end of the second part can be equal to a height of the second part near one end of the first part.

[0029] In some possible implementations, the first part can include a strip pattern extending along the first boundary.

[0030] In some possible implementation manners, in the first mode, a projection length of the first virtual element on the first boundary can be equal to a projection length of the first virtual element on the first boundary in the second mode.

[0031] In some possible implementation manners, the first virtual element can be used to display navigation information and / or prompt information.

[0032] In some possible implementation manners, before displaying the first virtual element corresponding to the first road element in the display area, the display unit can further be configured to: determine that a distance between the first road element and the vehicle is less than or equal to a first threshold.

[0033] In some possible implementation manners, the first road element can include a first fitting area, the first fitting area being an area on the road element used to fit the virtual element when the road element is in the display area. The first road element being outside the display area can include that the first fitting area is outside the display area; the first road element being in the display area can include that the first fitting area is in the display area; and the part of the first road element being in the display area can include that the part of the first fitting area is in the display area.

[0034] In a third aspect, an electronic device is provided, including: one or more processors; one or more memories; the one or more memories storing one or more programs, when the one or more programs are executed by the one or more processors, causing the display method as described in the first aspect and any possible implementation manner thereof to be performed.

[0035] In a fourth aspect, a chip is provided, including a processor and a communication interface, the communication interface being configured to receive a signal and transmit the signal to the processor, and the processor being configured to process the signal, causing the display method as described in the first aspect and any possible implementation manner thereof to be performed.

[0036] In a fifth aspect, a readable storage medium is provided, the readable storage medium storing instructions, when the instructions are executed on a device, causing the display method as described in the first aspect and any possible implementation manner thereof to be performed.

[0037] In a sixth aspect, a program product is provided, the program product including program code, when the program code is executed on a device, causing the display method as described in the first aspect and any possible implementation manner thereof to be performed. BRIEF DESCRIPTION OF DRAWINGS

[0038] FIG. 1 is a schematic diagram of a display method of a HUD according to an embodiment of the present application;

[0039] FIG. 2 is a schematic diagram of a display effect of an AR-HUD according to an embodiment of the present application;

[0040] FIG. 3 is a schematic diagram of a set of display interfaces according to an embodiment of the present application;

[0041] FIG. 4 is a schematic diagram of another set of display interfaces according to an embodiment of the present application;

[0042] FIG. 5 is a schematic diagram of two display interfaces according to an embodiment of the present application;

[0043] FIG. 6 is a schematic diagram of another set of display interfaces according to an embodiment of the present application;

[0044] FIG. 7 is a schematic diagram of another set of display interfaces according to an embodiment of the present application;

[0045] FIG. 8 is a schematic diagram of two display interfaces according to an embodiment of the present application;

[0046] FIG. 9 is a schematic diagram of another set of display interfaces according to an embodiment of the present application;

[0047] FIG. 10 is a schematic diagram of a display method according to an embodiment of the present application;

[0048] FIG. 11 is a schematic diagram of another display method according to an embodiment of the present application;

[0049] FIG. 12 is a schematic diagram of another display method according to an embodiment of the present application;

[0050] FIG. 13 is a schematic diagram of a structure of an apparatus according to an embodiment of the present application;

[0051] FIG. 14 is a schematic diagram of a structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0052] The technical solutions in the present application will be described below with reference to the accompanying drawings.

[0053] AR-HUD is a technology that combines augmented reality (AR) with head-up display. Augmented reality refers to the technology of superimposing and fusing virtual information generated by a computer with an actual physical environment through photoelectric display, interaction and sensing, etc., to enhance the user's perception of the real environment and achieve the effect of enhancing the real world. Head-up display refers to displaying information required for driving in a region that can be seen by the driver when he / she looks up, so that the driver can obtain information such as vehicle speed and navigation without looking down at the instrument panel, central control screen or other devices.

[0054] Exemplarily, FIG. 1 is a schematic diagram of a display mode of a HUD according to an embodiment of the present application.

[0055] As shown in FIG. 1, in the scenario of augmented reality, the area for presenting the virtual image of AR-HUD can be referred to as a display area. In actual applications, the virtual image can be presented by projection components in the HUD, and the virtual image observed by the human eye can be an image formed by multiple reflections or refractions.

[0056] In an application example, the HUD can project virtual information to the front windshield, as shown in FIG. 1. The image observed by the human eye can be a virtual image formed by reflection and / or refraction of the front windshield. The user can see the virtual image through the front windshield and also see the real environment in front of the vehicle; that is, the user can see the virtual image superimposed on the real scene.

[0057] In the augmented display scenario, the driving required information such as navigation information and warning information can be displayed in the form of a virtual image in front of the vehicle, and can be fitted with the corresponding road elements in the surrounding environment (such as a lane, an obstacle with potential collision risk, a lane / driving path corresponding to the navigation information, etc.) in the display area. The following is an exemplary description in conjunction with FIGS. 2 and 3.

[0058] Exemplarily, FIG. 2 is a schematic diagram of a display effect of an AR-HUD provided by an embodiment of the present application.

[0059] As shown in (a) of FIG. 2, the vehicle 10 can be provided with a HUD. The HUD can project a virtual image, which can be presented in the display area 100; accordingly, the virtual image superimposed on the real environment can be observed in the driver's view. For example, one or more of the vehicle speed information, the road speed limit information, the distance information between the vehicle and the next intersection, etc. can be presented in the display area 100 in the form of a virtual element (such as a virtual element 111), as shown in (b), (c) of FIG. 2. For another example, the virtual element 111 can be updated in real time according to the corresponding information.

[0060] In one embodiment, it is assumed that there are road elements A and B with potential collision risk in front of the vehicle 10. As shown in (a) of FIG. 2, the road element A can be a cyclist, and the road element B can be a pedestrian.

[0061] When the road elements A and B are within the display area 100, the display area 100 can display prompt patterns 112 and 113; the prompt patterns 112 and 113 can play a prompting / warning role for the user, as shown in (b) of FIG. 2. That is, the HUD can warn the user of the possible risks by displaying the prompt patterns 112 and 113 in the augmented reality manner; the prompt patterns 112 and 113 can be understood as virtual elements corresponding to the road elements A and B.

[0062] Further, the prompt pattern 112 can be fitted with the road element A, and the prompt pattern 113 can be fitted with the road element B. For example, the prompt patterns 112 and 113 can be fitted with the torso parts of the cyclists and the pedestrians respectively; when the cyclists and the pedestrians move in the display region 100, the prompt patterns 112 and 113 can move with the road elements A and B respectively.

[0063] When part or all of the torso parts of the cyclists / pedestrians are out of the display region 100, the prompt patterns 112 and 113 cannot be fitted with the corresponding cyclists / pedestrians; in order to avoid the inappropriate fitting of the prompt patterns and the resulting trouble to the users, the prompt patterns 112 and 113 can not be displayed in the display region 100; that is, the virtual elements corresponding to the road elements A and B can not be displayed. For example, as shown in (c) of FIG. 2, when the road element A is out of the left border of the display region 100, the prompt pattern 112 cannot be fitted with the road element A, and thus the prompt pattern 112 can not be displayed in the display region 100. For another example, assuming that the region on the road element B for fitting the prompt pattern 113 is the torso, when part of the region is in the right border of the display region 100 and the other part of the region is out of the display region, the prompt pattern 113 cannot be fitted with the region; in this case, the prompt pattern 113 can not be displayed in the display region 100, as shown in (c) of FIG. 2.

[0064] However, in the case where the vehicle 10 can sense, through the sensors such as the camera, the road elements (such as the road elements A and B) that have the risk of collision, if the user cannot be prompted of the risk due to the limitation of the range of the display region, on the one hand, the safety of the vehicle and the user is not conducive; on the other hand, the performance of the vehicle sensing function is greatly wasted.

[0065] In view of this, the embodiments of the present application provide a display method and device. In the technical solution, even if the road element is out of the display region, the corresponding virtual element can be displayed in the display region, the augmented reality effect can be avoided from being limited by the display region of the head-up display device, and the expression capability of the augmented reality effect can be improved.

[0066] The display method and device of the embodiments of the present application will be introduced below in combination with several sets of display interfaces.

[0067] Exemplarily, FIG. 3 is a schematic diagram of a set of display interfaces provided by the embodiments of the present application. The display interfaces shown in FIG. 3 can be understood as display interfaces corresponding to virtual images projected by the HUD from the perspective of the driver; the display interfaces shown in FIGS. 4-9 below also adopt the same perspective, which is uniformly described here and will not be repeated below. Among them, (a)-(d) in FIG. 3 can correspond to the scene in which the road elements A and B move from outside the display area 100 to inside the display area 100.

[0068] Referring to (a) in FIG. 3, the display interface can be a display interface when the road elements A and B are both outside the display area 100.

[0069] At this time, in the display area 100, a virtual element 122 can be displayed; the virtual element 122 can correspond to the road element A to prompt the user that there is a pedestrian (such as a cyclist, a pedestrian, an old person, a child, etc.) on the left side. The display area 100 can display a virtual element 123; the virtual element 123 can correspond to the road element B to prompt the user that there is a pedestrian on the right side. For example, the virtual element 122 can include text information (which can also be referred to as a text pattern) such as "pay attention to pedestrians", "pay attention to pedestrians on the left", etc.; similarly, the virtual element 123 can include text patterns such as "pay attention to pedestrians", "pay attention to pedestrians on the right", etc. to remind the user that there is a pedestrian on the right side. For example, in order to prevent misguidance caused by inappropriate display position of the virtual element, the virtual element 122 can be displayed in a left side area of the display area 100, such as the upper left, lower left, left side boundary, etc.; similarly, the virtual element 123 can be displayed in a right side area of the display area 100, such as the upper right, lower right, right side boundary, etc.

[0070] Exemplarily, the relative positions between the road elements A and B and the display area 100 can change due to the movement of the road elements A and B and / or the movement of the vehicle 10. For example, when the road element A is outside the left side boundary of the display area 100 and part of the road element B is inside the right side boundary of the display area 100, the display area 100 can display a virtual image as shown in (b) in FIG. 3. For example, the road elements A and B continue to move relative to the display area 100, when part of the road element A is inside the left side boundary of the display area 100 and part of the road element B is inside the right side boundary of the display area, the display area 100 can display an interface as shown in (c) in FIG. 3. For example, the road elements A and B continue to move relative to the display area 100, when the road elements A and B are inside the display area 100, the display area 100 can display an interface as shown in (d) in FIG. 3.

[0071] It can be understood that the embodiments of the present application do not limit the specific way in which the display area 100 enters the display interface as shown in (b) and (c) of FIG. 3. For example, when the HUD is enabled, the road elements A and B can already be in the positions as shown in (c) of FIG. 3; in this case, the display area 100 can not display the interface as shown in (a) and (b) of FIG. 3.

[0072] For a certain road element, whether the road element is outside the display area can be determined based on the positional relationship between the area on the road element for fitting the virtual element and the display area. That is, part of the road element is outside the display area, which can be understood as part of the area on the road element for fitting the virtual element (such as the torso of the road element B) is outside the display area; the road element is inside the display area, which can be understood as the entire area is inside the display area; the road element is outside the display area, which can be understood as the entire area is outside the display area. Here, a unified description is made, and the following will not be repeated.

[0073] Referring to (b) of FIG. 3, the display interface can be the display interface when part of the road element B is inside the display area 100 and the road element A is outside the display area 100. For example, for the road elements A and B, the positions thereof in the scenario shown in (b) of FIG. 3 can be the same as the positions thereof in the scenario shown in (c) of FIG. 2.

[0074] Referring to (c) of FIG. 3, the display interface can be the display interface when part of the road element A is inside the left boundary of the display area 100 and part of the road element B is inside the right boundary of the display area 100.

[0075] Referring to (d) of FIG. 3, the display interface can be the display interface when the road elements A and B are inside the display area 100. When the road elements A and B are inside the display area 100, the virtual elements 122 and 123 can be fitted with the corresponding road elements; in this case, the user can not need to be prompted in the way of the text pattern, but can be prompted in the way of the virtual image (such as the prompt patterns 112 and 113 as adopted in (b) of FIG. 2) that can be fitted with the road elements.

[0076] In one embodiment, the road elements A, B can move relative to the vehicle. When the road elements A, B are still outside the display region 100 or only partially inside the display region 100, the virtual elements 122, 123 can be displayed in the form of a text pattern due to the difficulty in matching the virtual elements 122, 123 with the corresponding road elements A, B. When the road elements A, B move into the display region 100, the virtual elements 122, 123 can be displayed in the form of a prompt pattern to match the corresponding road elements, so as to more intuitively enhance the user's perception of the real environment.

[0077] In the embodiments of the present application, the display method of the present application is exemplarily described by taking the example of displaying the virtual elements 122, 123 in the form of a text pattern when part or all of the road elements A, B are outside the display region 110. In other embodiments, the virtual elements 122, 123 can be presented in other ways (such as an obstacle model, an arrow pattern, a bar pattern, etc.) in the display region 110 under the same circumstances. The other presentation methods of the virtual elements are exemplarily described below in combination with FIGS. 4-6.

[0078] Exemplarily, FIG. 4 is a schematic diagram of another set of display interfaces provided by the embodiments of the present application. In FIG. 4, (a)-(d) can correspond to the scenario that the road elements A, B move from outside the display region 100 into the display region 100.

[0079] When the road elements A, B are outside the display region 100 or part of them are inside the display region 100, unlike (a)-(c) in FIG. 3, the virtual elements 122, 123 can be presented in the form of an obstacle model to prompt the user that there are obstacles with collision risks in the surrounding environment. For example, the road elements A and B correspond to the obstacle type of pedestrians, and the obstacle model corresponding to this type can be displayed in the display region 100, as shown in (a)-(c) in FIG. 4.

[0080] When the road elements A, B are inside the display region 100, similar to (d) in FIG. 3, the virtual elements 122, 123 can be displayed in the form of a prompt pattern. Compared with the obstacle model, the prompt pattern can more intuitively enhance the user's perception of the real environment.

[0081] In the embodiments of the present application, the display method of the present application is exemplarily described by taking the example of presenting the virtual elements in the form of an obstacle model when the obstacle type is pedestrians. It can be understood that when the road elements correspond to other obstacle types (such as plants, vehicles, etc.), the obstacle model corresponding to the type can be displayed in the display region 100.

[0082] In other embodiments, the way of dividing the obstacle types can be different from that shown in FIG. 4. For example, although both road element A and road element B are pedestrians, they can correspond to different obstacle types; that is, the pedestrians can be further divided into multiple types such as walkers and cyclists. The following is an exemplary description in conjunction with FIG. 5.

[0083] Exemplarily, FIG. 5 is a schematic diagram of two display interfaces provided by an embodiment of the present application. In FIG. 5, (a) and (b) show the case where road element A and road element B are outside the display area 100. For example, for road element A and road element B, the positions they are in under the scenario shown in (a) and (b) of FIG. 5 can be the same as the positions they are in under the scenario shown in (a) of FIG. 3.

[0084] In one embodiment, unlike the way shown in FIG. 4, road element A and road element B can correspond to different obstacle types, for example, the obstacle types they correspond to can be cyclists and walkers respectively; accordingly, the obstacle models corresponding to the two types can be called cyclist model and walker model respectively. For example, as shown in (a) of FIG. 5, when road element A and road element B are both outside the display area 100, the display area 100 can display virtual elements 122 and 123; among them, virtual element 122 can be presented by using the cyclist model, and virtual element 123 can be presented by using the walker model, corresponding to road element A. For another example, similar to (b) and (c) of FIG. 3, when part of the road element corresponding to virtual elements 122 and 123 is inside the display area 100, virtual elements 122 and 123 can be displayed in the display area 100 in the way of cyclist model and walker model respectively.

[0085] In yet another embodiment, an arrow pattern can be used to represent the moving direction of the road element. For example, assuming that road element A moves to the right, when road element A is outside the display area 100, virtual element 122 can include the model of a cyclist and also include a pattern of a right arrow; the right arrow pattern can indicate that the road element A moves to the right, as shown in (b) of FIG. 5. For another example, assuming that road element A moves to the left, when road element B is outside the display area 100, virtual element 123 can include the model of a walker and also include a pattern of a left arrow; the left arrow can indicate that the road element B moves to the left.

[0086] In (a) of FIG. 5 described above, virtual elements 122 and 123 are presented only by using obstacle models; in (b) of FIG. 5, virtual elements 122 and 123 can be presented by using the combination of arrow patterns and obstacle models. In some possible implementation manners, unlike the way shown in (a) and (b) of FIG. 5, virtual elements 122 and 123 can be presented only by using arrow patterns.

[0087] In the above-mentioned solutions of FIG. 3 to FIG. 5, when a certain road element is outside the display region 100 or part of the road element is inside the display region 100, a virtual element corresponding to the road element can be displayed in the display region 100 in one form (such as a text pattern); and when the road element is inside the display region 100, the virtual element corresponding to the road element can be presented in another form.

[0088] In this implementation, when the road element is moved from outside the display region 100 to inside the display region, the corresponding virtual element will be switched / transited from one form to another form. If the difference between the two forms is large, the switching / transiting process may not be smooth enough and may interfere with the user. Further, when a certain road element is outside the display region 100 or part of the road element is inside the display region 100, the corresponding virtual element can be displayed in a bar pattern at the corresponding boundary position of the display region 100 to achieve a smooth transition between different forms. The following is exemplarily described in conjunction with FIG. 6.

[0089] Exemplarily, FIG. 6 is a schematic diagram of another set of display interfaces provided by the embodiments of the present application. In FIG. 6, (a) to (d) show scenarios in which road elements A and B are relatively moved from outside the display region 100 to inside the display region 100.

[0090] It is assumed that when the road elements A and B are inside the display region 100, the corresponding virtual elements are presented in the form of a prompt pattern, and the prompt pattern is fitted with the corresponding road element, as shown in (d) of FIG. 3.

[0091] Referring to (a) of FIG. 6, the display interface can be a display interface when the road elements A and B are both outside the display region 100. For example, since the road element A is outside the left boundary of the display region 100, in the interface, the virtual element 122 can include a bar pattern near or adjacent to the left boundary. Similarly, since the road element B is outside the right boundary of the display region 100, in the interface, the virtual element 123 can include a bar pattern near or adjacent to the right boundary.

[0092] Referring to (b) of FIG. 6, the display interface can be a display interface when the road element A is outside the display region 100 and part of the road element B is inside the display region 100.

[0093] Referring to (c) of FIG. 6, the display interface can be a display interface when part of the road element A is inside the left boundary of the display region 100 and part of the road element B is inside the right boundary of the display region.

[0094] When a part of the road element A, B is within the display area 100 and another part is outside the display area 100, the virtual element 122, 123 can include a bar pattern that fits the corresponding boundary of the display area 100, and can also include a part of the corresponding prompt pattern.

[0095] In one embodiment, as shown in (b) and (c) of FIG. 6, when a part of the road element B is within the right boundary of the display area 100 and another part is outside the boundary, the virtual element 123 can include a virtual part 1231 and a virtual part 1232; the virtual part 1231 can be a bar pattern, and the virtual part 1232 can be a part of the prompt pattern 113. For example, the virtual part 1231 and 1232 can be adjacent. For another example, the height of the virtual element 123 can be equal to the height of the prompt pattern 113, or the height difference between the prompt pattern 113 can be less than or equal to a certain threshold. For another example, the virtual part 1232 can be understood as a part of the prompt pattern 113 that is within the right boundary of the display area 100, assuming that the prompt pattern 113 can fit the road element B without being limited by the display area. For another example, the virtual part 1231 can be understood as a part of the bar pattern 123 shown in (a) of FIG. 6 that is not covered by the virtual part 1232, assuming that the prompt pattern 113 can fit the road element B without being limited by the display area.

[0096] In another embodiment, as shown in (c) of FIG. 6, when a part of the road element A is within the right boundary of the display area 100 and another part is outside the boundary, the virtual element 122 can include a virtual part 1221 and a virtual part 1222; the virtual part 1221 can be a bar pattern, and the virtual part 1222 can be a part of the prompt pattern 112. For example, the virtual part 1221 and 1222 can be adjacent. For another example, the height of the virtual element 122 can be equal to the height of the prompt pattern 112, or the height difference between the prompt pattern 112 can be less than or equal to a certain threshold. For another example, the virtual part 1222 can be understood as a part of the prompt pattern 112 that is within the left boundary of the display area 100, assuming that the prompt pattern 112 can fit the road element A without being limited by the display area. For another example, the virtual part 1221 can be understood as a part of the bar pattern 122 shown in (a) of FIG. 6 that is not covered by the virtual part 1222, assuming that the prompt pattern 112 can fit the road element A without being limited by the display area.

[0097] Exemplarily, taking the road element B as an example, when the road element B is relatively moved from outside the right boundary of the display region 100 to inside the display region 100, in the display region 100, the virtual element 123 will be smoothly transitioned from the bar pattern to the prompt pattern. Accordingly, in the user's visual experience, the prompt pattern 113 can be presented as being slowly unfolded to the left (corresponding to the virtual element 123 shown in (b) and (c) of FIG. 6) from the rolled-up state (corresponding to the virtual element 123 shown in (a) of FIG. 6) until the fully unfolded state, and kept in the fully unfolded state (corresponding to the virtual element 123 shown in (d) of FIG. 6) to move following the movement of the road element B. In this way, the effect of smooth transition of the virtual element can be presented.

[0098] The display method of the present application is exemplarily described above by taking the road element as a pedestrian. It can be understood that the road element can be other types than a pedestrian, for example, the road element can include: moving and / or stationary obstacles (such as anti-collision cone barrels, fences, vehicles, animals, trees, etc.), road pavements, lanes, etc.

[0099] The display method of the present application is exemplarily described below by taking the road element as a road / lane corresponding to the navigation information, in combination with FIG. 7 to FIG. 9.

[0100] Exemplarily, FIG. 7 is a schematic diagram of another set of display interfaces provided by an embodiment of the present application. In (a) to (c) of FIG. 7, the display interfaces when the vehicle 10 moves from the pose 1 to the pose 3 are shown.

[0101] Referring to (a) of FIG. 7, the display interface can be the display interface when the vehicle 10 is at the pose 1. The display region 100 can include a navigation pattern 152, which can be fitted to the road corresponding to the navigation information to more intuitively navigate the user.

[0102] The relative position between the road and the display region 100 can change due to the movement of the vehicle 10. During the driving of the vehicle, a situation can occur that part or all of the road is outside the display region 100, and only part of the navigation pattern 152 can be displayed on the display region 100, or even no navigation pattern is displayed. For example, the vehicle can drive from the pose 1 to the pose 3 via the pose 2. For another example, when the vehicle 10 is at the pose 3, the road corresponding to the navigation information can be outside the display region 100, and the display region 100 can display the interface as shown in (b) of FIG. 7; that is, to avoid the navigation pattern 152 being inappropriately fitted to the real environment and interfering with the user, the navigation pattern 152 can not be displayed in the display region 100.

[0103] However, if the navigation pattern 152 is not displayed, it will be difficult for the user to effectively obtain the navigation information, and it will be not conducive to the safety of the user and the vehicle.

[0104] In some possible implementation manners, when the road corresponding to the navigation information is outside the display region 100, a virtual element corresponding to the road can be presented in the form of a text pattern or the like.

[0105] By way of example, FIG. 8 is a schematic diagram of two display interfaces provided by an embodiment of the present application. In FIG. 8, (a) and (b) show two possible display interfaces when the vehicle 10 is in pose 3.

[0106] Referring to (a) in FIG. 8, when the vehicle 10 is in pose 3, a virtual element 162 can be displayed in the display region 100 to indicate the navigation information; the virtual element 162 can include text patterns such as "please turn left", "please turn into the left road", "please turn into the left second lane", and the like.

[0107] Referring to (b) in FIG. 8, when the vehicle is in pose 3, a virtual element 162 can be displayed in the display region 100 in the form of an arrow pattern to indicate the navigation information.

[0108] In one embodiment, when the road corresponding to the navigation information is within the display region 100, the vehicle can be navigated for the user in the manner shown in (a) in FIG. 7; and when part or all of the road corresponding to the navigation information is outside the display region 100, the vehicle can be navigated for the user in the manner shown in FIG. 8. In this technical solution, during the driving of the vehicle, the virtual element corresponding to the road will be switched / transited between the two different presentation manners; the switching / transiting process is not smooth enough, and can interfere with the user. Similar to FIG. 6, when part or all of the road is outside the display region, the corresponding virtual element can be displayed in the form of a bar pattern at the boundary position corresponding to the display region 100, to achieve smooth transition between different forms. By way of example, FIG. 9 is a schematic diagram of another set of display interfaces provided by an embodiment of the present application. (a) to (c) in FIG. 9 show the display interfaces when the vehicle 10 moves from pose 1 to pose 3. In FIG. 9, the navigation pattern 152 is represented by a hatched oblique line, and the virtual element 162 is represented by a gray pattern.

[0109] By way of example, FIG. 9 is a schematic diagram of another set of display interfaces provided by an embodiment of the present application. (a) to (c) in FIG. 9 show the display interfaces when the vehicle 10 moves from pose 1 to pose 3. In FIG. 9, the navigation pattern 152 is represented by a hatched oblique line, and the virtual element 162 is represented by a gray pattern.

[0110] The display interface when the vehicle is in pose 1 can be as shown in (a) in FIG. 9; at this time, the display region 100 can display a virtual element 162 to indicate the navigation information. The virtual element 162 can be fitted with the road corresponding to the navigation information. The road corresponding to the navigation information (or the navigation pattern 152) is all above the lower boundary of the display region 100.

[0111] Referring to (b) in FIG. 9, the display interface can be a display interface when the vehicle is in pose 2.

[0112] Exemplarily, assuming that the vehicle is in pose 2, a part of the road corresponding to the navigation information is outside the lower boundary of the display region 100, and another part of the road is inside the lower boundary of the display region 100. The virtual element 162 can include a virtual part 1621 and a virtual part 1622; the virtual part 1621 can be a bar pattern, and the virtual part 1622 can be a part of the navigation pattern 152. For example, the virtual parts 1621 and 1622 can be adjacent. For another example, the width of the virtual element 162 can be the same as the width of the navigation pattern 152, or the width difference between the width of the virtual element 162 and the width of the navigation pattern 152 is less than or equal to a certain threshold. For another example, the virtual part 1622 can be understood as a part of the navigation pattern 152 that is inside the display region 100, assuming that the navigation pattern 152 can be displayed to fit the road without being affected by the display region. For another example, the virtual part 1621 can be understood as a part of the bar pattern 162 that is not covered by the virtual part 1622, as shown in (c) in FIG. 9, assuming that the navigation pattern 152 can be displayed to fit the road without being affected by the display region.

[0113] Referring to (c) in FIG. 9, the display interface can be a display interface when the vehicle is in pose 3. For example, the virtual element 162 can be displayed in the display region 100 in the form of a bar pattern; the bar pattern can fit the lower boundary of the display region 100.

[0114] Exemplarily, during the movement of the vehicle from pose 1 to pose 3, the virtual element 162 will be smoothly transitioned from the navigation pattern to the bar pattern in the display region 100. Accordingly, in the user's perception, the navigation pattern 152 can be presented as being slowly folded down from the fully unfolded state (the virtual element 162 shown in (a) in FIG. 9) to the virtual element 162 shown in (b) in FIG. 9), and then to the fully folded state (the virtual element 162 shown in (c) in FIG. 9). Thus, the effect of smooth transition of the virtual element can be presented.

[0115] The above describes, in combination with (b) and (c) in FIG. 9, the case where part or all of the road / lane corresponding to the navigation information is outside the lower boundary of the display region 100.

[0116] In some embodiments, when part or all of the road / lane is outside the left / right boundary of the navigation area, the impact of the lack of this part of the navigation information can be considered to be small due to the long distance between the road / lane on the left / right side of the display area and the vehicle, and the virtual element corresponding to this part of the road / lane can be displayed without the bar pattern as shown in (b) and (c) in FIG. 9.

[0117] For example, FIG. 10 is a schematic diagram of a display method provided in an embodiment of the present application. As shown in FIG. 10, the method 500 can include steps S510 and S520.

[0118] S510, obtaining a spatial position of a first road element and a display area of a head-up display device.

[0119] For example, the first road element can be any road element. The road element can include a moving and / or stationary obstacle (such as a crash cone, a fence, a vehicle, an animal, a tree, a pedestrian, etc.), a road, a lane, etc. For example, referring to FIG. 3, the first road element can be the road element A or the road element B. For another example, referring to FIG. 8, the first road element can be the road / lane corresponding to the navigation information.

[0120] The display area of the head-up display device can be used to display a virtual image. For example, the display area can display the interface shown in FIGS. 2 to 9. For example, according to the steering information of the vehicle (such as one or more of the steering angle of the vehicle, the pitch angle of the vehicle, etc.) and the arrangement of the head-up display device on the vehicle, the position of the display area in space can be determined.

[0121] S520, when the first road element is outside a first boundary of the display area or part of the first road element is inside the first boundary, displaying a first virtual element corresponding to the first road element in the display area.

[0122] For example, referring to FIGS. 3 to 6, the virtual element corresponding to the road element A can be the virtual element 122; the virtual element corresponding to the road element B can be the virtual element 123. For another example, referring to FIG. 9, the first road element can be the road / lane corresponding to the navigation information; the virtual element corresponding to the road element can be the virtual element 162.

[0123] In an embodiment of the present application, even if part or all of a road element is outside the first boundary, the virtual element corresponding to the road element can still be displayed in the display area. In this way, the augmented reality effect can be prevented from being limited by the display area of the head-up display device, the performance of the perception system can be prevented from being wasted, and the expression ability of the augmented reality effect can be improved.

[0124] Exemplarily, the road element being outside the display region can be understood as the first fitting region on the road element being outside the display region; the road element being inside the display region can be understood as the first fitting region on the road element being inside the display region; the part of the road element being inside the display region can be understood as the part of the first fitting region on the road element being inside the display region. The first fitting region can be used to fit the virtual element on the road element when the road element is inside the display region.

[0125] For example, for the road elements A and B in FIG. 3 to FIG. 6, the trunk part can be the first fitting region of the road element. For another example, for the road / lane corresponding to the navigation information, the road surface at the center part of the road / lane can correspond to the first fitting region. For another example, for different road elements, the first fitting region of the road element can be determined based on specific requirements and implementation manners.

[0126] In some embodiments, before displaying the first virtual element corresponding to the first road element in the display region, the method can further include: determining that the distance between the first road element and the vehicle is less than or equal to a first threshold. The first threshold can be 4 meters, 5 meters, or can be other numerical values.

[0127] Exemplarily, the first threshold can be related to the type of the road element. For example, when the road element is an obstacle, the first threshold can be 3 meters, 4 meters, to reduce the interference caused by the obstacle without collision risk. For another example, when the road element is the road / lane corresponding to the navigation information, the first threshold can be 15 meters, 20 meters.

[0128] In actual scenarios, there can be multiple obstacles in the surrounding environment of the vehicle; some of the obstacles can have collision risk, and another part of the obstacles can not have collision risk. For the obstacles with collision risk, in the case that the obstacles are outside the display region or part of the obstacles are inside the display region, the corresponding virtual elements are displayed in the display region to prompt the user. For the obstacles without collision risk, in the case that the obstacles are outside the display region or part of the obstacles are inside the display region, if the virtual elements corresponding to the obstacles are displayed in the display region, the user can be interfered.

[0129] In the embodiments of the present application, based on the distance between the road element and the vehicle, it is determined whether to display the corresponding virtual element in the display region, which can reduce the invalid information presented in the display region, and can reduce the interference caused to the user by too many invalid virtual elements in the display region.

[0130] In some possible implementation manners, the first virtual element can include a first form and a second form. The first form of the first virtual element is used to display in the display area when the first road element is outside the first boundary of the display area or a part of the first road element is inside the first boundary. The second form of the first virtual element can be used to display in the display area in a manner of fitting the first road element when the first road element is inside the first boundary of the display area.

[0131] For example, taking the scenario shown in FIG. 3 as an example, for the road element A, the first form of the virtual element 122 can be a text pattern, as shown in (a) to (c) in FIG. 3, and the second form of the virtual element 122 can be a prompt pattern, as shown in (d) in FIG. 3.

[0132] For example, taking the scenario shown in FIG. 6 as an example, for the road element B, the first form of the virtual element 123 can be a bar pattern, as shown in (a) to (c) in FIG. 6, and the second form of the virtual element 123 can be a prompt pattern, as shown in (d) in FIG. 6.

[0133] For example, taking the scenario shown in FIG. 9 as an example, for the road / roadway corresponding to the navigation information, the first form of the virtual element 162 can be a bar pattern, as shown in (b) and (c) in FIG. 9, and the second form of the virtual element 162 can be a navigation pattern, as shown in (a) in FIG. 9.

[0134] In the embodiments of the present application, when a part or all of a road element is outside the display area, the corresponding virtual element is displayed in the first form, and when the road element is inside the display area, the corresponding virtual element can be displayed in a manner of fitting the road element. In this way, on the one hand, even if the road element exceeds the display range, the corresponding virtual element can still be displayed to avoid the loss of relevant information, and on the other hand, when the road element is inside the display area, the user can be provided with relevant information in a more intuitive manner. Therefore, the use requirements of different use scenarios can be taken into account.

[0135] In some possible implementation manners, the first form can include a first part fitting the first boundary, and the second form can include a first pattern fitting the first road element.

[0136] Exemplarily, the first form can only include the first part, or can further include other parts. For example, in the scenario shown in FIG. 3, for the virtual element 122, the first form / first part can be a text pattern, as shown in (a)-(c) of FIG. 3; and the second form can include a prompt pattern, as shown in (d) of FIG. 3. For another example, as shown in (b) and (c) of FIG. 9, for the virtual element 162, the first part can include a bar pattern; and as shown in (a) of FIG. 9, the first pattern can include a navigation pattern. For another example, as shown in (b) and (c) of FIG. 6, for the virtual element 123, the virtual part 1231 can correspond to the first part, and the virtual element 123 can further include a virtual part 1232 in the first form.

[0137] Exemplarily, the first part is fitted to the first boundary, which can be understood as that the distance between the first part and the first boundary is less than or equal to a threshold value; and when the first road element moves relative to the first boundary, the relative distance between the first part and the first boundary remains unchanged.

[0138] For example, when the virtual element 123 includes a bar pattern, the bar pattern can be fitted to the right boundary of the display region 100; and there can be no gap between the bar pattern and the right boundary of the display region 100, as shown in (a)-(c) of FIG. 6. For another example, when the virtual element 123 is presented in the form of a text pattern, there can be a gap between the text pattern and the right boundary of the display region 100, but the distance between the text pattern and the right boundary can remain unchanged, as shown in (a)-(c) of FIG. 3.

[0139] In the embodiments of this application, when the first road element is outside the display region, the first virtual element includes the first part fitted to the first boundary, which can avoid the situation that the first virtual element is improperly displayed and causes interference to the user.

[0140] In some possible implementation manners, when the first road element is outside the first boundary of the display region, displaying the first virtual element corresponding to the first road element in the display region can include: displaying the first part in the display region when the first road element is outside the first boundary of the display region. For example, as shown in (a) of FIG. 3, (a) of FIG. 4, (a) of FIG. 6, and (c) of FIG. 9.

[0141] In some possible implementation manners, the first form can further include a second part, and the second part can be one of the parts of the first pattern. When the part of the first road element is within the first boundary of the display region, displaying the first virtual element corresponding to the first road element in the display region can include: when the part of the first road element is within the first boundary and another part is outside the first boundary, displaying the first part and the second part in the display region, and the first part and the second part are adjacent.

[0142] For example, as shown in (b) and (c) of FIG. 6, for the virtual element 123, the virtual part 1231 can correspond to the first part, and the virtual part 1232 can correspond to the second part. For another example, as shown in (b) of FIG. 9, for the virtual element 162, the virtual part 1621 can correspond to the first part, and the virtual part 1622 can correspond to the second part.

[0143] In some possible implementation manners, the height of the first part near one end of the second part can be equal to the height of the second part near one end of the first part. For example, referring to (b) of FIG. 6, the height between one end of the virtual part 1231 near the virtual part 1232 and one end of the virtual part 1232 near the virtual part 1231 is less than or equal to the second threshold.

[0144] In some possible implementation manners, the first part can include a bar pattern extending along the first boundary. For example, the virtual parts 1221 and 1231 shown in (c) of FIG. 6.

[0145] In some possible implementation manners, in the first form, the projection length of the first virtual element on the first boundary can be equal to the projection length of the first virtual element on the first boundary in the second form.

[0146] For example, in the scenario shown in FIG. 6, for the virtual element 123, the length of the projection of the bar pattern shown in (a) of FIG. 6 on the right boundary of the display region 100 can be equal to the length of the projection of the prompt pattern shown in (d) of FIG. 6 on the boundary. For another example, when the virtual element 123 is displayed in the manner shown in (b) and (c) of FIG. 6, the length of the projection of the virtual element 123 on the right boundary of the display region 100 can be equal to the projection of the corresponding prompt pattern (for example, as shown in (d) of FIG. 6) on the boundary.

[0147] In the embodiments of this application, because the projection lengths of the first virtual element on the first boundary in the two different forms are the same, when the first road element moves relatively from outside the display region to inside the display region, the size of the first virtual element can be prevented from changing suddenly and interfering with the user.

[0148] In some possible implementation manners, the first virtual element can be used to display navigation information and / or prompt information. For example, as shown in FIGS. 3-6, the virtual elements 122 and 123 can be used to display prompt information. For another example, as shown in FIGS. 8 and 9, the virtual element 162 can be used to display navigation information.

[0149] Exemplarily, FIG. 11 is a schematic flowchart of a display method provided by an embodiment of the present application. As shown in FIG. 11, the method 600 can include the following steps:

[0150] S601, triggering AR display.

[0151] Exemplarily, according to the surrounding environment, a road element in the environment that needs to trigger AR display can be determined. For example, when the surrounding environment contains an obstacle (such as the road elements A and B in FIGS. 3-6) that has a collision risk with the vehicle, the AR display can be triggered. For another example, when the navigation information is presented by using AR-HUD, the AR display can be triggered.

[0152] Regarding the manner of triggering AR display, the embodiments of the present application do not make any limitation.

[0153] S602, inputting a three-dimensional (3D) point set to a rendering engine.

[0154] Exemplarily, the data collected by sensors such as radars and cameras can determine the spatial position of the region on the road element for fitting the AR pattern (such as the spatial position of the torso of a cyclist or a pedestrian). The region can be understood as a set of three-dimensional position points (referred to as a 3D point set). When displaying the AR pattern, the corresponding AR pattern needs to be anchored to the three-dimensional position points in the 3D point set, so as to realize the effect of fitting the AR pattern to the road element. Inputting the 3D point set to the rendering engine can fit the corresponding AR pattern to the road element, so that the AR pattern can move along with the movement of the road element.

[0155] S603, displaying AR effect.

[0156] For example, as shown in (d) of FIG. 3, when the road element is in the display region, the prompt pattern can be displayed in a manner of fitting to the road element A and the road element B in the display region.

[0157] S604, obtaining parameters of the head-up display device.

[0158] Exemplarily, the arrangement position, the projection angle, the spatial position of the projection region on the front windshield, and the like of the head-up display device can be obtained. The parameters of the head-up display device can be used to determine the display region of the head-up display device.

[0159] S605, determining whether there is a 3D position point beyond the field of view.

[0160] It can be determined according to the spatial position of the display area whether there is a 3D position point beyond the display area in the point set used for anchoring the AR pattern.

[0161] S606, mapping the 3D position point beyond the field of view to a two-dimensional (2D) area.

[0162] S607, 2D / 3D fusion display.

[0163] The following will be exemplarily described with respect to the road element B and the virtual element 123 in the scenario shown in FIG. 6, to illustrate the steps S603, S606 and S607.

[0164] For example, when all the 3D position points in the 3D point set are within the display area, it can be considered that the road element is within the display area, and the AR pattern can be attached to the road element, as shown in (d) of FIG. 6. The display effect can be referred to as 3D display.

[0165] For another example, when all the 3D position points in the 3D point set are beyond the display area, it can be considered that the road element is beyond the display area, and the part of the 3D position points can be mapped to the vicinity of the boundary of the display area for anchoring the bar pattern; correspondingly, the display interface can be as shown in (a) of FIG. 6. The display effect can be referred to as 2D display.

[0166] For another example, in the case that part of the 3D position points in the 3D point set are within the display area and the other part of the 3D position points are beyond the display area, for the 3D position points not beyond the display area, they can be used to anchor the part of the AR pattern not beyond the display area, which can correspond to the virtual part 1232 in the virtual element 123, as shown in (b) and (c) of FIG. 6. For the 3D position points beyond the display area, they can be mapped to the right boundary of the display area to anchor the corresponding bar pattern (such as the virtual part 1231 in the virtual element 123), as shown in (b) and (c) of FIG. 6. The display effect can be referred to as 2D and 3D fusion display.

[0167] Exemplarily, FIG. 12 is a schematic flowchart of a display method provided by an embodiment of the present application. As shown in FIG. 12, the method 700 can include the following steps:

[0168] S701, acquiring a signal of an advanced driving assistance system (ADAS).

[0169] The ADAS signal can be used to indicate the steering information of the vehicle for the advanced driver assistance function. For example, the ADAS signal can be output by an autonomous driving domain controller; the HUD can obtain the ADAS signal through the communication network inside the vehicle.

[0170] At S702, the vehicle control information is obtained.

[0171] The vehicle control information can be used to indicate the steering of the vehicle by the driver.

[0172] At S703, the AR space is calculated.

[0173] The AR space is a space for presenting the augmented reality effect. In the AR-HUD scenario, the AR space can correspond to the display area.

[0174] Exemplarily, according to the ADAS signal and / or the vehicle control information, the running state of the vehicle can be determined; in combination with the parameters of the head-up display device, the real-time position of the display area can be determined.

[0175] At S704, the 3D point set is obtained.

[0176] The description of the 3D point set can refer to the related description of the method 600.

[0177] At S705, the out-of-frame determination is performed.

[0178] Through S705, it can be determined whether there is a three-dimensional position point in the 3D point set that is outside the display area.

[0179] When all the three-dimensional position points in the 3D point set are within the display area, S706 and S707 can not be performed.

[0180] When all the three-dimensional position points in the 3D point set are outside the display area, S708 and S709 can not be performed.

[0181] At S706, the out-of-frame part is determined.

[0182] Through S706, the three-dimensional position points in the 3D point set that are outside the display area can be screened out.

[0183] At S707, the 2D projection is performed.

[0184] The 2D projection can be understood as rendering the pattern that needs to be presented by the 2D display mode.

[0185] Exemplarily, for the three-dimensional position points outside the display area, they can be mapped to the corresponding positions inside the display area, such as the positions near the boundaries corresponding to the display area. The mapped position points can be used to fit the bar pattern, the text pattern, etc.

[0186] For example, in the scenario shown in (b) and (c) of FIG. 6, for the virtual element 123, the position of the virtual part 1231 in the display region can be determined by 2D projection.

[0187] S708, determining the out-of-frame part.

[0188] By step S708, the three-dimensional position points in the 3D point set that do not exceed the display region can be screened out.

[0189] S709, 3D rendering.

[0190] 3D rendering can be understood as rendering a pattern that needs to be presented by a 3D display mode.

[0191] For example, in the scenario shown in (b) and (c) of FIG. 6, for the virtual element 123, the position of the virtual part 1232 in the display region can be determined by 3D rendering.

[0192] S710, image fusion.

[0193] For example, in the scenario shown in (b) and (c) of FIG. 6, the virtual element 123 containing the virtual parts 1231 and 1232 can be displayed.

[0194] The method provided by the embodiments of the present application is described in detail above in combination with FIG. 3 to FIG. 12. The device provided by the embodiments of the present application will be described in detail below in combination with FIG. 13 and FIG. 14. The description of the device embodiments corresponds to the description of the method embodiments, and therefore, the content not described in detail can be referred to the method embodiments described above.

[0195] Exemplarily, FIG. 13 shows a schematic block diagram of a device 800 provided by the embodiments of the present application. The device 800 can include modules or units for implementing the above-mentioned method embodiments.

[0196] Exemplarily, the device 800 can include an acquisition unit 810 and a display unit 820.

[0197] Exemplarily, the acquisition unit 810 can be configured to acquire the spatial position of the first road element and the display region of the HUD. The display unit 820 can be configured to display a first virtual element corresponding to the first road element in the display region when the first road element is outside the first boundary of the display region or part of the first road element is inside the first boundary.

[0198] In some possible implementation manners, the display unit 820 can be configured to display the first part in the display region when the first road element is outside the first boundary of the display region.

[0199] In some possible implementation, the first form can further include a second part, and the second part can be a part of the first pattern. The display unit 820 can be configured to display the first part and the second part on the display area when the part of the first road element is within the first boundary and the other part is outside the first boundary, and the first part and the second part are adjacent.

[0200] In some possible implementation, before displaying the first virtual element corresponding to the first road element on the display area, the display unit 820 can be configured to determine that the distance between the first road element and the vehicle is less than or equal to a first threshold.

[0201] The display area, the first virtual element, the first road element, the first part, and the second part can be described in the above method embodiments.

[0202] FIG. 14 is a schematic block diagram of an electronic device according to an embodiment of the present application. As shown in FIG. 14, the electronic device 900 can include one or more processors 910, and one or more memories 920. The one or more memories 920 store one or more instructions, and when the instructions are executed by the one or more processors 910, the display method described in any of the possible implementation manners above is performed.

[0203] An apparatus is also provided in the embodiments of the present application, which includes a processor and a communication interface. The communication interface is configured to receive a signal and transmit the signal to the processor. The processor processes the signal, so that the display method described in any of the possible implementation manners above is performed.

[0204] The apparatus can be a chip. For example, the chip can be a chip system or a single chip, etc.

[0205] The embodiments of the present application also provide a readable storage medium, which stores instructions. When the instructions are run on a device, the device performs the above-mentioned related method steps to implement the display method in the above-mentioned embodiments.

[0206] The embodiments of the present application also provide a program product, which, when run on a device, causes the device to perform the above-mentioned related steps to implement the display method in the above-mentioned embodiments.

[0207] The embodiments of the present application also provide an apparatus, which includes modules for implementing the display method described in any of the above embodiments.

[0208] In addition, the embodiment of the present application further provides a device, which can be a chip, a component or a module. The device can include a processor and a memory connected to each other. The memory is used to store instructions. When the device is running, the processor can execute the instructions stored in the memory, so that the device executes the display method in the above method embodiments.

[0209] The device, the readable storage medium, the program product or the apparatus provided in the embodiment can be used to execute the corresponding method provided above, and thus the beneficial effects thereof can refer to the beneficial effects of the corresponding method provided above, which will not be repeated here.

[0210] Those skilled in the art can understand that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be realized in electronic hardware or a combination of computer software and electronic hardware. Whether the functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0211] Those skilled in the art can clearly understand that, for the convenience and brevity of the description, the specific working processes of the above-described system, device and unit can refer to the corresponding processes in the foregoing method embodiments, which will not be repeated here.

[0212] In several embodiments provided in the present application, it should be understood that the disclosed system, device and method can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division, and there can be another division manner in actual implementation. For example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interface, device or unit, and can be electrical, mechanical or in other forms.

[0213] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e. they can be located in one place, or can be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0214] In addition, each functional unit in each embodiment of the present application can be integrated into one processing unit, or each unit can exist physically, or two or more units can be integrated into one unit.

[0215] If the functions are realized in the form of software function units and sold or used as independent products, they can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application or the parts of the present application that essentially contribute to the prior art or the parts of the technical solutions can be embodied in the form of software products. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the embodiments of the present application. The aforementioned storage medium includes a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0216] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or replacements within the technical scope disclosed in the present application, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A display method characterized by comprising: The method comprises: acquiring a spatial position of a first road element and a display region of a head-up display (HUD), the display region of the HUD being used to display a virtual image; displaying a first virtual element corresponding to the first road element in the display region when the first road element is outside a first boundary of the display region or a part of the first road element is inside the first boundary.

2. The method of claim 1, wherein: the first virtual element comprises a first form and a second form, the first form of the first virtual element is used to be displayed in the display region when the first road element is outside the first boundary of the display region or a part of the first road element is inside the first boundary; and the second form of the first virtual element is used to be displayed in the display region in a manner of fitting the first road element when the first road element is inside the first boundary of the display region.

3. The display method according to claim 2, wherein the first form comprises a first part fitting the first boundary, and the second form comprises a first pattern fitting the first road element.

4. The method of claim 3, wherein: the displaying the first virtual element corresponding to the first road element in the display region when the first road element is outside the first boundary of the display region or a part of the first road element is inside the first boundary comprises: the displaying the first part in the display region when the first road element is outside the first boundary of the display region.

5. The display method according to claim 3, wherein the first form further comprises a second part being a part of the first pattern, the displaying the first virtual element corresponding to the first road element in the display region when the first road element is outside the first boundary of the display region or a part of the first road element is inside the first boundary comprises: the displaying the first part and the second part in the display region when a part of the first road element is inside the first boundary and another part of the first road element is outside the first boundary, the first part and the second part being adjacent.

6. The method of claim 5, wherein: a height of the first part near an end of the second part is equal to a height of the second part near an end of the first part.

7. The display method according to any one of claims 3 to 6, wherein, the first part comprises a strip pattern extending along the first boundary.

8. The method of any one of claims 2 to 7, wherein: a projection length of the first virtual element in the first frame in the first form is equal to a projection length of the first virtual element in the first frame in the second form.

9. The display method according to any one of claims 1 to 8, wherein, the first virtual element is used to display navigation information and / or prompt information.

10. The display method according to any one of claims 1 to 9, wherein, Before the displaying the first virtual element corresponding to the first road element in the display region, the method further comprises: determining that a distance between the first road element and a vehicle is less than or equal to a first threshold.

11. The display method of any one of claims 1-10, wherein: the first road element comprises a first fitting region, the first fitting region being configured to fit the first virtual element when the first road element is within the display region; the first road element being outside the first boundary of the display region comprises the first fitting region being outside the first boundary of the display region; the portion of the first road element being within the first boundary of the display region comprises a portion of the first fitting region being within the first boundary of the display region.

12. A display device, characterized by comprising: comprising: an acquisition unit configured to acquire a spatial position of a first road element and a display region of a head-up display (HUD), the display region of the HUD being configured to display a virtual image; a display unit configured to display a first virtual element corresponding to the first road element in the display region when the first road element is outside a first boundary of the display region or a portion of the first road element is within the first boundary.

13. The display device of claim 12, wherein, the first virtual element comprises a first form and a second form, the first form of the first virtual element is configured to be displayed in the display region when the first road element is outside the first boundary of the display region or the portion of the first road element is within the first boundary; the second form of the first virtual element is configured to be displayed in the display region in a manner of fitting the first road element when the first road element is within the first boundary of the display region.

14. The display device of claim 13, wherein, the first form comprises a first portion fitting the first boundary, and the second form comprises a first pattern fitting the first road element.

15. The display device of claim 14, wherein, the display unit is configured to: display the first portion in the display region when the first road element is outside the first boundary of the display region.

16. The display device of claim 14, wherein: the first form further comprises a second portion, the second portion being a portion of the first pattern, the display unit is configured to: display the first portion and the second portion in the display region when the portion of the first road element is within the first boundary and another portion of the first road element is outside the first boundary, the first portion and the second portion being adjacent.

17. The display device of claim 16, wherein: a height of the first portion near an end of the second portion is equal to a height of the second portion near an end of the first portion.

18. A display device according to any one of claims 14 to 17, wherein, the first portion comprises a strip pattern extending along the first boundary.

19. A display device according to any one of claims 13 to 18, wherein, a projection length of the first virtual element on the first frame in the first form is equal to a projection length of the first virtual element on the first frame in the second form.

20. A display device according to any one of claims 12 to 19, wherein, the first virtual element is configured to display navigation information and / or prompt information.

21. A display device according to any one of claims 12 to 20, wherein, before the display unit displays the first virtual element corresponding to the first road element in the display region, the display unit is further configured to: determine that a distance between the first road element and the vehicle is less than or equal to a first threshold.

22. The display apparatus according to any one of claims 12 to 21, wherein the first road element includes a first fitting region configured to fit the first virtual element when the first road element is within the display region; the first road element being outside the first boundary of the display region includes the first fitting region being outside the first boundary of the display region; the portion of the first road element being within the first boundary of the display region includes a portion of the first fitting region being within the first boundary of the display region.

23. An electronic device, comprising: comprise: one or more processors; one or more memories storing one or more programs, which, when executed by the one or more processors, cause the display method according to any one of claims 1-11 to be performed.

24. A chip, characterized by The chip comprises a processor and a communication interface, the communication interface is used for receiving a signal and transmitting the signal to the processor, and the processor processes the signal, so that the display method according to any one of claims 1-11 is executed.

25. A readable storage medium, characterized by, The readable storage medium stores instructions, when the instructions run on the electronic device, so that the display method according to any one of claims 1-11 is executed.

26. A program product, characterized by The program product comprises program code, when the program code runs on the electronic device, so that the display method according to any one of claims 1-11 is executed.

Images