Prompt identifier display method and device, equipment and medium

Abstract

The invention relates to a prompt identifier display method and device, equipment and a medium. The method comprises the steps of determining a target imaging position of a first prompt identifier in a simulated three-dimensional space based on monitored environment information data, and configuring a capturer for capturing an image in the simulated three-dimensional space; according to the target imaging position and the imaging parameters of the AR-HUD, view angle information of the first prompt identifier is determined, and the view angle information is used for indicating the capturer to view the view angle of the first prompt identifier so as to form a 2D or 3D first prompt identifier; according to the target imaging position, the visual angle information and the parameters of the first prompt identifier, rendering the first prompt identifier in the simulated three-dimensional space; an image containing the rendered first prompt identifier is captured through a capturer to form a virtual image containing the first prompt identifier, and the imaging position of the first prompt identifier in the virtual image is matched with the target imaging position. According to the invention, a user can clearly recognize the 2D identifier and the 3D identifier of the same information.

Description

Technical Field

[0001] The present disclosure relates to the field of intelligent driving technology, and in particular to a prompt sign display method, device, equipment and medium. Background Art

[0002] AR-HUD (Augmented Reality Heads-Up Display) can provide many functions to help users drive cars more safely and comfortably.

[0003] At present, AR-HUD usually sets a first virtual image layer with a fixed imaging distance. It can also configure an AR imaging layer with an imaging distance different from the first virtual image layer. The first virtual image layer is used to display the vehicle's driving information and various prompts about different monitoring objects on the road, so that these prompt information is on a virtual image layer with a fixed distance in the user's field of view. When certain judgment conditions are met, AR-HUD will display the prompt information related to the monitoring object in the AR imaging layer in the form of AR, so that users can receive these prompt information more conveniently.

[0004] However, the existing function prompts are likely to confuse users when jumping between different layers, making it difficult for users to quickly understand the meaning of the function prompts, resulting in difficulty in achieving the prompting effect. Summary of the invention

[0005] In order to solve the above technical problem or at least partially solve the above technical problem, the present disclosure provides a prompt mark display method, device, equipment and medium.

[0006] In a first aspect, the present disclosure provides a prompt mark display method, which is applied to an AR-HUD, and the method includes:

[0007] Based on the monitored environmental information data, determining a target imaging position of the first prompt mark in a simulated three-dimensional space, wherein a capturer for capturing images is configured in the simulated three-dimensional space;

[0008] Determine, according to the target imaging position and the imaging parameters of the AR-HUD, viewing angle information of the first prompt mark, wherein the viewing angle information is used to indicate the viewing angle of the capturer to view the first prompt mark, so as to form a 2D or 3D first prompt mark;

[0009] Rendering the first prompt mark in the simulated three-dimensional space according to the target imaging position, the viewing angle information and the parameters of the first prompt mark;

[0010] An image including the rendered first prompt mark is captured by a capture device to form a virtual image including the first prompt mark, wherein an imaging position of the first prompt mark in the virtual image matches the target imaging position.

[0011] Optionally, determining a target imaging position of the first prompt mark in the simulated three-dimensional space based on the monitored environmental information data includes:

[0012] Based on the environmental information data, determining whether a switching display condition of the first prompt mark is met;

[0013] If the switching display condition is met, the target object corresponding to the first prompt mark is determined according to the environmental information data, and the target imaging position of the first prompt mark in the simulated three-dimensional space is determined according to the position of the target object.

[0014] Optionally, the AR-HUD includes a first imaging layer and an AR imaging layer, the imaging parameters of the AR-HUD include a first imaging distance of the first imaging layer and an imaging distance interval of the AR imaging layer, and the imaging distance interval includes at least one second imaging distance;

[0015] The determining, according to the target imaging position and the imaging parameters of the AR-HUD, the viewing angle information of the first prompt mark includes:

[0016] If the imaging distance of the target imaging position matches the imaging distance of the first imaging layer, the viewing angle information of the first prompt mark is determined to be a normal viewing angle to form the first prompt mark of 2D; or,

[0017] If the imaging distance of the target imaging position matches the imaging distance interval of the AR imaging layer, the viewing angle information of the first prompt mark is determined to be a set viewing angle different from the normal viewing angle, so as to form the first 3D prompt mark.

[0018] Optionally, determining a target imaging position of the first prompt mark in the simulated three-dimensional space according to the position of the target object includes:

[0019] Taking the position of the target object as the end position of the first prompt mark, determining a planned path according to the current display position of the first prompt mark, and determining the target imaging position of the first prompt mark according to the planned path; or,

[0020] The position of the target object is used as the target imaging position of the first prompt mark.

[0021] Optionally, rendering the first prompt mark in the simulated three-dimensional space according to the target imaging position, the viewing angle information and the parameters of the first prompt mark includes:

[0022] Determining a thickness of the first prompt mark in the parameters of the first prompt mark according to a distance between the target imaging position and a target object corresponding to the first prompt mark;

[0023] According to the target imaging position, the first prompt mark is rendered in the simulated three-dimensional space according to the parameters for determining the first prompt mark.

[0024] In a second aspect, the present disclosure provides a prompt identification display device, which is applied to an AR-HUD, and the device includes:

[0025] A monitoring module, used to determine a target imaging position of a first prompt mark in a simulated three-dimensional space based on monitored environmental information data, wherein a capturer for capturing images is configured in the simulated three-dimensional space;

[0026] A determination module, configured to determine viewing angle information of the first prompt mark according to the target imaging position and the imaging parameters of the AR-HUD, wherein the viewing angle information is used to indicate the viewing angle of the capturer to view the first prompt mark, so as to form a 2D or 3D first prompt mark;

[0027] A rendering module, configured to render the first prompt mark in the simulated three-dimensional space according to the target imaging position, the viewing angle information and the parameters of the first prompt mark;

[0028] An imaging module is used to capture an image including the rendered first prompt mark through a capture device to form a virtual image including the first prompt mark, wherein an imaging position of the first prompt mark in the virtual image matches the target imaging position.

[0029] Optionally, when the monitoring module determines the target imaging position of the first prompt logo in the simulated three-dimensional space based on the monitored environmental information data, the monitoring module is specifically used to determine whether a switching display condition of the first prompt logo is met based on the environmental information data; if the switching display condition is met, the target object corresponding to the first prompt logo is determined according to the environmental information data, and the target imaging position of the first prompt logo in the simulated three-dimensional space is determined according to the position of the target object.

[0030] Optionally, the AR-HUD includes a first imaging layer and an AR imaging layer, the imaging parameters of the AR-HUD include a first imaging distance of the first imaging layer and an imaging distance interval of the AR imaging layer, and the imaging distance interval includes at least one second imaging distance;

[0031] When the determination module determines the viewing angle information of the first prompt mark according to the target imaging position and the imaging parameters of the AR-HUD, it is specifically used to determine that the viewing angle information of the first prompt mark is a normal viewing angle if the imaging distance of the target imaging position matches the imaging distance of the first imaging layer, so as to form the first prompt mark in 2D; or, if the imaging distance of the target imaging position matches the imaging distance interval of the AR imaging layer, the viewing angle information of the first prompt mark is determined to be a set viewing angle different from the normal viewing angle, so as to form the first prompt mark in 3D.

[0032] Optionally, when the monitoring module determines the target imaging position of the first prompt marker in the simulated three-dimensional space based on the position of the target object, the monitoring module is specifically used to use the position of the target object as the end point position of the first prompt marker, determine a planned path based on the current display position of the first prompt marker, and determine the target imaging position of the first prompt marker based on the planned path; or, use the position of the target object as the target imaging position of the first prompt marker.

[0033] Optionally, when the rendering module renders the first prompt marker in the simulated three-dimensional space according to the target imaging position, the viewing angle information and the parameters of the first prompt marker, the rendering module is specifically used to determine the thickness of the first prompt marker in the parameters of the first prompt marker according to the distance between the target imaging position and the target object corresponding to the first prompt marker; and render the first prompt marker in the simulated three-dimensional space according to the parameters of the first prompt marker determined according to the target imaging position.

[0034] In a third aspect, the present disclosure provides an electronic device, including:

[0035] Memory;

[0036] Processor; and

[0037] Computer programs;

[0038] The computer program is stored in the memory and is configured to be executed by the processor to implement the method as described in any one of the first aspects.

[0039] In a fourth aspect, the present disclosure provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the method as described in any one of the first aspects.

[0040] Compared with the prior art, the technical solution provided by the present invention has the following advantages:

[0041] The prompt mark display method, device, equipment and medium provided by the present disclosure render a 2D or 3D first prompt mark at a target imaging position corresponding to the position to be displayed of the first prompt mark in a simulated three-dimensional space of the imaging space simulating the AR-HUD, and different 2D or 3D states can indicate whether the function corresponding to the prompt mark is triggered, and then capture an image containing the 2D or 3D first prompt mark rendered at the target imaging position through a capturer, so as to form a 2D or 3D virtual image of the first prompt mark at the position to be displayed of the first prompt mark in the imaging space of the AR-HUD, so as to present the first prompt mark in 2D form or 3D form in the user's field of view, because the first prompt mark is displayed in different states at different imaging positions, and the user can see the process of state switching, so that the user can determine that the 2D first prompt mark and the 3D first prompt mark reflect the same prompt information, and at the same time realize the mutually exclusive display between the 2D first prompt mark and the 3D first prompt mark, so that the user can clearly recognize the 2D mark and the 3D mark of the same information, and avoid confusion and confusion caused by the lack of association between marks of the same function. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0043] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0044] Figure 1 A flowchart of a method for displaying a prompt mark provided by an embodiment of the present disclosure;

[0045] Figure 2 A schematic diagram of an application scenario provided by an embodiment of the present disclosure;

[0046] Figure 3 A schematic diagram of another application scenario provided by an embodiment of the present disclosure;

[0047] Figure 4 A schematic diagram of another application scenario provided by an embodiment of the present disclosure;

[0048] Figure 5 A schematic diagram of the structure of a prompt sign display device provided in an embodiment of the present disclosure;

[0049] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0050] In order to more clearly understand the above-mentioned objectives, features and advantages of the present disclosure, the scheme of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features in the embodiments can be combined with each other without conflict.

[0051] In the following description, many specific details are set forth to facilitate a full understanding of the present disclosure, but the present disclosure may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present disclosure, rather than all of the embodiments.

[0052] Figure 1 A flowchart of a prompt identification display method provided in an embodiment of the present disclosure. The method is applicable to AR-HUD, and the method can be executed by a prompt identification display device, which can be implemented in software and / or hardware, and the prompt identification display device can be configured in AR-HUD.

[0053] Figure 2 A schematic diagram of an application scenario provided by an embodiment of the present disclosure. Figure 2 As shown, the scene includes the vehicle 210, the front windshield 211 of the vehicle 210, the driver 220, the PGU module 230 of the AR-HUD, the AR engine 232 of the AR-HUD, the front vehicle 260, etc.

[0054] For the sake of convenience, the structure of AR-HUD is first explained:

[0055] The PGU module 230 of the AR-HUD includes a display screen. In addition, the AR-HUD also includes relay optical components, etc. The display screen is used to output image light, which is amplified by the relay optical components, incident on the windshield, and then reflected by the windshield into the eye box. When the user's eyes are in the eye box, they can receive the light and see the virtual image. In this embodiment, the AR-HUD includes two PGU modules 230, one of which is used to form a first imaging layer. The other PGU module 230 is used to form an AR imaging layer 250.

[0056] The imaging distance of the first imaging layer is fixed, and can be a virtual image perpendicular to the reference surface, and the reference surface can be the support plane of the vehicle. The first imaging distance is 2 meters to 6 meters, for example, 2 meters, 3 meters, 4 meters, 5 meters, 6 meters, etc.

[0057] Usually AR-HUD can form a spatial light field, so as to achieve different distances ( Figure 2 center left and right) and height ( Figure 2In this embodiment, for ease of understanding, the AR imaging layer 250 formed in the spatial light field is taken as an example.

[0058] The AR imaging layer may have only one imaging distance different from the first imaging distance, such as 10 meters, 11 meters, 12 meters, 13 meters, 14 meters, 15 meters, etc. Alternatively, the AR imaging layer may have multiple different imaging distances to form an imaging distance interval, and the imaging distance interval may include multiple interval imaging distances, such as 10 meters, 12 meters and 15 meters, or may include multiple continuous imaging distances, such as 10 meters to 20 meters, etc. The AR imaging layer 250 may be regarded as an imaging space.

[0059] The AR engine 232 can be configured in the vehicle's controller, or in other processors with data processing capabilities that are independent of the vehicle controller, without limitation. Generally, one or more sensors can be configured on the vehicle to detect data from the external environment. The data of the external environment may include, but is not limited to: external vehicles, pedestrians, buildings, etc. The data of the external environment can be transmitted to the AR engine 232 so that it can render the content to be presented based on the external environment, and then output the image to the PGU module 230 for display.

[0060] It is understandable that the prompt identification display method provided by the embodiment of the present disclosure can also be applied in other scenarios. Figure 2 The application scenario shown is Figure 1 The prompt mark display method shown in the figure is introduced. The prompt mark display method includes the following steps:

[0061] S101. Determine a target imaging position of a first prompt mark in a simulated three-dimensional space based on monitored environmental information data, wherein a capture device for capturing images is disposed in the simulated three-dimensional space.

[0062] The simulated three-dimensional space refers to a virtual three-dimensional space used to simulate a real environment space, which is configured in the AR engine. A capturer is configured in the simulated three-dimensional space for capturing images, and the capturer is used to simulate the human eye. The spatial position relationship of the first prompt mark rendered in the simulated three-dimensional space relative to the capturer is consistent with the relative position between the first prompt mark displayed by the virtual image and the human eye.

[0063] In some embodiments, in order to reduce the load of the capturer, the capturer is configured with a capture boundary and depth. For example, for the capturer corresponding to the PGU module used to form the first imaging layer, the capture boundary is consistent with the boundary corresponding to the FOV (which can be the viewing angle range) of the first imaging layer, and the captured content is the content at the first imaging distance of the first imaging layer. For the capturer corresponding to the PGU module used to form the AR imaging layer, the capture boundary can be the range corresponding to the maximum FOV of the AR imaging layer, and the capture distance can be the maximum imaging distance of the AR imaging layer.

[0064] For example, refer to Figure 2 During the driving process of the vehicle 210, the environmental information data collected by the sensors of the vehicle 210 can be transmitted to the AR engine 232. The AR engine 232 identifies and analyzes these environmental information data to determine the prompt logo currently provided to the driver and the position where the prompt logo should be displayed in the imaging space of the AR-HUD.

[0065] Taking the preceding vehicle locking prompt as an example, when the user turns on the preceding vehicle locking function, the preceding vehicle 260 travels in front of the vehicle 210, and the perception sensors of the vehicle 210 (such as cameras, laser radar collection, and millimeter-wave radar, etc.) collect environmental information data in front of the vehicle 210, and the environmental information data includes perception data about the preceding vehicle 260. The AR engine 232 can measure the real-time distance between the vehicle 210 and the preceding vehicle 260 by identifying and analyzing these perception data. When the AR engine 232 measures that the preceding vehicle 260 is greater than 100m (or other set values) from the vehicle 210, the first prompt mark corresponding to the preceding vehicle locking function is displayed on the first imaging layer 240 of the AR-HUD, such as Figure 2 The logo 241 shown is a 2D first prompt logo, which indicates that the user has turned on the front vehicle locking function and there are no vehicles that affect driving safety in the current environment. When the AR engine 232 measures that the front vehicle 260 is less than 100m away from the vehicle 210, the first prompt logo can be displayed in the AR imaging layer 250 of the AR-HUD, and the displayed position matches the front vehicle 260 to achieve the effect of highlighting the front vehicle. In this state, Figure 2 The logo 251 shown is the first prompt logo of 3D. This can enhance the stereoscopic effect on the one hand, and let the user know on the other hand that the logo 251 is the logo 241 converted to 3D state, and the logo 241 is no longer displayed. The two are the same logo.

[0066] In some embodiments, determining a target imaging position of a first prompt mark in a simulated three-dimensional space based on monitored environmental information data includes:

[0067] S1011, determining whether a switching display condition of a first prompt mark is met based on the environmental information data;

[0068] The first prompt mark corresponds to different functions, and its switching display conditions may also be different. For example, if the first prompt mark is the mark of the front vehicle locking function, its switching display condition may be: the triggering state of the front vehicle locking function changes, such as when the front vehicle locking function is in the triggered state, when it is detected that the front vehicle 260 is more than 100m away from the vehicle 210, the state of the front vehicle locking function switches from the triggered state to the waiting state. In this case, the first prompt mark switches from the 3D mode to the 2D mode. Vice versa.

[0069] Of course, the switching display condition may be other appropriate conditions and is not limited thereto.

[0070] S1012: If the switching display condition is met, determine the target object corresponding to the first prompt mark according to the environmental information data, and determine the target imaging position of the first prompt mark in the simulated three-dimensional space according to the position of the target object.

[0071] After determining that the environmental information data meets the switching display condition of the first prompt mark, the AR engine 232 determines the target imaging position of the first prompt mark in the simulated three-dimensional space according to the position of the target object corresponding to the first prompt mark. Figure 2 In the illustrated scene, the target object corresponding to the first prompt mark locked by the front vehicle is the front vehicle 260. The target objects of the prompt marks corresponding to other functions may be different, such as pedestrians, buildings, POIs, etc., without limitation.

[0072] The disclosed embodiment exemplarily provides two switching display modes. Based on the different switching display modes, the methods of determining the target imaging position of the first prompt mark in the simulated three-dimensional space according to the position of the target object during the switching process are also different. The following introduces the methods of determining the target imaging position in the two switching display modes.

[0073] One of the switching display methods takes into account the linkage of the first prompt mark when switching the display, and displays the process of the first prompt mark moving from the current display position to the end position through a multi-frame animation. For example, the position of the front vehicle relative to the vehicle is determined based on the environmental information data, and it is determined that the distance between the front vehicle and the vehicle is less than 100m. The first prompt mark needs to be displayed at a position that matches the front vehicle. If the front vehicle is within the AR-HUD imaging coverage range, the end position of the first prompt mark is consistent with the position of the front vehicle. If the front vehicle exceeds the AR-HUD imaging distance in terms of distance, the end position of the first prompt mark is aligned with the front vehicle in the X and Y directions, and is the maximum imaging distance in the Z direction.

[0074] In this case, the target imaging position of the first prompt marker in the simulated three-dimensional space is determined according to the position of the target object, including: taking the position of the target object as the end point position of the first prompt marker, determining the planned path according to the current display position of the first prompt marker, and determining the target imaging position of the first prompt marker according to the planned path.

[0075] For example, when switching to display the first prompt mark, two frames of moving pictures can be used as a transition animation for the first prompt mark to move to the end position, that is, when the third frame is displayed, the first prompt mark is displayed at the end position.

[0076] When the first prompt mark of 2D is switched to the first prompt mark of 3D, the terminal position of the first prompt mark can be the rear position of the preceding vehicle 260. At the beginning of the switching display, the current display position of the first prompt mark refers to the position of the mark 241. The AR engine 232 determines the planned path A1 according to the current display position of the first prompt mark and the rear position of the preceding vehicle 260, and then determines the target imaging position of the first prompt mark in the next frame according to the planned path A1. The target imaging position is recorded as S1. The position S1 can be determined according to a preset rule. For example, the position S1 can be a position in the planned path A1 that is one-third of the length of the planned path A1 from the current display position. After that, the AR engine 232 executes S102, S103, and S104 to form a virtual image containing the first prompt mark through the PGU module 230. The virtual image shows the first frame of the transition animation of the movement of the first prompt mark. The first prompt mark comes to the position L1 in the AR imaging layer 250 that matches the target imaging position S1.

[0077] At this time, the current display position of the first prompt mark is position L1, and the AR engine 232 repeats the above steps. First, the planned path A2 is determined based on position L1 and the rear position of the front vehicle 260. It should be noted that during driving, compared with the planned path A1, the position of the rear of the front vehicle 260 in the planned path A2 relative to the vehicle 210 may have changed. The AR engine 232 determines the target imaging position of the first prompt mark in the next frame based on the planned path A2, and the target imaging position is recorded as S2. For example, position S1 can be the center position of the planned path A2. Then S102, S103, and S104 are executed again, so that the virtual image of the second frame in the transition animation is formed through the PGU module 230, and the first prompt mark comes to the position L2 in the AR imaging layer 250 that matches the target imaging position S2.

[0078] After the above operation, the two frames of the transition animation of the first prompt mark switching from 2D to 3D have been displayed. At this time, the current display position of the first prompt mark is position L2. The first prompt mark will be displayed at the end position in the next frame, that is, the third frame. At this time, the position of the rear of the front vehicle 260 relative to the vehicle 210 may also change. The AR engine 232 determines the target imaging position of the first prompt mark in the next frame according to the rear position of the front vehicle 260. The target imaging position is recorded as S3, and the position S3 matches the rear position of the front vehicle 260. S102, S103, and S104 are executed again, and the virtual image of the third frame is formed by the PGU module 230. In the virtual image, the first prompt mark comes to the rear position of the front vehicle 260 in the AR imaging layer 250. Or when the rear position of the front vehicle 260 exceeds the AR-HUD imaging coverage, the position of the first prompt mark in the virtual image is aligned with the front vehicle in the X and Y directions, and is the maximum imaging distance in the Z direction.

[0079] When the first 3D prompt mark is switched to the first 2D prompt mark, the terminal position of the first prompt mark can be a preset position in the first imaging layer 240, which can be a fixed position or a position of the front vehicle 260 relative to the vehicle 210 in the plane map. When the first 3D prompt mark starts to switch to the first 2D prompt mark, the current display position of the first prompt mark is the rear position of the front vehicle 260, referring to the position of the mark 251. The AR engine 232 determines the planned path B1 according to the rear position of the front vehicle 260 and the preset position, and then determines the target imaging position of the first prompt mark in the next frame according to the planned path B1, and the target imaging position is recorded as S4. For example, position S4 can be a position in the planned path B1 that is one-third of the length of the planned path B1 from the current display position. Then the AR engine 232 executes S102, S103, and S103 to form a virtual image of the first frame in the transition animation through the PGU module 230, and the first prompt mark in the virtual image is at a position L4 in the AR imaging layer 250 that matches the target imaging position S4.

[0080] At this time, the current display position of the first prompt mark is position L4. The AR engine 232 determines the planned path B2 based on position L4 and the preset position in the first imaging layer 240. When the preset position represents the position of the front vehicle 260 relative to the vehicle 210 in the plane map, since the position of the front vehicle 260 relative to the vehicle 210 may have changed, the preset position may also change with the change of the position of the front vehicle 260 relative to the vehicle 210. Then, the target imaging position of the first prompt mark in the next frame is determined according to the planned path B2, and the target imaging position is recorded as S5. For example, position S5 can be the center position of the planned path B2. S102, S103, and S104 are executed again, and a virtual image of the second frame in the transition animation is formed through the PGU module 230. In the virtual image, the first prompt mark comes to the position L5 in the AR imaging layer 250 that matches the target imaging position S5.

[0081] At this time, the two frames of the transition animation of the first prompt mark switching from 3D to 2D have been displayed, and the current display position of the first prompt mark is position L5. The first prompt mark in the next frame will be displayed at the preset position in the first imaging layer 240. The AR engine 232 determines the target imaging position of the first prompt mark in the next frame according to the preset position, and the target imaging position is recorded as S6. Position S6 matches the preset position in the first imaging layer 240. S102, S103, and S104 are also executed to form a virtual image of the third frame in the transition animation through the PGU module 230. In the virtual image, the first prompt mark is at the preset position in the first imaging layer 240.

[0082] Another switching display method provided by the embodiment of the present disclosure is to display the first prompt mark by jumping from the current display position to the end position.

[0083] In this case, determining the target imaging position of the first prompt marker in the simulated three-dimensional space according to the position of the target object includes: using the position of the target object as the target imaging position of the first prompt marker.

[0084] When the 2D first prompt mark is switched to the 3D first prompt mark, the AR engine 232 uses the rear position of the front vehicle 260 as the target imaging position of the first prompt mark. Then execute S102, S103, and S104, and form a virtual image containing the 3D first prompt mark through the PGU module 230, in which the first prompt mark is located at the rear position of the front vehicle 260 in the AR imaging layer 250. Or when the rear position of the front vehicle 260 exceeds the AR-HUD imaging coverage, the position of the first prompt mark in the virtual image is aligned with the front vehicle in the X and Y directions, and is the maximum imaging distance in the Z direction.

[0085] When the 3D first prompt mark is switched to the 2D first prompt mark, the AR engine 232 uses the preset position in the first imaging layer 240 as the target imaging position of the first prompt mark. Then, S102, S103, and S104 are executed to form a virtual image including the 2D first prompt mark through the PGU module 230.

[0086] The disclosed embodiment provides two switching display modes for users to choose and apply, which can increase the flexibility of the solution.

[0087] S102: Determine viewing angle information of the first prompt mark according to the target imaging position and the imaging parameters of the AR-HUD, where the viewing angle information is used to indicate the viewing angle of the capturer to view the first prompt mark, so as to form a 2D or 3D first prompt mark.

[0088] Exemplarily, after determining the target imaging position of the first prompt mark, the AR engine 232 determines the viewing angle information of the first prompt mark according to the target imaging position and the imaging parameters of the AR-HUD, and the viewing angle information is used to indicate the viewing angle of the first prompt mark of the capturer, that is, the viewing angle of the capturer looking at the first prompt mark at the target imaging position. When the first prompt mark is at different target imaging positions, the first prompt mark viewed by the capturer can be presented as a 2D first prompt mark or a 3D first prompt mark.

[0089] It has been introduced above that the AR-HUD includes a first imaging layer 240 and an AR imaging layer 250. The imaging parameters of the AR-HUD in the embodiment of the present disclosure may include a first imaging distance of the first imaging layer 240 and an imaging distance interval of the AR imaging layer 250, and the imaging distance interval includes at least one second imaging distance.

[0090] In some embodiments, the viewing angle information of the first prompt mark is determined according to the target imaging position and the imaging parameters of the AR-HUD, including: if the imaging distance of the target imaging position matches the imaging distance of the first imaging layer, the viewing angle information of the first prompt mark is determined to be a normal viewing angle to form a 2D first prompt mark; or, if the imaging distance of the target imaging position matches the imaging distance interval of the AR imaging layer, the viewing angle information of the first prompt mark is determined to be a set viewing angle different from the normal viewing angle to form a 3D first prompt mark.

[0091] In the disclosed embodiment, when the imaging distance of the target imaging position matches the imaging distance of the first imaging layer, it is determined that the viewing angle information of the capturer when viewing the first prompt marker is a front view perspective, so that after the AR engine renders the first prompt marker based on the front view perspective, the picture obtained by the capturer after capturing the rendered first prompt marker can be a front view of the first prompt marker to form a 2D first prompt marker. Figure 3301 and a dashboard 302. Figure 3 Logo 301 is shown.

[0092] When the imaging distance of the target imaging position matches the imaging distance interval of the AR imaging layer, the viewing angle information of the first prompt marker is determined to be any set viewing angle that is different from the front view viewing angle. When looking at the first prompt marker from the set viewing angle that is different from the front view viewing angle, the details on the side of the first prompt marker can be observed, such as the thickness of the first prompt marker, so that the first prompt marker appears three-dimensional, so that after the AR engine renders the first prompt marker based on the set viewing angle, the picture obtained by the capturer after capturing the rendered first prompt marker can be a three-dimensional image of the first prompt marker to form a 3D first prompt marker. Figure 4 is a schematic diagram of another application scenario provided by an embodiment of the present disclosure, and the scenario includes a logo 401 and a dashboard 402. The first prompt logo rendered based on a certain set perspective different from the front view perspective can be referred to Figure 4 Logo 401 is shown.

[0093] S103: Rendering the first prompt mark in the simulated three-dimensional space according to the target imaging position, the viewing angle information and the parameters of the first prompt mark.

[0094] The parameters of the first prompt mark in the disclosed embodiment refer to the appearance geometric parameters, including shape, thickness, etc. The AR engine can determine whether the rendered first prompt mark is a 2D first prompt mark or a 3D first prompt mark based on the viewing angle information, and then generate a 2D or 3D model of the corresponding appearance shape at the target imaging position in the simulated three-dimensional space based on the parameters of the first prompt mark, and then calculate the shadow, reflection, and lighting simulation information of the model in the simulated three-dimensional space based on the viewing angle information of the capture device viewing the model at the target imaging position, so as to render the final effect of the 2D first prompt mark or the 3D first prompt mark.

[0095] In one embodiment, the first prompt marker is rendered in a simulated three-dimensional space according to the target imaging position, viewing angle information and parameters of the first prompt marker, including: determining the thickness of the first prompt marker in the parameters of the first prompt marker according to the distance between the target imaging position and the target object corresponding to the first prompt marker; and rendering the first prompt marker in the simulated three-dimensional space according to the parameters of the first prompt marker determined according to the target imaging position.

[0096] In the embodiment of the present disclosure, the smaller the distance between the target imaging position and the target object corresponding to the first prompt mark, the larger the imaging distance corresponding to the target imaging position.

[0097] When the AR engine renders the first prompt marker in a simulated three-dimensional space, the difference between the 2D and 3D first prompt markers can be highlighted by realizing a gradual thickness change of the first prompt marker. As the imaging distance corresponding to the target imaging position increases, the thickness of the first prompt marker gradually becomes thicker. That is to say, the smaller the distance between the target imaging position and the target object corresponding to the first prompt marker, the greater the thickness of the first prompt marker in the parameters of the first prompt marker, thereby presenting the difference in thickness between the virtual images of the first prompt markers at different imaging distances. Furthermore, when displaying the transition animation of the first prompt marker moving to the end position, the gradual thickness change process of the first prompt marker in the transition animation can be displayed, so that the user can intuitively feel the transition of the first prompt marker from 2D to 3D or from 3D to 2D.

[0098] S104: Capture the image including the rendered first prompt mark by a capture device to form a virtual image including the first prompt mark, wherein the imaging position of the first prompt mark in the virtual image matches the target imaging position.

[0099] After the AR engine 232 renders the first prompt mark at the target imaging position in the simulated three-dimensional space, it captures the image including the rendered first prompt mark through the capture device, and then outputs and projects the image light corresponding to the image to the front windshield 211 through the PGU module 230, forming a virtual image including the first prompt mark in the field of view of the driver 220.

[0100] For example, the target imaging position is position S3, and the image captured by the capturer is the image presented by the 3D first prompt mark rendered at position S3. After the AR engine 232 captures the image containing the rendered first prompt mark through the capturer, it transmits the data of the image to the PGU module 230, and the image light corresponding to the image output by the PGU module 230 is projected onto the front windshield 211 to form a virtual image containing the 3D first prompt mark. The 3D first prompt mark in the virtual image matches the position S3, which can be the rear position of the front vehicle 260, such as the position of the mark 251. The 3D first prompt mark in the virtual image refers to Figure 4 Logo 401 is shown.

[0101] For example, the target imaging position is position S6, and the image captured by the capturer is the image presented by the 2D first prompt mark rendered at position S6. The AR engine 232 transmits the image data captured by the capturer to the PGU module 230, and the PGU module 230 outputs the image light corresponding to the image and projects it onto the front windshield 211, forming a virtual image including the 2D first prompt mark. The 3D first prompt mark in the virtual image matches the position S6, which can be a preset position in the first imaging layer, such as the position of the mark 241. The 2D first prompt mark in the virtual image refers to Figure 3Logo 301 is shown.

[0102] In addition, in order to prevent the first prompt mark from jumping back and forth between the AR imaging layer and the first imaging layer, it can be set that after the environmental information data meets the switching display condition of the first prompt mark and maintains a preset time, the subsequent switching display scheme of the first prompt mark in the embodiment of the present disclosure is executed. For example, when the distance between the front vehicle 260 and the vehicle 210 is greater than 100m and becomes less than 100m, after the distance between the front vehicle 260 and the vehicle 210 is less than 100m for 2 seconds, the 2D first prompt mark is switched to the 3D first prompt mark.

[0103] In the embodiment of the present disclosure, a 2D or 3D first prompt mark is rendered at a target imaging position corresponding to the position to be displayed of the first prompt mark in a simulated three-dimensional space of an imaging space simulating an AR-HUD. Different 2D or 3D states can indicate whether the function corresponding to the prompt mark is triggered. Then, an image including the 2D or 3D first prompt mark rendered at the target imaging position is captured by a capturer, thereby forming a virtual image of the 2D or 3D first prompt mark at the position to be displayed of the first prompt mark in the imaging space of the AR-HUD, thereby presenting the first prompt mark in a 2D form or a 3D form in the user's field of view. Because the first prompt mark is displayed in different states at different imaging positions and the user can see the process of state switching, the user can determine that the 2D first prompt mark and the 3D first prompt mark represent the same prompt information, and at the same time, mutually exclusive display between the 2D first prompt mark and the 3D first prompt mark is achieved, so that the user can clearly recognize the 2D mark and the 3D mark of the same information, and avoid confusion and confusion caused by the lack of association between marks of the same function.

[0104] Figure 5 Schematic diagram of the structure of the prompt identification display device provided in the embodiment of the present disclosure. The prompt identification display device may be a component or assembly in the terminal of the above embodiment. The prompt identification display device provided in the embodiment of the present disclosure may execute the processing flow provided in the prompt identification display method embodiment, such as Figure 5 As shown, the prompt mark display device 500 includes:

[0105] The monitoring module 501 is used to determine the target imaging position of the first prompt mark in the simulated three-dimensional space based on the monitored environmental information data, and a capture device for capturing images is configured in the simulated three-dimensional space;

[0106] A determination module 502 is used to determine the viewing angle information of the first prompt mark according to the target imaging position and the imaging parameters of the AR-HUD, where the viewing angle information is used to indicate the viewing angle of the capturer to view the first prompt mark, so as to form a 2D or 3D first prompt mark;

[0107] A rendering module 503, used to render the first prompt mark in the simulated three-dimensional space according to the target imaging position, the viewing angle information and the parameters of the first prompt mark;

[0108] The imaging module 504 is used to capture the image including the rendered first prompt mark through a capture device to form a virtual image including the first prompt mark, and the imaging position of the first prompt mark in the virtual image matches the target imaging position.

[0109] In some embodiments, when the monitoring module 501 determines the target imaging position of the first prompt logo in the simulated three-dimensional space based on the monitored environmental information data, it is specifically used to determine whether the switching display condition of the first prompt logo is met based on the environmental information data; if the switching display condition is met, the target object corresponding to the first prompt logo is determined according to the environmental information data, and the target imaging position of the first prompt logo in the simulated three-dimensional space is determined according to the position of the target object.

[0110] In some embodiments, the AR-HUD includes a first imaging layer and an AR imaging layer, the imaging parameters of the AR-HUD include a first imaging distance of the first imaging layer and an imaging distance interval of the AR imaging layer, and the imaging distance interval includes at least one second imaging distance;

[0111] When determining the viewing angle information of the first prompt mark based on the target imaging position and the imaging parameters of the AR-HUD, the determination module 502 is specifically used to determine that the viewing angle information of the first prompt mark is a normal viewing angle if the imaging distance of the target imaging position matches the imaging distance of the first imaging layer, so as to form a 2D first prompt mark; or, if the imaging distance of the target imaging position matches the imaging distance interval of the AR imaging layer, then determine that the viewing angle information of the first prompt mark is a set viewing angle different from the normal viewing angle, so as to form a 3D first prompt mark.

[0112] In some embodiments, when the monitoring module 501 determines the target imaging position of the first prompt marker in the simulated three-dimensional space based on the position of the target object, it is specifically used to use the position of the target object as the end point position of the first prompt marker, determine the planned path according to the current display position of the first prompt marker, and determine the target imaging position of the first prompt marker according to the planned path; or, use the position of the target object as the target imaging position of the first prompt marker.

[0113] In some embodiments, when rendering the first prompt marker in a simulated three-dimensional space according to the target imaging position, viewing angle information and parameters of the first prompt marker, the rendering module 503 is specifically used to determine the thickness of the first prompt marker in the parameters of the first prompt marker according to the distance between the target imaging position and the target object corresponding to the first prompt marker; and render the first prompt marker in the simulated three-dimensional space according to the parameters of the first prompt marker determined according to the target imaging position.

[0114] Figure 5 The prompt sign display device of the illustrated embodiment can be used to implement the technical solution of the above-mentioned method embodiment, and its implementation principle and technical effect are similar and will not be repeated here.

[0115] Figure 6 Schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure. The electronic device may be a server as described in the above embodiment. The electronic device provided in an embodiment of the present disclosure may execute the processing flow provided in the prompt identification display method embodiment, such as Figure 6 As shown, the electronic device 600 includes: a memory 601, a processor 602, a computer program and a communication interface 603; wherein the computer program is stored in the memory 601 and is configured to be executed by the processor 602 to perform the prompt identification display method as described above. In a specific embodiment, the memory 601 can be a non-volatile solid-state memory. In a specific embodiment, the memory 601 includes a read-only memory ROM (Read-Only Memory). In appropriate cases, the ROM can be a mask-programmed ROM, a programmable ROM (Programmable ROM, PROM), an erasable PROM (Electrical Programmable ROM, EPROM), an electrically erasable PROM (Electrically Erasable Programmable ROM, EEPROM), an electrically alterable ROM (Electrically Alterable ROM, EAROM) or a flash memory, or a combination of two or more of these.

[0116] In addition, the embodiment of the present disclosure further provides a computer-readable storage medium on which a computer program is stored, and the computer program is executed by a processor to implement the prompt identification display method described in the above embodiment. The storage medium can be a non-volatile / non-temporary computer-readable storage medium, for example, the non-volatile / non-temporary computer-readable storage medium can be a ROM, a random access memory (Random Access Memory, RAM), a compact disc read-only memory (Compact Disc ROM, CD-ROM), a magnetic tape, a floppy disk, and an optical data storage device.

[0117] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0118] The above description is only a specific embodiment of the present disclosure, so that those skilled in the art can understand or implement the present disclosure. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure will not be limited to the embodiments described herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A prompt mark display method, applied to AR-HUD, characterized in that: The method comprises: Based on the monitored environmental information data, determining a target imaging position of the first prompt mark in a simulated three-dimensional space, wherein a capturer for capturing images is configured in the simulated three-dimensional space; Determine, according to the target imaging position and the imaging parameters of the AR-HUD, viewing angle information of the first prompt mark, wherein the viewing angle information is used to indicate the viewing angle of the capturer to view the first prompt mark, so as to form a 2D or 3D first prompt mark; Rendering the first prompt mark in the simulated three-dimensional space according to the target imaging position, the viewing angle information and the parameters of the first prompt mark; An image including the rendered first prompt mark is captured by a capture device to form a virtual image including the first prompt mark, wherein an imaging position of the first prompt mark in the virtual image matches the target imaging position.

2. The method according to claim 1, characterized in that The step of determining a target imaging position of the first prompt mark in the simulated three-dimensional space based on the monitored environmental information data includes: Based on the environmental information data, determining whether a switching display condition of the first prompt mark is met; If the switching display condition is met, the target object corresponding to the first prompt mark is determined according to the environmental information data, and the target imaging position of the first prompt mark in the simulated three-dimensional space is determined according to the position of the target object.

3. The method according to claim 1, characterized in that The AR-HUD includes a first imaging layer and an AR imaging layer, the imaging parameters of the AR-HUD include a first imaging distance of the first imaging layer and an imaging distance interval of the AR imaging layer, and the imaging distance interval includes at least one second imaging distance; The determining the viewing angle information of the first prompt mark according to the target imaging position and the imaging parameters of the AR-HUD includes: If the imaging distance of the target imaging position matches the imaging distance of the first imaging layer, the viewing angle information of the first prompt mark is determined to be a normal viewing angle to form the first prompt mark of 2D; or, If the imaging distance of the target imaging position matches the imaging distance interval of the AR imaging layer, the viewing angle information of the first prompt mark is determined to be a set viewing angle different from the normal viewing angle, so as to form the first 3D prompt mark.

4. The method according to claim 2, characterized in that The step of determining the target imaging position of the first prompt mark in the simulated three-dimensional space according to the position of the target object includes: Taking the position of the target object as the end position of the first prompt mark, determining a planned path according to the current display position of the first prompt mark, and determining the target imaging position of the first prompt mark according to the planned path; or, The position of the target object is used as the target imaging position of the first prompt mark.

5. The method according to claim 2, characterized in that The rendering of the first prompt mark in the simulated three-dimensional space according to the target imaging position, the viewing angle information and the parameters of the first prompt mark includes: Determining a thickness of the first prompt mark in the parameters of the first prompt mark according to a distance between the target imaging position and a target object corresponding to the first prompt mark; According to the target imaging position, the first prompt mark is rendered in the simulated three-dimensional space according to the parameters for determining the first prompt mark.

6. A prompt sign display device, applied to AR-HUD, characterized in that: The device comprises: A monitoring module, used to determine a target imaging position of a first prompt mark in a simulated three-dimensional space based on monitored environmental information data, wherein a capturer for capturing images is configured in the simulated three-dimensional space; A determination module, configured to determine viewing angle information of the first prompt mark according to the target imaging position and the imaging parameters of the AR-HUD, wherein the viewing angle information is used to indicate the viewing angle of the capturer to view the first prompt mark, so as to form a 2D or 3D first prompt mark; A rendering module, configured to render the first prompt mark in the simulated three-dimensional space according to the target imaging position, the viewing angle information and the parameters of the first prompt mark; An imaging module is used to capture an image including the rendered first prompt mark through a capture device to form a virtual image including the first prompt mark, wherein an imaging position of the first prompt mark in the virtual image matches the target imaging position.

7. The device according to claim 6, characterized in that When the monitoring module determines the target imaging position of the first prompt mark in the simulated three-dimensional space based on the monitored environmental information data, the monitoring module is used to determine whether the switching display condition of the first prompt mark is met based on the environmental information data; if the switching display condition is met, the target object corresponding to the first prompt mark is determined according to the environmental information data, and the target imaging position of the first prompt mark in the simulated three-dimensional space is determined according to the position of the target object.

8. The device according to claim 6, characterized in that The AR-HUD includes a first imaging layer and an AR imaging layer, the imaging parameters of the AR-HUD include a first imaging distance of the first imaging layer and an imaging distance interval of the AR imaging layer, and the imaging distance interval includes at least one second imaging distance; When the determination module determines the viewing angle information of the first prompt mark according to the target imaging position and the imaging parameters of the AR-HUD, it is specifically used to determine that the viewing angle information of the first prompt mark is a normal viewing angle if the imaging distance of the target imaging position matches the imaging distance of the first imaging layer, so as to form the first prompt mark in 2D; or, if the imaging distance of the target imaging position matches the imaging distance interval of the AR imaging layer, the viewing angle information of the first prompt mark is determined to be a set viewing angle different from the normal viewing angle, so as to form the first prompt mark in 3D.

9. An electronic device, characterized in that: include: Memory; processor; as well as Computer programs; The computer program is stored in the memory and is configured to be executed by the processor to implement the method according to any one of claims 1 to 5.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Images