Image element display method and apparatus, electronic device, and storage medium

Abstract

An image element display method and apparatus, an electronic device, and a storage medium. The image element display method is applied to a head-up display device, and the head-up display device comprises a first image display unit for displaying an element in a three-dimensional first imaging area and a second image display unit for imaging in a two-dimensional second imaging area. The method comprises: acquiring a three-dimensional imaging element, and a two-dimensional imaging element comprising a first-type element and a second-type element (S501); on the basis of the extension of a second imaging area in a first imaging area, determining a first target imaging area of the first-type element in the first imaging area (S503); on the basis of a first image display unit, displaying the first-type element in the first target imaging area, and displaying the three-dimensional imaging element in the remaining imaging area of the first imaging area (S505); and on the basis of a second image display unit, displaying the second-type element in the second imaging area (S507). The method implements free stitching of an imaging area of the two-dimensional imaging element and an imaging area of the three-dimensional imaging element.

Description

Methods, devices, electronic equipment, and storage media for displaying image elements

[0001] This application claims priority to Chinese Patent Application No. 202410997684.0, filed on July 24, 2024, the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of head-up display technology, such as methods, apparatus, electronic devices, and storage media for displaying image elements. Background Technology

[0003] With the advancement of technology, HUD (Head-Up Display) technology has continuously developed and its functions have become increasingly rich, finding applications in areas such as car navigation and driver assistance. Among them, ARHUD (Augmented Reality Head-Up Display) technology is widely used in the automotive and aviation fields. An ARHUD typically consists of two main display areas: an AR (Augmented Reality) area and a non-AR area. Specifically, as shown in Figures 1 and 2, the AR area is used to display content that blends virtual information with the real world, such as navigation instructions, speed, and warning information, while the non-AR area is used to display static or vehicle status-related information, such as instrument readings.

[0004] As shown in Figure 3, the display requirements of the AR area are concentrated in the center and surrounding area of ​​the screen. The area of ​​the non-AR area can be appropriately increased on both sides to accommodate more elements. However, in related technologies, since the PGU (Picture Generating Unit) is usually a regular shape, it is not easy to achieve non-linear splicing between the AR area and the non-AR area, which restricts the interface design and cannot make full use of the field of view.

[0005] In related technologies, as shown in Figure 4, the field of view overlap method is used to achieve arbitrary splicing of different regions. The image generated by each PGU covers a region of a specific shape in the field of view, so that the image generated by the PGU only appears within a preset range in the overlapping area, so as to achieve non-linear splicing of AR area and non-AR area; however, this will cause a waste of PGU panel size. Summary of the Invention

[0006] To address the problems in related technologies, embodiments of this application provide a method, apparatus, electronic device, and storage medium for displaying image elements. The technical solutions are as follows:

[0007] On one hand, a method for displaying image elements is provided, the method being applied to a head-up display device, the head-up display device including a first image display unit and a second image display unit, the first image display unit being used to display elements in a first imaging area, and the second image display unit being used to image in a second imaging area, the first imaging area being a three-dimensional area, and the second imaging area being a two-dimensional area; the method includes:

[0008] Acquire the three-dimensional imaging elements and two-dimensional imaging elements to be displayed; the two-dimensional imaging elements include a first type of elements and a second type of elements;

[0009] Based on the extension of the second imaging region over the first imaging region, a first target imaging region of the first type of element in the first imaging region is determined.

[0010] Based on the first image display unit, the first type of element is displayed in the first target imaging area, and the three-dimensional imaging element is displayed in the remaining imaging area of ​​the first imaging area; the remaining imaging area is the imaging area in the first imaging area other than the first target imaging area.

[0011] The second type of element is displayed in the second imaging area based on the second image display unit.

[0012] On the other hand, an image element display device is provided, the device being applied to a head-up display device, the head-up display device including a first image display unit and a second image display unit, the first image display unit being used to display an element in a first imaging area, and the second image display unit being used to image in a second imaging area, the first imaging area being a three-dimensional area, and the second imaging area being a two-dimensional area; the device includes:

[0013] The element acquisition module is used to acquire the three-dimensional imaging elements and two-dimensional imaging elements to be displayed; the two-dimensional imaging elements include a first type of elements and a second type of elements.

[0014] A region extension module is used to determine a first target imaging region of the first type of element in the first imaging region based on the extension of the second imaging region in the first imaging region.

[0015] A first image display module is configured to display the first type of element in the first target imaging region based on the first image display unit, and to display the three-dimensional imaging element in the remaining imaging region of the first imaging region; the remaining imaging region is the imaging region in the first imaging region other than the first target imaging region.

[0016] The second image display module is used to display the second type of element in the second imaging area based on the second image display unit.

[0017] In one exemplary embodiment, the first image display module includes:

[0018] The location acquisition module is used to obtain the eye position information of the target object;

[0019] The parallax determination module is used to determine image parallax information based on the first target imaging area and the eye position information;

[0020] A parallax display module is used to display the first type of elements based on the first image display unit and the image parallax information to obtain a parallax image, so that the target object can observe the first type of elements formed in the first target imaging area through the parallax image.

[0021] In one exemplary embodiment, the first image display unit includes a first sub-display unit, and the first imaging region includes a first sub-imaging region corresponding to the first sub-display unit; the region extension module includes:

[0022] The first region determination module is used to determine the first target imaging region of the first type of element in the first sub-imaging region based on the extension of the second imaging region in the first sub-imaging region.

[0023] In one exemplary embodiment, the first sub-display unit includes a first optical element for compressing the imaging area; the first image display module includes:

[0024] The ratio determination module is used to determine the target compression ratio based on the proportion of the first target imaging region in the first sub-imaging region;

[0025] A first focal length adjustment module is used to adjust the focal length of the first optical element based on the target compression ratio;

[0026] The region compression module is used to compress the first sub-imaging region corresponding to the first imaging region of the first sub-display unit into the first target imaging region based on the adjusted first optical element.

[0027] The first element display module is used to display the first type of element based on the first sub-display unit, so that the first type of element is formed in the first target imaging area.

[0028] In one exemplary embodiment, the first image display unit further includes a second sub-display unit, and the first imaging area further includes a second sub-imaging area corresponding to the second sub-display unit; the first image display module includes:

[0029] The second element display module is used to display the three-dimensional imaging element in the second sub-imaging region of the remaining imaging region based on the second sub-display unit.

[0030] In one exemplary embodiment, the first sub-display unit includes a display screen; the second element display module includes:

[0031] The second region determination module is used to determine the second target imaging region corresponding to the three-dimensional imaging element;

[0032] The first light adjustment module is used to adjust the emission direction of the emitted light from the display screen when the second target imaging area exceeds the second sub-imaging area, so as to project the emitted light from the display screen onto the second sub-display unit.

[0033] An element classification module is used to determine the third type of elements and the fourth type of elements in the three-dimensional imaging elements; the second target imaging region corresponding to the third type of elements is outside the second sub-imaging region, and the second target imaging region corresponding to the fourth type of elements is inside the second sub-imaging region.

[0034] The third element display module is used to display the third type of element based on the display screen, so that the light emitted from the display screen is projected onto the first sub-imaging area through the second sub-display unit; and to display the fourth type of element in the second sub-imaging area based on the second sub-display unit.

[0035] In one exemplary embodiment, the apparatus further includes a first display switching module for switching the display unit to display the first type of element when the display of the third type of element ends, the first display switching module comprising:

[0036] The second light adjustment module is used to adjust the emission direction of the emitted light of the display screen in response to the end display command for the third type of element, so as to project the emitted light of the display screen onto the first sub-display unit.

[0037] The fourth element display module is used to display the first type of element based on the display screen, so that the light emitted from the display screen is projected onto the first sub-imaging area through the first sub-display unit.

[0038] In one exemplary embodiment, the apparatus further includes a switching display unit for displaying a second display switch of the first type of elements during an element switching cycle update, the second display module comprising:

[0039] A periodic update module is used to determine the element currently displayed on the display screen in response to an update of the element switching cycle;

[0040] The third light adjustment module is used to adjust the emission direction of the emitted light of the display screen when the element currently displayed on the display screen is the third type of element, so as to project the emitted light of the display screen onto the first sub-display unit;

[0041] The fifth element display module is used to display the first type of element based on the display screen, so that the light emitted from the display screen is projected onto the first sub-imaging area through the first sub-display unit.

[0042] In one exemplary embodiment, the apparatus further includes a switching display unit for displaying a third display switch of a third type of element during an element switching cycle update, the third display module comprising:

[0043] The fourth light adjustment module is used to adjust the emission direction of the emitted light of the display screen when the element currently displayed on the display screen is the first type of element, so as to project the emitted light of the display screen onto the second sub-display unit;

[0044] The sixth element display module is used to display the third type of element based on the display screen, so that the light emitted from the display screen is projected onto the first sub-imaging area through the second sub-display unit.

[0045] In an exemplary embodiment, the first sub-display unit further includes a second optical element disposed between the display screen and the first sub-imaging area, the focal length of the second optical element including a first focal length, the first focal length being used to project the emitted light from the display screen onto the second sub-display unit; the first light adjustment module or the fourth light adjustment module includes:

[0046] The second focal length adjustment module is used to adjust the focal length of the second optical element to the first focal length, so that the light emitted from the display screen is projected onto the second sub-display unit through the second optical element.

[0047] In one exemplary embodiment, the focal length of the second optical element includes a second focal length used to project the emitted light from the display screen onto the first sub-display unit; the second light adjustment module or the third light adjustment module includes:

[0048] The third focal length adjustment module is used to adjust the focal length of the second optical element to the second focal length, so that the light emitted from the display screen is projected onto the first sub-display unit through the second optical element.

[0049] On the other hand, an electronic device is provided, including a processor and a memory, wherein the memory stores at least one instruction or at least one program, the at least one instruction or the at least one program being loaded and executed by the processor to implement the method for displaying image elements in any of the above aspects.

[0050] On the other hand, a computer-readable storage medium is provided, wherein at least one instruction or at least one program is stored therein, the at least one instruction or the at least one program being loaded and executed by a processor to implement a method for displaying image elements as described above.

[0051] On the other hand, a computer program product or computer program is provided, which includes computer instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the electronic device to perform the method for displaying image elements according to any of the above aspects.

[0052] This application embodiment acquires three-dimensional and two-dimensional imaging elements to be displayed. The two-dimensional imaging elements include a first type of element and a second type of element. A first image display unit displays the first type of element based on a second imaging region within an extended area of ​​the first imaging region, and displays the three-dimensional imaging element based on the remaining imaging region of the first imaging region. A second image display unit displays the second type of element based on the second imaging region. The first imaging region is a three-dimensional region, and the second imaging region is a two-dimensional region. This allows for the free splicing of the imaging regions of the two-dimensional and three-dimensional imaging elements. Based on this, the two-dimensional imaging element can be displayed using the first imaging region at the edge of the field of view, achieving the expansion of the corresponding display area of ​​the two-dimensional imaging element without introducing additional hardware costs. The first and second types of elements seen by the user are on the same plane, eliminating the need for eye zoom adjustment when observing the two-dimensional imaging element, thus enhancing the user experience. Attached Figure Description

[0053] Figure 1 is a schematic diagram of the first type of ARHUD region division provided in the embodiment of this application;

[0054] Figure 2 is a schematic diagram of the second type of ARHUD area division provided in the embodiments of this application;

[0055] Figure 3 is a schematic diagram of the third type of ARHUD region division provided in the embodiments of this application;

[0056] Figure 4 is a schematic diagram of the correspondence between display area allocation and PGU provided in an embodiment of this application;

[0057] Figure 5 is a flowchart illustrating a method for displaying image elements according to an embodiment of this application;

[0058] Figure 6 is a schematic diagram of the fourth type of ARHUD region division provided in the embodiments of this application;

[0059] Figure 7 is a schematic diagram of the fifth type of ARHUD region division provided in the embodiments of this application;

[0060] Figure 8 is a schematic diagram of the optical principle of the first head-up display device provided in the embodiment of this application;

[0061] Figure 9 is a schematic diagram of the optical principle of a second head-up display device provided in an embodiment of this application;

[0062] Figure 10 is a schematic diagram of the sixth type of ARHUD region division provided in the embodiments of this application;

[0063] Figure 11 is a schematic diagram of the optical principle of a third head-up display device provided in an embodiment of this application;

[0064] Figure 12 is a schematic diagram of the optical principle of the fourth head-up display device provided in the embodiment of this application;

[0065] Figure 13 is a structural block diagram of an image element display device provided in an embodiment of this application;

[0066] Figure 14 is a hardware structure block diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0067] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings.

[0068] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or server that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0069] It is understood that in the specific embodiments of this application, data such as user information are involved. When the above embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0070] Please refer to Figure 5, which shows a flowchart illustrating a method for displaying image elements according to an embodiment of this application. It should be noted that this specification provides method operation steps as shown in the embodiments or flowcharts, but based on conventional or non-inventive methods, more or fewer operation steps may be included. The order of steps listed in the embodiments is merely one possible execution order among many steps and does not represent the only possible execution order. In actual system or product execution, the methods can be executed sequentially according to the embodiments or accompanying drawings, or in parallel (e.g., in a parallel processor or multi-threaded processing environment).

[0071] Specifically, this method is applied to a head-up display (PGU), which includes a first image display unit and a second image display unit. The first image display unit displays elements in a first imaging area, and the second image display unit images in a second imaging area. The first imaging area is a three-dimensional area, and the second imaging area is a two-dimensional area. Specifically, the first image display unit forms a parallax image on a two-dimensional plane in the first imaging area. Different two-dimensional planes in the first imaging area indicate different virtual image distances, i.e., the distance between the user's eye and the virtual image. More specifically, the first image display unit is applied to ARHUD technology, and the second image display unit is applied to WHUD (Windshield Head-Up Display) technology. Both the first and second image display units include at least a light source, an image generator, optical elements, and control circuitry. The light source is one of the core components of the PGU, responsible for generating the light beam. The light source can be an LED (Light-Emitting Diode), a laser, or other types of light sources. The image generator is responsible for converting electronic signals into visible images. This typically involves a microdisplay, such as a DLP (Digital Light Processing) chip, an LCOS (Liquid Crystal On Silicon) display, an OLED (Organic Light-Emitting Diode) panel, or an LCD (Liquid Crystal Display), which can generate high-resolution, high-brightness images. Optical elements, such as lenses, mirrors, and diffraction gratings, are used to correctly magnify and project the image onto the imaging area, specifically adjusting the image size, focus, and position to ensure a clear and stable display in front of the driver's line of sight. The control circuit manages the operation of the light source and image generator, receiving data from the vehicle's ECU (Electronic Control Unit), such as speed and navigation information, and converting it into image signals. In practice, in a driving scenario, the first imaging area typically refers to the three-dimensional space in front of the windshield, and the second imaging area typically refers to the two-dimensional space in front of the windshield. As shown in Figure 5, the method may include:

[0072] S501, acquire the 3D imaging elements and 2D imaging elements to be displayed.

[0073] Among them, the three-dimensional imaging elements indicate dynamic information directly related to the external environment.

[0074] The two-dimensional imaging elements indicate static or vehicle-state-related information. Specifically, the two-dimensional imaging elements include a first type of element and a second type of element. The first type of element refers to the two-dimensional imaging elements that need to be displayed in the first imaging area, and the second type of element refers to the two-dimensional imaging elements that need to be displayed in the second imaging area. For example, as shown in Figure 6, the AR area in the figure is the first imaging area, the static elements displayed in the AR area are the first type of elements, the other elements displayed in the AR area are the three-dimensional imaging elements, and steering information, speed limit information, etc. are the second type of elements. In specific implementations, which elements in the two-dimensional imaging elements are determined to be the first type of elements and which are determined to be the second type of elements can be randomly assigned by the HUD system, pre-specified by the HUD system, or set by the user based on the HUD system as needed.

[0075] In practice, in driving scenarios, the three-dimensional and two-dimensional imaging elements to be displayed are usually derived from data from the vehicle's ECU.

[0076] S503, based on the extension of the second imaging region into the first imaging region, determine the first target imaging region of the first type of element in the first imaging region.

[0077] In this process, the first target imaging region is located in the same plane as the second imaging region within the first imaging region. Specifically, the first imaging region is in three-dimensional space, and the second imaging region is in two-dimensional space. The second imaging region is extended within the plane of the first imaging region, and this extended region is denoted as the extended region. The extended region intersects with the first imaging region. The imaging region of the first type of element is selected from the intersection region of the extended region and the first imaging region to obtain the first target imaging region.

[0078] In practice, the display requirements of three-dimensional imaging elements are concentrated in the center or near the center of the first imaging area. Therefore, in the horizontal direction, the first target imaging area can be determined near the two side edges of the first imaging area.

[0079] In practice, to avoid the two-dimensional imaging elements being displayed in a scattered manner and affecting the user experience, in the vertical direction, the first target imaging area is usually located close to the second imaging area in the first imaging area, so that the imaging area of ​​the two-dimensional imaging element appears as a whole.

[0080] In practice, determining the first target imaging area also requires considering factors such as the imaging size of the first type of element and image parallax. Specifically, after determining the imaging size of the first type of element, the position and size of the first target imaging area are adjusted based on the parallax information to ensure that the first type of element can be completely captured by the user's eye.

[0081] In an exemplary embodiment, the first image display unit includes a first sub-display unit, and the first imaging region includes a first sub-imaging region corresponding to the first sub-display unit; step S503 above may include the following steps:

[0082] Based on the extension of the second imaging region into the first sub-imaging region, the first target imaging region of the first type of element in the first sub-imaging region is determined.

[0083] Specifically, a first sub-imaging region is delineated in the first imaging region. For example, referring to Figures 7 and 8, the AR region is divided into a side region (i.e., the first sub-imaging region) and a main region. Correspondingly, pixels corresponding to the first sub-imaging region are delineated from the pixels of the PGU of the first image display unit as the PGU of the first sub-display unit. For optical elements, the first sub-display unit can share all or part of the optical elements with the first image display unit. Alternatively, optical elements can be set separately for the first sub-display unit. Figure 8 adopts a partially shared scheme.

[0084] In this process, the first target imaging region is located in the same plane as the second imaging region within the first sub-imaging region. Specifically, the first sub-imaging region is in three-dimensional space, and the second imaging region is in two-dimensional space. The second imaging region is extended within the plane of the first sub-imaging region, and this extended region is denoted as the extended region. The extended region intersects with the first sub-imaging region. The imaging region of the first type of element is selected from the intersection region of the extended region and the first sub-imaging region to obtain the first target imaging region.

[0085] As can be seen from the above technical solutions of the embodiments of this application, the embodiments of this application delineate a first sub-imaging region in the first imaging region, and correspondingly delineate the pixels corresponding to the first sub-imaging region in the pixels of the PGU of the first image display unit as the PGU of the first sub-display unit. Based on the first sub-display unit, the first type of element in the two-dimensional imaging element is displayed in the extended area of ​​the second imaging region in the first sub-imaging region, thereby realizing the free splicing of the imaging region of the two-dimensional imaging element and the imaging region of the three-dimensional imaging element.

[0086] S505, based on the first image display unit, displays a first type of element in the first target imaging area, and displays three-dimensional imaging elements in the remaining imaging area of ​​the first imaging area.

[0087] The remaining imaging area is the imaging area in the first imaging area other than the first target imaging area.

[0088] Specifically, the imaging positions of the 3D imaging elements correspond to the real-world scene. The HUD system integrates sensors such as cameras, radar, and lidar. These sensors capture real-time information about the vehicle's surrounding environment, including road conditions, other vehicles, pedestrians, and traffic signs. Based on sensor data, the system constructs a real-time environmental model, identifying important reference points and features, such as lane lines, curbs, and traffic lights. Vehicle status and motion prediction are also required. Specifically, dynamic data such as vehicle speed, acceleration, and steering angle are acquired, along with position information provided by GPS (Global Positioning System) and IMU (Inertial Measurement Unit) to understand the vehicle's current state and future trajectory. Combining the vehicle status and environmental model, the system predicts the vehicle's path in the next few seconds and the optimal position and angle for the virtual information to be displayed. The vehicle coordinate system is aligned with the world coordinate system to ensure the relative position of the virtual image corresponding to the 3D imaging elements is accurate. Considering the curvature of the windshield and the driver's perspective, perspective correction is applied to the virtual image to make it appear as if it were projected from a distance. As vehicles move and the environment changes, the system continuously updates the position and size of the virtual images to ensure that the information always matches the real-world scene.

[0089] Specifically, the first image display unit includes at least a backlight, an image source (image generation unit PGU), and optical elements. The image source generates the first type of elements and three-dimensional imaging elements to be displayed. This is typically accomplished by a microdisplay (such as LCD, OLED, or LCOS), which can generate high-resolution images. The backlight provides sufficient light to the image source, ensuring a bright and high-contrast image. The backlight can be an LED light source, or a diffuser can be used to evenly distribute the light, avoiding hot spots or dark areas. The optical elements can include plane mirrors and curved mirrors. The main function of the plane mirror is to change the direction of the light path, guiding the light emitted from the image source to the curved mirror. The plane mirror also helps adjust the image position to better suit the user's viewing angle. The curved mirror (usually a concave mirror) is used to focus and magnify the image generated by the image source, while projecting it onto the first imaging area. It can not only adjust the image size but also correct distortion caused by the curvature of the windshield, ensuring that the user sees a clear, distortion-free image.

[0090] Specifically, the image source generates the image elements to be displayed based on the input data. The image elements include first-class elements and three-dimensional imaging elements. In driving scenarios, the input data of the image source is usually provided by the vehicle ECU. The backlight provides the necessary illumination. The generated image light is reflected by the plane mirror and reaches the curved mirror. The curved mirror focuses and magnifies the light through its curved surface characteristics, while correcting distortion. The processed light passes through the windshield to form a virtual image in front of the user, and the user can view this information without shifting their gaze.

[0091] In an exemplary embodiment, the process of displaying a first type of element in the first target imaging area based on the first image display unit in step S505 above may include the following steps:

[0092] Obtain the eye position information of the target object;

[0093] Based on the imaging area of ​​the first target and the eye position information, determine the image parallax information;

[0094] Based on the first image display unit and image parallax information, a first type of element is displayed to obtain a parallax image, so that the target object can observe the first type of element formed in the first target imaging area through the parallax image.

[0095] The target audience is the user currently using the head-up display device. Specifically, in a driving scenario, the target audience is typically the driver.

[0096] Among them, eye position information is mainly represented by the coordinates of the target object's eyes.

[0097] The first target imaging region is the imaging region corresponding to the first type of element, which aims to enable the target object to observe the first type of element in the first target imaging region.

[0098] The parallax image includes a left-eye virtual image for the target's left eye and a right-eye virtual image for the target's right eye, with a parallax between the two images. Specifically, due to the parallax, the target's brain automatically fuses the binocular perspectives, thus producing stereoscopic vision with a sense of depth.

[0099] The image parallax information indicates the distance between the virtual images of the left and right eyes, and may also include the positions of the left and right virtual images. Specifically, based on eye position information, the distance between the centers of the pupils of both eyes is determined, as is the distance from the eye to the virtual image of the left or right eye. The distance from the eye to the first target imaging area is calculated. Based on the principle of similar triangles, the distance between the virtual images of the left and right eyes can be calculated. Using the obtained parallax value, a parallax image is generated for each pixel. Specifically, each pixel in the original image is offset horizontally by a certain number of pixels, determined by the parallax value. For the left and right eye images, the offset directions are opposite. Combined with the eye position information, the positions of the virtual images of the left and right eyes can be determined. In a head-up display device, the images with appropriate parallax seen by the left and right eyes are displayed separately, allowing the brain to synthesize a stereoscopic image with a sense of depth. In practice, the PGU fuses the images from the left and right eyes, displaying only one fused image. The outgoing light passes through a beam splitter, causing a portion of the fused image to enter the left eye and a portion to enter the right eye, thus allowing the left and right eyes to see different images. Alternatively, a time-division multiplexing design can be employed, allowing the PGU to alternately display the left and right eye images. Simultaneously, MEMS (Micro-Electro-Mechanical Systems) technology can be used to ensure that the left-eye image enters the left eye and the right-eye image enters the right eye.

[0100] Specifically, image parallax information also includes the positions of the left-eye and right-eye images displayed by the PGU (Power Placement Unit) of the first image display unit. In practice, the HUD system is calibrated to accurately understand the characteristics of optical components (such as lenses and mirrors) and their relative positions to the vehicle's windshield and the driver's eyes. This step is typically completed during the system design phase and may require periodic fine-tuning to adapt to different driving environments and driver positions. A three-dimensional coordinate system is established in the HUD system. The origin of the coordinate system is usually chosen at the midpoint of the line connecting the driver's eyes, and the direction of the coordinate system is defined based on the vehicle's orientation and the direction of gravity. Using the known positions of the left-eye and right-eye virtual images, they can be converted into three-dimensional coordinates in the aforementioned coordinate system. In practice, factors such as the distance and angle of the virtual images relative to the driver's eyes, as well as the refraction effect of the windshield, also need to be considered. The three-dimensional coordinates of the virtual images are then mapped back to two-dimensional coordinates on the PGU. Specifically, starting from the position of the virtual image, passing through the windshield and the optical components of the HUD, until they intersect on the PGU, the positions of the left-eye and right-eye images are obtained.

[0101] As can be seen from the above technical solutions of the embodiments of this application, the embodiments of this application determine the image parallax information by using the first target imaging area corresponding to the first type of element and the eye position information of the target object, and then display the first type of element based on the first image display unit according to the image parallax information to obtain a parallax image, so that the target object can observe the first type of element formed in the first target imaging area through the parallax image. The first type of element formed and the second type of element formed in the second imaging area are on the same plane. When the user observes the two-dimensional imaging element, there is no need to adjust the focus of the eye, which enhances the user experience.

[0102] In an exemplary embodiment, the first sub-display unit includes a first optical element for compressing the imaging area; the process of displaying the first type of element in the first target imaging area based on the first image display unit in step S505 may further include the following steps:

[0103] The target compression ratio is determined based on the proportion of the first target imaging region in the first sub-imaging region;

[0104] Adjust the focal length of the first optical element based on the target compression ratio;

[0105] Based on the adjusted first optical element, the first sub-imaging area corresponding to the first imaging area of ​​the first sub-display unit is compressed into the first target imaging area.

[0106] The first type of elements are displayed based on the first sub-display unit, so that the first type of elements are formed in the first target imaging area.

[0107] Specifically, because the first imaging area needs to be covered by a grating to achieve parallax display, meaning the PGU needs to generate both left-eye and right-eye images simultaneously, the resolution of the first imaging area will be half that of the second imaging area when using PGUs of the same specifications. This will result in a significant visual difference between the two-dimensional imaging elements displayed in the first imaging area and those displayed in the second imaging area, affecting the user experience. Using a PGU with a larger panel size or better display parameters to display the content of the first imaging area can solve this problem, but it will increase the cost and the size of the HUD.

[0108] In specific implementation, the first optical element can be set as a curved mirror, a curved lens or other functional optical element. For example, as shown in Figure 9, the plane mirror 2 in Figure 8 is replaced with a concave mirror as the first optical element. When the image generated by the light source is projected onto the concave mirror, the concave mirror focuses the light and reduces the imaging area corresponding to the first sub-display unit.

[0109] The target compression ratio is the ratio of the area of ​​the first target imaging region to the area of ​​the first sub-imaging region, and it is also the resolution enhancement factor of the virtual image formed by the first type of elements. In specific implementations, the target compression ratio can be set to a fixed value, such as 1 / 2, and correspondingly, the focal length of the first optical element is fixed. In specific implementations, when the target compression ratio is an indefinite value, the focal length of the first optical element is adjustable. For example, by using the technology of deformable mirrors, the shape of the first optical element is changed by applying an external force (such as electromagnetic force), thereby dynamically adjusting the focal length. In this case, the precision requirements for electromagnetic adjustment are relatively high.

[0110] Specifically, based on the target compression ratio and other imaging requirements, such as the size, position, and sharpness of the virtual image, and by analyzing the optical path design of the HUD system, specifically the relative positions and distances between the light source, multiple optical elements, and the first sub-imaging area, and combining the imaging formula of the first optical element, the focal length of the first optical element can be calculated, and the focal length of the first optical element can be adjusted to the calculated focal length value.

[0111] Specifically, as shown in Figure 10, since the first sub-imaging area is compressed into the first target imaging area, the imaging area corresponding to the first imaging area of ​​the first sub-display unit is the first target imaging area. In order to make the first type of element form in the first target imaging area without changing the size of the virtual image corresponding to the first type of element, all pixels of the PGU of the first sub-display unit are used to display the first type of element.

[0112] As can be seen from the above technical solutions of the embodiments of this application, the embodiments of this application provide a first optical element for compressing the imaging area in the first sub-display unit. By adjusting the focal length of the first optical element, the first sub-imaging area corresponding to the first imaging area of ​​the first sub-display unit is compressed to the first target imaging area, so that the imaging area corresponding to the first sub-display unit is the first target imaging area. In order to form the first type of element in the first target imaging area without changing the size of the virtual image corresponding to the first type of element, the first type of element is displayed based on all pixels of the PGU of the first sub-display unit, thereby improving the resolution of the virtual image formed by the first type of element and overcoming the resolution difference between the first imaging area and the second imaging area.

[0113] In an exemplary embodiment, the first image display unit further includes a second sub-display unit, and the first imaging area further includes a second sub-imaging area corresponding to the second sub-display unit; the process of displaying three-dimensional imaging elements in the remaining imaging area of ​​the first imaging area in step S505 above may include the following steps:

[0114] Based on the second sub-display unit, three-dimensional imaging elements are displayed in the second sub-imaging region of the remaining imaging region.

[0115] Specifically, a second sub-imaging region is delineated in the first imaging region. For example, referring to Figures 7 and 8, the AR region is divided into a side region and a main region (i.e., the second sub-imaging region). Correspondingly, pixels corresponding to the second sub-imaging region are delineated from the pixels of the PGU of the first image display unit as the PGU of the second sub-display unit. For optical elements, the second sub-display unit can share all or part of the optical elements with the first sub-display unit. Alternatively, optical elements can be set separately for the second sub-display unit. Figure 8 adopts a partially shared scheme.

[0116] The remaining imaging region includes the second sub-imaging region and the imaging region in the first sub-imaging region excluding the first target imaging region.

[0117] As can be seen from the above technical solutions of the embodiments of this application, the embodiments of this application delineate a second sub-imaging region in the first imaging region, and correspondingly delineate the pixels corresponding to the second sub-imaging region in the pixels of the PGU of the first image display unit as the PGU of the second sub-display unit. Based on the second sub-display unit, three-dimensional imaging elements are displayed in the second sub-imaging region, and the first type of elements in the two-dimensional imaging elements are displayed in the first sub-imaging region, so that the first image display unit can display two-dimensional imaging elements and three-dimensional imaging elements at the same time.

[0118] In one exemplary embodiment, the first sub-display unit includes a display screen; the step of displaying three-dimensional imaging elements in a second sub-imaging region of the remaining imaging region based on the second sub-display unit may include the following steps:

[0119] Determine the second target imaging region corresponding to the three-dimensional imaging element;

[0120] When the second target imaging area exceeds the second sub-imaging area, the emission direction of the emitted light from the display screen is adjusted so as to project the emitted light from the display screen onto the second sub-display unit.

[0121] Identify the third and fourth types of elements in the three-dimensional imaging elements; the second target imaging region corresponding to the third type of element is outside the second sub-imaging region, and the second target imaging region corresponding to the fourth type of element is inside the second sub-imaging region.

[0122] The third type of element is displayed on the display screen so that the light emitted from the display screen is projected onto the first sub-imaging area through the second sub-display unit; and the fourth type of element is displayed in the second sub-imaging area based on the second sub-display unit.

[0123] The display screen serves as the image source for the first sub-display unit, and includes, but is not limited to, LCD, OLED, and LCOS.

[0124] Specifically, the imaging position of the 3D imaging element corresponds to the real scene. Therefore, the second target imaging area changes with the display scene. Within the range of the first imaging area, the second target imaging area may exceed the second sub-imaging area. At this time, the 3D imaging element that exceeds the first sub-imaging area can be temporarily displayed.

[0125] Specifically, referring to Figure 11, the display screen of the first sub-display unit corresponds to the side area pixels in the figure. The emitted light rays after the side area pixels are lit are shown by the solid lines in the figure. They are used to display the first type of elements in the two-dimensional imaging elements under the action of the first sub-display unit. By adjusting the emission direction of the emitted light rays to the second sub-display unit, the emission direction shown by the dashed lines corresponding to the side area pixels in the figure is formed. This allows the side area pixels to be used for imaging some three-dimensional imaging elements in the first sub-imaging area under the action of the second sub-display unit.

[0126] Specifically, based on the display screen of the first sub-display unit and the optical elements of the second sub-display unit, a third type of element in the three-dimensional imaging elements is displayed in the first sub-imaging area, and a fourth type of element in the three-dimensional imaging elements is displayed in the second sub-imaging area based on the second sub-display unit.

[0127] As can be seen from the above technical solutions of the embodiments of this application, when the second target imaging area corresponding to the three-dimensional imaging element exceeds the second sub-imaging area, the emission direction of the emitted light of the display screen of the first sub-display unit is adjusted, and the three-dimensional imaging element exceeding the second sub-imaging area is temporarily displayed using the first sub-imaging area used to display the first type of element, thereby avoiding the display conflict between the three-dimensional imaging element and the two-dimensional imaging element.

[0128] In one exemplary implementation, the following steps may also be included:

[0129] In response to the end display command for the third type of element, the emission direction of the emitted light from the display screen is adjusted so as to project the emitted light from the display screen onto the first sub-display unit;

[0130] The first type of element is displayed on the display screen, so that the light emitted from the display screen is projected onto the first sub-imaging area through the first sub-display unit.

[0131] The end-of-display command is typically issued by the vehicle's ECU. Specifically, the vehicle ECU determines whether the display of the third type of element can be stopped. If it determines that the display of the third type of element can be stopped, the HUD system issues an end-of-display command for the third type of element.

[0132] In practice, the ECU continuously collects data from vehicle sensors, GPS modules, cameras, etc. This data includes the vehicle's current status (such as speed, direction, engine status), surrounding environment (such as traffic conditions, weather conditions), and navigation information. The ECU analyzes the collected data to determine whether the third-category elements still need to be displayed. The ECU also assesses the impact of displaying or hiding specific information on driving safety. Once the ECU determines that the display of the third-category elements can be stopped, it sends a command to the HUD system via the vehicle network (such as the CAN bus) to instruct the HUD system to stop displaying these elements. This command may include the specific element ID or type, as well as a timestamp or condition for ending the display. Upon receiving the command from the ECU, the HUD system immediately performs the corresponding operation and updates the displayed content. This may involve removing specific icons, text, or animations from the display screen.

[0133] Specifically, when determining the end of displaying the third type of element, referring to Figure 11, the display screen of the first sub-display unit corresponds to the side area pixel in the figure. When the side area pixel is used to display the third type of element, the emitted light after the side area pixel is lit is as shown by the dashed line corresponding to the side area pixel in the figure. It is used to display the third type of element in the three-dimensional imaging elements under the action of the second sub-display unit. By adjusting the emission direction of the emitted light to the first sub-display unit, the emission direction shown by the solid line in the figure is formed, so that the side area pixel is used to image the first type of element in the two-dimensional imaging elements in the first sub-imaging area under the action of the first sub-display unit.

[0134] As can be seen from the above technical solutions of the embodiments of this application, the embodiments of this application adjust the emission direction of the emitted light of the display screen of the first sub-display unit when the third type of element ends to make the first sub-imaging area continue to be used to display the first type of element, thereby avoiding the display conflict between the three-dimensional imaging element and the two-dimensional imaging element.

[0135] In one exemplary implementation, the following steps may also be included:

[0136] In response to updates in the element switching cycle, determine the element currently displayed on the screen;

[0137] When the element currently displayed on the screen is a third type of element, adjust the emission direction of the emitted light from the screen to project the emitted light from the screen onto the first sub-display unit;

[0138] The first type of element is displayed on the display screen, so that the light emitted from the display screen is projected onto the first sub-imaging area through the first sub-display unit.

[0139] The element switching cycle is used to switch between displaying the cycles of the first type of elements and the third type of elements in the first sub-imaging area.

[0140] Specifically, during the element switching cycle update, if the first sub-imaging area is currently displaying a third type of element, it is switched to display a first type of element. Referring to Figure 11, the display screen of the first sub-display unit corresponds to the side area pixel in the figure. When the side area pixel is used to display a third type of element, the emitted light after the side area pixel is lit is as shown by the dashed line corresponding to the side area pixel in the figure. It is used to display the third type of element in the three-dimensional imaging elements under the action of the second sub-display unit. By adjusting the emission direction of the emitted light to the first sub-display unit, the emission direction shown by the solid line in the figure is formed, so that the side area pixel is used to image the first type of element in the two-dimensional imaging elements in the first sub-imaging area under the action of the first sub-display unit.

[0141] As can be seen from the above technical solutions of the embodiments of this application, when the element switching cycle is updated, if the first sub-imaging area is currently displaying a third type of element, the emission direction of the emitted light from the display screen of the first sub-display unit is adjusted so that the first sub-imaging area is used to display a first type of element, thus avoiding the display conflict between three-dimensional imaging elements and two-dimensional imaging elements.

[0142] In one exemplary implementation, the following steps may also be included:

[0143] When the element currently displayed on the screen is a first type of element, adjust the emission direction of the emitted light from the screen to project the emitted light from the screen onto the second sub-display unit;

[0144] The third type of element is displayed on the display screen so that the light emitted from the display screen is projected onto the first sub-imaging area through the second sub-display unit.

[0145] Specifically, during the element switching cycle update, if the first sub-imaging area is currently displaying a first type of element, it is switched to a third type of element. Referring to Figure 11, the display screen of the first sub-display unit corresponds to the side area pixel in the figure. When the side area pixel is used to display a first type of element, the emitted light after the side area pixel is lit is shown by the solid line in the figure. It is used to display the first type of element in the two-dimensional imaging elements under the action of the first sub-display unit. By adjusting the emission direction of the emitted light to the second sub-display unit, the emission direction shown by the dashed line corresponding to the side area pixel in the figure is formed, so that the side area pixel is used for imaging the third type of element in the first sub-imaging area under the action of the second sub-display unit.

[0146] As can be seen from the above technical solutions of the embodiments of this application, when the element switching cycle is updated, if the first sub-imaging area is currently displaying a first type of element, the emission direction of the emitted light from the display screen of the first sub-display unit is adjusted so that the first sub-imaging area is used to display a third type of element, thus avoiding the display conflict between three-dimensional imaging elements and two-dimensional imaging elements.

[0147] In an exemplary embodiment, the first sub-display unit further includes a second optical element disposed between the display screen and the first sub-imaging area. The focal length of the second optical element includes a first focal length, which is used to project the emitted light from the display screen onto the second sub-display unit. The step of adjusting the emission direction of the emitted light from the display screen to project the emitted light from the display screen onto the second sub-display unit may include the following steps:

[0148] The focal length of the second optical element is adjusted to the first focal length so that the light emitted from the display screen is projected onto the second sub-display unit through the second optical element.

[0149] The second optical element is a zoomable optical element, such as a zoom lens, used to change the direction of the light emitted from the display screen, thereby switching the optical path. Specifically, the second optical element is positioned between the display screen and other optical elements in the HUD system; that is, the light emitted from the display screen must pass through the second optical element before it can be projected onto the other optical elements of the HUD system.

[0150] In specific implementation, referring to Figure 12, the second optical element is set as an electro-optic lens. The zoom of the electro-optic lens depends on the electro-optic effect, which uses an electric field to change its optical properties. That is, the refractive index of certain materials will change under the action of an electric field. Specifically, an electric field is applied to both sides of the lens, and the intensity of the electric field is controlled by a voltage source. The presence of the electric field causes the refractive index of the lens material (such as liquid crystal or electro-optic polymer) to change. Since the refractive index is proportional to the electric field intensity, the degree of change of the refractive index can be controlled by adjusting the voltage. The change of the refractive index directly affects the degree of refraction of light when it passes through the lens, thereby changing the focusing ability of the lens. Therefore, by controlling the electric field intensity, the zoom effect can be achieved without moving the physical position of the lens.

[0151] In practice, the angle of the electro-zoom lens can be adjusted to the first focal length by adjusting the electric field strength, based on the relationship between electric field strength and focal length.

[0152] Specifically, when the focal length of the second optical element is the first focal length, under the action of the second optical element, the light emitted from the display screen of the first sub-display unit is projected onto the second sub-display unit, so that the display screen, under the action of the second sub-display unit, is used for imaging the third type of element in the first sub-imaging area.

[0153] As can be seen from the above technical solutions of the embodiments of this application, by setting a second optical element in the first sub-display unit, when the third type of element needs to be displayed in the first sub-imaging area, it is only necessary to adjust the focal length of the second optical element to the first focal length, thereby reducing the directional output requirements of the display screen.

[0154] In one exemplary embodiment, the focal length of the second optical element includes a second focal length used to project the emitted light from the display screen onto the first sub-display unit; the step of adjusting the emission direction of the emitted light from the display screen to project the emitted light from the display screen onto the first sub-display unit may include the following steps:

[0155] The focal length of the second optical element is adjusted to the second focal length so that the light emitted from the display screen is projected onto the first sub-display unit through the second optical element.

[0156] In a specific implementation, the second optical element can be set as an electro-zoom lens. Based on the correspondence between electric field strength and focal length, the angle of the electro-zoom lens can be adjusted to the second focal length by adjusting the electric field strength.

[0157] Specifically, when the focal length of the second optical element is the second focal length, under the action of the second optical element, the light emitted from the display screen of the first sub-display unit is projected onto the first sub-display unit according to a predetermined path, so that the display screen is used for imaging the first type of element in the first sub-imaging area.

[0158] As can be seen from the above technical solutions of the embodiments of this application, by setting a second optical element in the first sub-display unit, when the first type of element needs to be displayed in the first sub-imaging area, only the focal length of the second optical element needs to be adjusted to the second focal length, thereby reducing the directional output requirements of the display screen.

[0159] S507, based on the second image display unit, displays a second type of element in the second imaging area.

[0160] The second image display unit includes at least a backlight, an image source (image generation unit, PGU), and optical elements. Specifically, the PGU is responsible for generating the second type of elements to be displayed, and can employ different technologies, such as TFT-LCD (Thin Film Transistor Liquid Crystal Display), LCOS, DLP, or OLED and LED technologies. In detail, the PGU generates corresponding image elements based on input data from the vehicle's ECU, such as vehicle speed, navigation instructions, and warning information. The backlight provides the necessary illumination for the PGU to ensure that the second type of elements are clearly visible under various lighting conditions. Optical components include a series of lenses, mirrors, and flat or curved mirrors, which amplify and project the second type of element generated by the PGU onto the windshield. Specifically, optical components mainly include condenser lenses, mirrors, combiners, or projectors. The condenser lens is located between the PGU and the mirror to collect and focus the light emitted from the PGU to ensure image clarity. The mirror guides the image from the PGU to the windshield. Multi-stage mirrors can be used to optimize the image path and size. The combiner or projector is usually part of the windshield and is specially treated to reflect the image while allowing the driver to see the external environment through it. The combiner can be flat, but in practice, a curved design is usually used to reduce distortion and improve image quality.

[0161] Specifically, the PGU generates an image indicating the second type of element based on vehicle data. The PGU is backlit to make the image visible, and optical elements magnify the image and guide it to the windshield. The combiner on the windshield reflects the image, making it appear to be suspended a few meters in front of the vehicle. The driver sees a virtual image superimposed on the actual road view through the windshield.

[0162] As can be seen from the above technical solutions of the embodiments of this application, the embodiments of this application display the first type of element in the two-dimensional imaging elements by displaying the extended area of ​​the second imaging area in the first imaging area, thereby realizing the free splicing of the imaging area of ​​the two-dimensional imaging elements and the imaging area of ​​the three-dimensional imaging elements. Based on this, the first imaging area at the edge of the field of view can be used to display the two-dimensional imaging elements. The expansion of the corresponding display area of ​​the two-dimensional imaging elements can be achieved without introducing additional hardware costs. The first type of element and the second type of element seen by the user are in the same plane, so that the user does not need to adjust the focus of the eye when observing the two-dimensional imaging elements, thus enhancing the user experience.

[0163] Corresponding to the image element display methods provided in the above embodiments, this application also provides an image element display device. Since the image element display device provided in this application corresponds to the image element display methods provided in the above embodiments, the implementation methods of the aforementioned image element display methods are also applicable to the image element display device provided in this embodiment, and will not be described in detail in this embodiment.

[0164] Please refer to Figure 13, which shows a schematic diagram of an image element display device provided in an embodiment of this application. This device has the function of implementing the image element display method in the above-described method embodiments. This function can be implemented by hardware or by hardware executing corresponding software. Specifically, this device is applied to a head-up display device, which includes a first image display unit and a second image display unit. The first image display unit is used to display elements in a first imaging area, and the second image display unit is used to image in a second imaging area. The first imaging area is a three-dimensional area, and the second imaging area is a two-dimensional area. As shown in Figure 13, the device may include:

[0165] The element acquisition module 1310 is used to acquire the three-dimensional imaging elements and two-dimensional imaging elements to be displayed; the two-dimensional imaging elements include a first type of elements and a second type of elements.

[0166] The region extension module 1320 is used to determine the first target imaging region of the first type of element in the first imaging region based on the extension of the second imaging region in the first imaging region.

[0167] The first image display module 1330 is used to display a first type of element in a first target imaging area based on the first image display unit, and to display three-dimensional imaging elements in the remaining imaging area of ​​the first imaging area; the remaining imaging area is the imaging area in the first imaging area other than the first target imaging area.

[0168] The second image display module 1340 is used to display a second type of element in a second imaging area based on the second image display unit.

[0169] In one exemplary embodiment, the first image display module includes:

[0170] The location acquisition module is used to obtain the eye position information of the target object;

[0171] The disparity determination module is used to determine image disparity information based on the first target imaging area and eye position information;

[0172] The parallax display module is used to display a first type of element based on the first image display unit and image parallax information to obtain a parallax image, so that the target object can observe the first type of element formed in the first target imaging area through the parallax image.

[0173] In one exemplary embodiment, the first image display unit includes a first sub-display unit, and the first imaging region includes a first sub-imaging region corresponding to the first sub-display unit; the region extension module includes:

[0174] The first region determination module is used to determine the first target imaging region of the first type of element in the first sub-imaging region based on the extension of the second imaging region in the first sub-imaging region.

[0175] In one exemplary embodiment, the first sub-display unit includes a first optical element for compressing the imaging area; the first image display module includes:

[0176] The ratio determination module is used to determine the target compression ratio based on the proportion of the first target imaging region in the first sub-imaging region;

[0177] The first focal length adjustment module is used to adjust the focal length of the first optical element based on the target compression ratio;

[0178] The region compression module is used to compress the first sub-imaging region corresponding to the first imaging region of the first sub-display unit into the first target imaging region based on the adjusted first optical element.

[0179] The first element display module is used to display a first type of element based on the first sub-display unit, so that the first type of element is formed in the first target imaging area.

[0180] In one exemplary embodiment, the first image display unit further includes a second sub-display unit, and the first imaging region further includes a second sub-imaging region corresponding to the second sub-display unit; the first image display module includes:

[0181] The second element display module is used to display three-dimensional imaging elements in the second sub-imaging region of the remaining imaging region based on the second sub-display unit.

[0182] In one exemplary embodiment, the first sub-display unit includes a display screen; the second element display module includes:

[0183] The second region determination module is used to determine the second target imaging region corresponding to the three-dimensional imaging element.

[0184] The first light adjustment module is used to adjust the emission direction of the emitted light from the display screen when the second target imaging area exceeds the second sub-imaging area, so as to project the emitted light from the display screen onto the second sub-display unit.

[0185] The element classification module is used to determine the third and fourth categories of elements in the three-dimensional imaging elements; the second target imaging region corresponding to the third category of elements is outside the second sub-imaging region, and the second target imaging region corresponding to the fourth category of elements is inside the second sub-imaging region.

[0186] The third element display module is used to display a third type of element on the display screen, so that the light emitted from the display screen is projected onto the first sub-imaging area through the second sub-display unit; and to display a fourth type of element in the second sub-imaging area based on the second sub-display unit.

[0187] In one exemplary embodiment, the apparatus further includes a first display switching module for switching the display unit to display the first type of element when the display of the third type of element ends, the first display switching module comprising:

[0188] The second light adjustment module is used to adjust the emission direction of the emitted light of the display screen in response to the end display command for the third type of element, so as to project the emitted light of the display screen onto the first sub-display unit.

[0189] The fourth element display module is used to display the first type of element based on the display screen, so that the light emitted from the display screen is projected onto the first sub-imaging area through the first sub-display unit.

[0190] In one exemplary embodiment, the apparatus further includes a switching display unit for displaying a second display switch of the first type of elements during an element switching cycle update, the second display module comprising:

[0191] The periodic update module is used to determine the element currently displayed on the screen in response to the element switching cycle update;

[0192] The third light adjustment module is used to adjust the emission direction of the emitted light from the display screen when the element currently displayed on the display screen is a third type of element, so as to project the emitted light from the display screen onto the first sub-display unit.

[0193] The fifth element display module is used to display the first type of element based on the display screen, so that the light emitted from the display screen is projected onto the first sub-imaging area through the first sub-display unit.

[0194] In one exemplary embodiment, the apparatus further includes a switching display unit for displaying a third display switch of a third type of element during an element switching cycle update, the third display module comprising:

[0195] The fourth light adjustment module is used to adjust the emission direction of the emitted light from the display screen when the element currently displayed on the display screen is a first type of element, so as to project the emitted light from the display screen to the second sub-display unit.

[0196] The sixth element display module is used to display the third type of element on the display screen, so that the light emitted from the display screen is projected onto the first sub-imaging area through the second sub-display unit.

[0197] In one exemplary embodiment, the first sub-display unit further includes a second optical element disposed between the display screen and the first sub-imaging area, the focal length of the second optical element including a first focal length, the first focal length being used to project the emitted light from the display screen onto the second sub-display unit; the first light adjustment module or the fourth light adjustment module includes:

[0198] The second focal length adjustment module is used to adjust the focal length of the second optical element to the first focal length so that the light emitted from the display screen is projected onto the second sub-display unit through the second optical element.

[0199] In one exemplary embodiment, the focal length of the second optical element includes a second focal length used to project the emitted light from the display screen onto the first sub-display unit; the second light adjustment module or the third light adjustment module includes:

[0200] The third focal length adjustment module is used to adjust the focal length of the second optical element to the second focal length so that the light emitted from the display screen is projected onto the first sub-display unit through the second optical element.

[0201] It should be noted that the apparatus provided in the above embodiments is only illustrated by the division of the above functional modules when implementing its functions. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0202] This application provides an electronic device including a processor and a memory. The memory stores at least one instruction or at least one program, which is loaded and executed by the processor to implement any of the image element display methods provided in the above method embodiments.

[0203] Memory is used to store software programs and modules. The processor executes these stored software programs and modules to perform various functional applications and data processing. Memory can primarily consist of a program storage area and a data storage area. The program storage area stores the operating system, application programs required for functionality, etc.; the data storage area stores data created based on device usage, etc. Furthermore, memory can include high-speed random access memory (RAM) and non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, memory can also include a memory controller to provide the processor with access to the memory.

[0204] The method embodiments provided in this application can be executed in a computer terminal, server, or similar computing device; that is, the aforementioned electronic device may include a computer terminal, server, or similar computing device. Figure 14 is a hardware structure block diagram of a computer device running a method for displaying image elements according to an embodiment of this application. As shown in Figure 14, the internal structure of the computer device may include, but is not limited to, a processor, a network interface, and a memory. The processor, network interface, and memory within the computer device can be connected via a bus or other means; in the embodiments shown in Figure 14 of this specification, a bus connection is used as an example.

[0205] The processor (or CPU, Central Processing Unit) is the computing and control core of the computer device. The network interface may optionally include a standard wired interface or a wireless interface (such as Wi-Fi, mobile communication interface, etc.). Memory is the storage device in the computer device used to store programs and data. It is understood that the memory here can be a high-speed RAM storage device, or a non-volatile storage device, such as at least one disk storage device; optionally, it can also be at least one storage device located remotely from the aforementioned processor. The memory provides storage space, which stores the operating system of the electronic device, including but not limited to: Windows (an operating system), Linux (an operating system), Android (a mobile operating system), iOS (a mobile operating system), etc., which are not limited in this application; and the storage space also stores one or more instructions suitable for being loaded and executed by the processor, which can be one or more computer programs (including program code). In the embodiments of this specification, the processor loads and executes one or more instructions stored in the memory to implement the image element display method provided in the above method embodiments.

[0206] Embodiments of this application also provide a computer-readable storage medium, which can be disposed in an electronic device to store at least one instruction or at least one program related to implementing a display method for an image element. The at least one instruction or the at least one program is loaded and executed by the processor to implement any of the image element display methods provided in the above-described method embodiments.

[0207] Optionally, in this embodiment, the storage medium may include, but is not limited to, various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0208] It should be noted that the order of the embodiments described above is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. Furthermore, specific embodiments have been described above. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps described in the claims can be performed in a different order than that shown in the embodiments and still achieve the desired result. Additionally, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0209] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the apparatus embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0210] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

Claims

1. A display method of image elements, the method being applied to a head-up display device, the head-up display device comprising a first image display unit and a second image display unit, the first image display unit being configured to display elements in a first imaging area, the second image display unit being configured to image in a second imaging area, the first imaging area being a three-dimensional area, and the second imaging area being a two-dimensional area; the method comprising: obtaining a three-dimensional imaging element and a two-dimensional imaging element to be displayed; the two-dimensional imaging element comprising a first type of element and a second type of element; determining a first target imaging area of the first type of element in the first imaging area based on an extension of the second imaging area in the first imaging area; displaying the first type of element in the first target imaging area based on the first image display unit, and displaying the three-dimensional imaging element in a remaining imaging area of the first imaging area; the remaining imaging area being an imaging area of the first imaging area other than the first target imaging area; and displaying the second type of element in the second imaging area based on the second image display unit. The displaying the first type of element in the first target imaging area based on the first image display unit comprises: obtaining eye position information of a target object; determining image parallax information based on the first target imaging area and the eye position information; and displaying the first type of element based on the first image display unit and the image parallax information to obtain a parallax image, so that the target object observes the first type of element formed in the first target imaging area through the parallax image. The first image display unit comprises a first sub-display unit, and the first imaging area comprises a first sub-imaging area corresponding to the first sub-display unit. The determining the first target imaging area of the first type of element in the first imaging area based on the extension of the second imaging area in the first imaging area comprises: determining a first target imaging area of the first type of element in the first sub-imaging area based on an extension of the second imaging area in the first sub-imaging area. The first sub-display unit comprises a first optical element configured to compress an imaging area. The displaying the first type of element in the first target imaging area based on the first image display unit comprises: determining a target compression ratio based on a proportion of the first target imaging area in the first sub-imaging area; adjusting a focal length of the first optical element based on the target compression ratio; compressing the first sub-display unit in the first sub-imaging area corresponding to the first imaging area to the first target imaging area based on the adjusted first optical element; and displaying the first type of element based on the first sub-display unit, so that the first type of element is formed in the first target imaging area.

2. The image element display method according to claim 1, wherein The first image display unit further comprises a second sub-display unit, and the first imaging area further comprises a second sub-imaging area corresponding to the second sub-display unit. ​ ​ ​ 3. The method of displaying image elements according to claim 1, wherein, ​ ​ ​ 4. The image element display method according to claim 3, wherein ​ ​ ​ ​ ​ ​ 5. The method of displaying image elements according to claim 4, wherein, ​ The displaying the three-dimensional imaging element in the remaining imaging area of the first imaging area comprises: displaying the three-dimensional imaging element in a second sub-imaging area in the remaining imaging area based on the second sub-display unit.

6. The method of displaying image elements according to claim 5, wherein, The first sub-display unit comprises a display screen; and the displaying the three-dimensional imaging element in the second sub-imaging area based on the second sub-display unit comprises: determining a second target imaging area corresponding to the three-dimensional imaging element; adjusting an exit direction of an exit light ray of the display screen to project the exit light ray of the display screen to the second sub-display unit in a case where the second target imaging area exceeds the second sub-imaging area; determining a third type of element and a fourth type of element in the three-dimensional imaging element; the second target imaging area corresponding to the third type of element is outside the second sub-imaging area, and the second target imaging area corresponding to the fourth type of element is inside the second sub-imaging area; displaying the third type of element based on the display screen to make the exit light ray of the display screen project to the first sub-imaging area through the second sub-display unit, and displaying the fourth type of element in the second sub-imaging area based on the second sub-display unit.

7. The method of claim 6, further comprising: adjusting the exit direction of the exit light ray of the display screen to project the exit light ray of the display screen to the first sub-display unit in response to an end display instruction for the third type of element; displaying the first type of element based on the display screen to make the exit light ray of the display screen project to the first sub-imaging area through the first sub-display unit.

8. The method of claim 6, further comprising: determining an element currently displayed by the display screen in response to an update of an element switching period; adjusting the exit direction of the exit light ray of the display screen to project the exit light ray of the display screen to the first sub-display unit in a case where the element currently displayed by the display screen is the third type of element; displaying the first type of element based on the display screen to make the exit light ray of the display screen project to the first sub-imaging area through the first sub-display unit.

9. The method of claim 8, further comprising: adjusting the exit direction of the exit light ray of the display screen to project the exit light ray of the display screen to the second sub-display unit in a case where the element currently displayed by the display screen is the first type of element; displaying the third type of element based on the display screen to make the exit light ray of the display screen project to the first sub-imaging area through the second sub-display unit.

10. The method of displaying image elements according to any one of claims 7 to 9, wherein, The first sub-display unit further comprises a second optical element disposed between the display screen and the first sub-imaging area, and a focal length of the second optical element comprises a first focal length, the first focal length being used to project the exit light ray of the display screen to the second sub-display unit. The adjusting the exit direction of the exit light rays of the display screen to project the exit light rays of the display screen to the second sub-display unit comprises: adjusting the focal length of the second optical element to a first focal length, so that the exit light rays of the display screen are projected to the second sub-display unit through the second optical element.

11. The method of displaying image elements according to claim 10, wherein, The focal length of the second optical element comprises a second focal length, which is used to project the exit light rays of the display screen to the first sub-display unit; The adjusting the exit direction of the exit light rays of the display screen to project the exit light rays of the display screen to the first sub-display unit comprises: adjusting the focal length of the second optical element to a second focal length, so that the exit light rays of the display screen are projected to the first sub-display unit through the second optical element.

12. A display device of image elements, the device being applied to a head-up display device, the head-up display device comprising a first image display unit and a second image display unit, the first image display unit being used to display elements in a first imaging area, and the second image display unit being used to image in a second imaging area, the first imaging area being a three-dimensional area, and the second imaging area being a two-dimensional area; the device comprising: an element obtaining module, configured to obtain a three-dimensional imaging element and a two-dimensional imaging element to be displayed; the two-dimensional imaging element comprises a first type of element and a second type of element; an area extension module, configured to determine a first target imaging area of the first type of element in the first imaging area based on an extension of the second imaging area in the first imaging area; a first image display module, configured to display the first type of element in the first target imaging area and the three-dimensional imaging element in a remaining imaging area of the first imaging area based on the first image display unit; the remaining imaging area being an imaging area of the first imaging area other than the first target imaging area; a second image display module, configured to display the second type of element in the second imaging area based on the second image display unit.

13. An electronic device, comprising a processor and a memory, the memory storing at least one instruction or at least one program, the at least one instruction or the at least one program being loaded and executed by the processor to implement the display method of image elements according to any one of claims 1-11.

14. A computer-readable storage medium, the storage medium storing at least one instruction or at least one program, the at least one instruction or the at least one program being loaded and executed by a processor to implement the display method of image elements according to any one of claims 1-11.

15. A computer program, the computer program being executed by a processor to implement the display method of image elements according to any one of claims 1-11.

Images