Virtual image display apparatus, method and apparatus for generating image data, and related device

Abstract

A virtual image display apparatus (50), a method and apparatus for generating image data, and a related device. The virtual image display apparatus (50) comprises an image generation unit (51) and an imaging unit (52). The image generation unit (51) is used for acquiring image data, displaying an image on the basis of the image data, and outputting image light. The image displayed by the image generation unit (51) comprises a target image, the position of the target image in the displayed image corresponds to a viewing position in an eyebox of the virtual image display apparatus, and the position of the target image in the displayed image changes along with changes to the viewing position. The imaging unit (52) is used for forming a virtual image in a display area on the basis of the image light. The size of the eyebox of the virtual image display apparatus (50) is large.

Description

Virtual image display device, image data generation method, device and related equipment

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office of China on June 27, 2022, with application number 202210742008.X and application name “Virtual image display device, image data generation method, device and related equipment”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of display technology, and in particular to a virtual image display device, a method for generating image data, a device and related equipment. Background Art

[0003] With the development of display technology, virtual image display devices for displaying virtual images are increasingly used, for example, desktop displays and head-up displays (HUDs).

[0004] In related art, a virtual image display device includes an image generation unit and an imaging unit. The image generation unit is used to acquire image data, generate an image based on the image data, and output image light. The imaging unit forms a virtual image within the entire display area based on the image light. The user can see the complete virtual image by moving their eyes within the eye box of the virtual image display device.

[0005] The viewing distance corresponding to any viewing position within the eyebox is smaller than the viewing distance corresponding to the reference point. The reference point is the vertex of the field of view corresponding to the display area. Given a fixed display area for the imaging unit and the optical aperture of the virtual image display device, to ensure that a complete virtual image is visible at every viewing position within the eyebox, the eyebox size must be relatively small. This means that the user can only view the complete virtual image within a relatively small area, which is inconvenient.

[0006] Summary of the Invention

[0007] The present application provides a virtual image display device, an image data generation method, a device and related equipment, which can increase the size of the eye box so that the user can see the virtual image corresponding to the target image within a larger range, which is convenient for the user to use.

[0008] In a first aspect, a virtual image display device is provided. The virtual image display device includes an image generation unit and an imaging unit. The image generation unit is configured to acquire image data, display an image based on the image data, and output image light. The image displayed by the image generation unit includes a target image. The position of the target image in the displayed image corresponds to a viewing position in an eye box of the virtual image display device, and the position of the target image in the displayed image changes as the viewing position changes. The imaging unit is configured to form a virtual image in a display area based on the image light.

[0009] In the present application, the position of the target image in the image displayed by the image generation unit corresponds to the viewing position in the eyebox of the virtual image display device, and the position of the target image in the image displayed by the image generation unit changes with changes in the viewing position. That is, different viewing positions in the eyebox of the virtual image display device correspond to different positions of the target image in the image displayed by the image generation unit. Because the image light output by the image generation unit corresponds to the image displayed by the image generation unit, and the virtual image is formed based on the image light, the position of the virtual image corresponding to the target image in the entire display area of ​​the virtual image display device will also change with changes in the viewing position. When the size of the virtual image corresponding to the target image is constant, the optical aperture of the virtual image display device is constant, and the viewing distance is constant, compared to when the virtual image corresponding to the target image is directly imaged in the entire display area of ​​the virtual image display device, the user can see the complete virtual image corresponding to the target image over a larger range, thereby increasing the size of the eyebox and facilitating user use.

[0010] Here, the viewing distance may refer to the distance between the viewing position and the center of the virtual image in the direction of the main light in the central field of view of the virtual image display device, or it may refer to the distance between the viewing position and the center of the reference optical element, where the reference optical element is the optical element closest to the viewer in the optical path among the optical elements involved in imaging, such as the curved mirror closest to the viewer in a desktop display, or the windshield of a vehicle where the HUD is located.

[0011] In the present application, the entire virtual image corresponding to the target image can be seen at any viewing position in the eye box.

[0012] Exemplarily, the position of the target image in the image displayed by the image generation unit refers to the position of the first image area where the target image is located in the entire image area of ​​the image displayed by the image generation unit, which can be represented by the coordinates of at least part of the pixel points in the first image area.

[0013] In some examples, the position of the target image can be represented by the coordinates of a reference point and the size of the target image. Here, the reference point can be any point in the target image, for example, the center point of the target image, a vertex of the target image (such as the upper left vertex or lower left vertex of a rectangular target image, etc.), any point on the outer contour line of the target image, etc. The coordinates of the reference point are the coordinates of the pixel point corresponding to the reference point in the image to be displayed. The size of the target image includes the length, width, and area of ​​the target image, etc. The position of the target image in the image displayed by the image generation unit can be uniquely determined by combining the coordinates of the reference point and the size of the target image.

[0014] In other examples, the location of the target image can be represented by the coordinates of a reference point set, which includes at least two reference points. The coordinates of all reference points in the reference point set not only serve as positioning but also indicate the size of the target image. For example, the reference point set may include the four vertices of a rectangular target image, or two vertices on a diagonal of a rectangular target image, and so on. The location of the target image in the image to be displayed can be uniquely determined based on the coordinates of the reference point set.

[0015] In some examples, the viewing distance corresponding to each viewing position in the eye box is greater than the viewing distance corresponding to the reference point, and the reference point is the vertex of the first field of view angle corresponding to the display area.

[0016] In the present application, the reference point is the vertex of the first field of view angle of the display area. If one eye of the user is located at the reference point, the user can see the complete virtual image displayed in the display area. When the viewing position of the user is defined by the center position of the binocular eyes, if the viewing distance of the user's viewing position is far relative to the reference point, the user can see less non-display area (i.e., the peripheral area of ​​the display area). If the viewing distance of the user's viewing position is far enough relative to the reference point, the virtual image in part of the display area will be seen, and the image in the non-display area will not be seen at all. Therefore, when the viewing distance corresponding to any viewing position in the eye box of the virtual image display device is greater than the viewing distance corresponding to the reference point, the display area seen by the user accounts for a larger proportion of the user's entire viewing area, which is conducive to improving the user experience.

[0017] In the present application, the display area includes a first sub-display area for displaying a virtual image corresponding to the target image. When the center of the first sub-display area coincides with the center of the display area, a ratio of a first distance to a second distance is within a ratio range, wherein the first distance is the distance between a reference point and a center point, and the second distance is the distance between an eyebox center position and the center point. The eyebox center position is located on a principal ray of the central field of view of the virtual image display device. The center point is the intersection of the principal ray and a plane containing the intersection of a first marginal ray and a second marginal ray. The first marginal ray is a marginal ray corresponding to the first field of view angle, and the second marginal ray is a marginal ray corresponding to the second field of view angle corresponding to the first sub-display area. The ratio range is [20%, 90%]. Here, there are an infinite number of first marginal rays and second marginal rays, and accordingly, there are an infinite number of intersection points of the first marginal rays and the second marginal rays, forming a substantially circular area. The radial dimension of the substantially circular area is the size of the optical aperture of the virtual image display device.

[0018] When the ratio of the first distance to the second distance is within this ratio range, each viewing position in the eye box can see most of the display area of ​​the virtual image display device, and the first sub-display area used to display the target image accounts for a relatively large area in the entire display area, which is convenient for user viewing and provides a better user experience.

[0019] In some examples, the imaging unit includes a curved mirror configured to reflect the image light output by the image generating unit to form the virtual image.

[0020] In other examples, the imaging unit includes a windshield, which is configured to reflect the image light output by the image generating unit to form the virtual image.

[0021] In some examples, the ratio of the first distance to the second distance is equal to the ratio of the difference between the optical aperture of the virtual image display device and the reference pupil distance to the sum of the optical aperture and the reference pupil distance. When this ratio of the first distance to the second distance meets this condition, when the user views from the center of the eye box, the virtual image corresponding to the target image can be located in the middle of the display area of ​​the entire virtual image display device, and in the overlapping area between the display areas seen by the user's two eyes, thereby improving the user's viewing experience.

[0022] In some examples, in the eyebox, the area of ​​the target image corresponding to the first viewing position is larger than the area of ​​the target image corresponding to the second viewing position, and the viewing distance corresponding to the first viewing position is smaller than the viewing distance corresponding to the second viewing position. For example, the target image corresponding to the first viewing position is obtained by enlarging or reducing the target image corresponding to the second viewing position.

[0023] In this way, the virtual image corresponding to the target image increases as the viewing distance corresponding to the viewing position decreases, or the virtual image corresponding to the target image decreases as the viewing distance corresponding to the viewing position increases. When the user views at a first viewing position closer to the virtual image, a larger virtual image of the target image can be seen, and when the user views at a second viewing position farther from the virtual image, a smaller virtual image of the target image can be seen. Due to the limitation of the aperture diaphragm, the closer the user is to the virtual image, the larger the range of the display area that can be seen, and the farther the user is from the virtual image, the smaller the range of the display area that can be seen. Therefore, by enlarging the virtual image of the target image when the user is closer to the virtual image and reducing the virtual image of the target image when the user is farther from the virtual image, it can be ensured that the user can always see the entire virtual image of the target image, while seeing less virtual image corresponding to the background image, or even no virtual image corresponding to the background image.

[0024] In other examples, in the eye box, the virtual image of the target image corresponding to the third viewing position is offset in a first direction relative to the virtual image of the target image corresponding to the fourth viewing position, with the principal ray of the central field of view of the virtual image display device as a reference, and the third viewing position is offset in a second direction relative to the fourth viewing position with the principal ray as a reference, and the first direction and the second direction are two opposite directions with the principal ray as a reference.

[0025] Here, the third viewing position is offset relative to the fourth viewing position in the second direction based on the main light, which means that the third viewing position is farther from the main light than the fourth viewing position in the second direction, and the second direction is the radial outward direction centered on the main light.

[0026] In this example, the virtual image corresponding to the target image translates as the viewing position translates, and the translation direction of the virtual image corresponding to the target image is opposite to the translation direction of the viewing position.

[0027] Because the user's viewing position deviates perpendicularly from the principal ray due to the limitations of the aperture, the virtual image they see moves in a direction perpendicular to the principal ray and opposite to the direction of the deviation. Thus, when the user views from a third viewing position, closer to the principal ray, the center of the virtual image of the target image is closer to the principal ray. Conversely, when the user views from a fourth viewing position, farther from the principal ray, the center of the virtual image of the target image is farther from the principal ray. This ensures that the user always sees the entire virtual image of the target image and cannot see the virtual image corresponding to the background image.

[0028] Optionally, the virtual image corresponding to the target image is a three-dimensional image or a two-dimensional image.

[0029] In some examples, the displayed image further includes a background image, which is an image other than the target image in the displayed image. Optionally, the background image is a static image or a dynamic image.

[0030] Optionally, the virtual image includes a virtual image corresponding to the target image and a virtual image corresponding to the background image, wherein the virtual image corresponding to the background image is peripheral to the virtual image corresponding to the target image or covers at least a portion of the virtual image corresponding to the background image. Here, the virtual image corresponding to the target image covers at least a portion of the virtual image corresponding to the background image, which means that the virtual image corresponding to the target image is in front of the virtual image corresponding to the background image and close to the viewer, so that at least a portion of the virtual image corresponding to the background image is obscured by the virtual image corresponding to the target image.

[0031] In a second aspect, a method for generating image data is provided. The method includes: determining an eye position of a viewer of a virtual image display device, the eye position being located within an eye box of the virtual image display device; determining position indication information based on the eye position, the position indication information being used to indicate a position of a target image in an image to be displayed, the position of the target image changing as the viewer's eye position changes; and generating image data for the image to be displayed based on the image data of the target image and the position indication information, the image data for the image to be displayed being used by the virtual image display device to form a virtual image in a display area.

[0032] In the present application, the method may be executed by a processor, which may be integrated into the virtual image display device, or may be an external component of the virtual image display device.

[0033] In the present application, the position of the target image in the image to be displayed is determined based on the eye position of the viewer of the virtual image display device, and image data of the image to be displayed is generated for the virtual image display device to form a virtual image in the display area. The position of the target image in the image to be displayed changes with the change of the viewer's eye position. Since the image light output by the image generation unit corresponds to the image data of the image to be displayed, and the virtual image is formed based on the image light, the position of the virtual image corresponding to the target image in the entire display area of ​​the virtual image display device will also change with the change of the eye position. When the size of the virtual image corresponding to the target image is constant and the viewing distance is constant, compared with the virtual image corresponding to the target image being directly imaged in the entire display area of ​​the virtual image display device, the user can see the complete virtual image corresponding to the target image in a larger range, thereby increasing the size of the eye box and facilitating user use.

[0034] In one possible implementation, the position indication information includes position information of a first sub-display area, the position information of the first sub-display area being used to indicate a position of the first sub-display area within a display area of ​​the virtual image display device, the first sub-display area being used to display a virtual image corresponding to the target image. Determining the position indication information based on the eye position includes determining the position information of the first sub-display area corresponding to the eye position based on a correspondence between a viewing position and the position information of the sub-display area.

[0035] Exemplarily, the position information of the first sub-display area includes geometric information of the first sub-display area and the position of a reference point in the first sub-display area, and the geometric information of the first sub-display area is used to indicate the area of ​​the first sub-display area. Here, the geometric information includes the shape and size of the first sub-display area.

[0036] In some examples, in the corresponding relationship, the area of ​​the sub-display area corresponding to the first viewing position is larger than the area of ​​the sub-display area corresponding to the second viewing position, and the viewing distance corresponding to the first viewing position is smaller than the viewing distance corresponding to the second viewing position.

[0037] In other examples, in the corresponding relationship, the position of the sub-display area corresponding to the third viewing position is offset in a first direction relative to the position of the sub-display area corresponding to the fourth viewing position, with the principal ray of the central field of view of the virtual image display device as a reference, and the third viewing position is offset in a second direction relative to the fourth viewing position with the principal ray as a reference, and the first direction and the second direction are two opposite directions with the principal ray as a reference.

[0038] In another possible implementation, the position indication information includes position information of a first image region, where the first image region is the region where the target image is located. Determining the position indication information based on the eye position includes determining the position information of the first image region corresponding to the eye position based on a correspondence between a viewing position and the position information of the image region.

[0039] In one possible embodiment, generating the image data of the image to be displayed based on the image data of the target image and the position indication information includes: combining the image data of the target image and the image data of the background image to obtain the image data of the image to be displayed, and the target image is located at the position indicated by the position indication information.

[0040] In some examples, the background image is located in an area outside the area where the target image is located in the image to be displayed.

[0041] To improve the display quality of the virtual image display device, the method may further include: performing distortion correction processing on the image data of the target image based on the eye position or the position indication information. Because the degree of distortion of virtual images in different parts of the display area of ​​the virtual image display device varies, distortion correction processing may be performed on the image data of the target image based on the eye position or the position indication information to improve the display quality of the virtual image corresponding to the target image.

[0042] Exemplarily, according to the eye position or the position indication information, the image data of the target image is subjected to distortion correction processing, including: obtaining distortion correction parameters corresponding to the eye position or the position indication information, the distortion correction parameters being used to compensate for the distortion caused by the optical elements of the virtual image display device; and using the distortion correction parameters to perform distortion correction processing on the image data of the target image.

[0043] In some examples, a correspondence between the viewing position and the position of a sub-display area, as well as a correspondence between the position of a sub-display area and a distortion correction parameter, can be pre-established. Based on the correspondence between the viewing position and the position of the sub-display area, the position of the first sub-display area corresponding to the eye position can be determined. Then, based on the correspondence between the position of the sub-display area and the distortion correction parameter, the distortion correction parameter corresponding to the position of the first sub-display area is determined. The distortion correction parameter corresponding to the position of the first sub-display area is the distortion correction parameter corresponding to the eye position.

[0044] In other examples, a correspondence between the viewing position and the position of the sub-display area, and a correspondence between the position of the sub-display area and the distortion correction parameter, can be generated. In this way, the distortion correction parameter corresponding to the eye position can be directly determined based on the correspondence between the viewing position and the distortion correction parameter.

[0045] In yet other examples, a correspondence between the position of the image area and the position of the sub-display area, as well as a correspondence between the position of the sub-display area and the distortion correction parameters, can be pre-established. Based on the correspondence between the position of the image area and the position of the sub-display area, the position of the first sub-display area corresponding to the position indicated by the position indication information can be determined. Then, based on the correspondence between the position of the sub-display area and the distortion correction parameters, the distortion correction parameters corresponding to the position of the first sub-display area are determined. The distortion correction parameters corresponding to the position of the first sub-display area are the distortion correction parameters corresponding to the position indication information.

[0046] In yet other examples, a correspondence between the position of the image area and the position of the sub-display area, and a correspondence between the position of the sub-display area and the distortion correction parameters, can be generated. In this way, the distortion correction parameters corresponding to the position indication information can be directly determined based on the correspondence between the position of the image area and the distortion correction parameters.

[0047] In a third aspect, a device for generating image data is provided. The device includes an eye position determination module, an information determination module, and a generation module. The eye position determination module is configured to determine the eye position of a viewer of a virtual image display device, where the eye position is located within an eye box of the virtual image display device. The information determination module is configured to determine position indication information based on the eye position, where the position indication information indicates the position of a target image in an image to be displayed, where the position of the target image changes with the viewer's eye position. The generation module is configured to generate image data for the image to be displayed based on the image data of the target image and the position indication information, where the image data for the image to be displayed is used by the virtual image display device to form a virtual image in a display area.

[0048] In one possible implementation, the position indication information includes position information of a first sub-display area, the position information of the first sub-display area being used to indicate a position of the first sub-display area within a display area of ​​the virtual image display device, the first sub-display area being used to display a virtual image corresponding to the target image. The information determination module is configured to determine the position information of the first sub-display area corresponding to the eye position based on a correspondence between the viewing position and the position information of the sub-display areas.

[0049] In another possible embodiment, the position indication information includes position information of a first image area, where the first image area is the area where the target image is located; the information determination module is used to determine the position information of the first image area corresponding to the eye position based on the correspondence between the viewing position and the position information of the image area.

[0050] In some examples, the generating module is configured to combine the image data of the target image and the image data of the background image to obtain the image data of the image to be displayed, and the target image is located at the position indicated by the position indication information.

[0051] Optionally, the device further includes: a correction module, configured to perform distortion correction processing on the image data of the target image according to the eye position or the position indication information.

[0052] Exemplarily, the correction module includes a parameter acquisition submodule and a correction submodule. The parameter acquisition submodule is configured to obtain distortion correction parameters corresponding to the eye position or the position indication information, wherein the distortion correction parameters are used to compensate for distortion caused by optical elements of the virtual image display device. The correction submodule is configured to perform distortion correction processing on the image data of the target image using the distortion correction parameters.

[0053] In a fourth aspect, a computer device is provided. The computer device includes a processor and a memory, wherein the memory is used to store a software program, and the processor executes the software program stored in the memory to enable the computer device to implement the method described in any example of the third aspect.

[0054] In a fifth aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and when the computer instructions in the computer-readable storage medium are executed by a computer device, the computer device executes the method described in any example of the third aspect.

[0055] In a sixth aspect, a computer program product is provided, wherein the computer program product includes computer instructions, and when the computer instructions are executed by a computer device, the computer device executes the method described in any example of the third aspect.

[0056] In a seventh aspect, a display device is provided. The display device includes a processor and a virtual image display device, wherein the virtual image display device is any of the aforementioned virtual image display devices, the processor is configured to generate image data of an image to be displayed based on any of the aforementioned methods, and the virtual image display device is configured to display a virtual image based on the image data.

[0057] In some examples, the display device further includes a power supply for powering the processor and the virtual image display device.

[0058] In some examples, the display device is a heads-up display device or a desktop display device.

[0059] In an eighth aspect, the present application provides a means of transport, the means of transport comprising any of the aforementioned display devices. Exemplarily, the means of transport includes but is not limited to a car, airplane, train, or ship. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] FIG1 is a schematic diagram of a display device in use according to an embodiment of the present application;

[0061] FIG2 is a schematic diagram of another display device in use according to an embodiment of the present application;

[0062] FIG3 is a schematic diagram of another display device in use according to an embodiment of the present application;

[0063] FIG4 is a schematic diagram of another display device in use according to an embodiment of the present application;

[0064] FIG5 is a schematic structural diagram of a virtual image display device provided in an embodiment of the present application;

[0065] FIG6 is a schematic structural diagram of another virtual image display device provided in an embodiment of the present application;

[0066] 7 is a schematic diagram showing the relationship between the viewing position and the reference point in the eye box of a virtual image display device provided by an embodiment of the present application;

[0067] FIG8 is a flow chart of a method for generating image data provided in an embodiment of the present application;

[0068] FIG9 is a schematic diagram of the relationship between a viewing position and a sub-display area provided in an embodiment of the present application;

[0069] FIG10 is a schematic diagram showing the relationship between the first sub-display area and the entire display area corresponding to each viewing position in FIG9;

[0070] FIG11 is a schematic diagram showing the relationship between another viewing position and a sub-display area provided in an embodiment of the present application;

[0071] FIG12 is a schematic diagram showing the relationship between the first sub-display area and the entire display area corresponding to each viewing position in FIG11;

[0072] FIG13 is a schematic structural diagram of an image data generating device provided in an embodiment of the present application;

[0073] FIG14 is a schematic structural diagram of a computer device provided in an embodiment of the present application;

[0074] FIG15 is a circuit diagram of a display device provided in an embodiment of the present application;

[0075] FIG16 is a schematic diagram of a functional framework of a vehicle provided in an embodiment of the present application. DETAILED DESCRIPTION

[0076] The virtual image display device provided in the embodiment of the present application is described in detail below with reference to the accompanying drawings. The virtual image display device can be used as an ordinary display (such as shown in 100a in Figure 1) for office use, and can also be used as a television (such as shown in 100b in Figure 2) for home entertainment (as a television), or can be used for vehicle display (such as shown in 100c in Figure 3, the virtual image display device is installed on the seat of the vehicle), or can be used for head-up display (such as shown in 100d in Figure 4, the virtual image display device is installed on the vehicle as a head-up display (HUD)), or can be used for near-eye display (such as a virtual reality helmet or VR glasses, etc.). The physical size, display size, and resolution of the virtual image display device can be adjusted according to the usage scenario. In this application, the virtual image display device can also be referred to as a virtual image display system.

[0077] FIG5 is a schematic structural diagram of a virtual image display device provided by an embodiment of the present application. As shown in FIG5 , the virtual image display device 50 includes an image generation unit 51 and an imaging unit 52. The image generation unit 51 is used to acquire image data, and to display an image based on the image data and output image light. The imaging unit 52 is used to form a virtual image in a display area based on the image light output by the image generation unit 51. The image displayed by the image generation unit 51 includes a target image. The position of the target image in the image displayed by the image generation unit 51 corresponds to the viewing position in the eye box of the virtual image display device, and the position of the target image in the image displayed by the image generation unit 51 changes with the change of the viewing position. The target image corresponding to different viewing positions has different positions in the image displayed by the image generation unit 51.

[0078] The eyebox of the virtual image display device is an area where the eye can move. If the eye is within this area, the virtual image corresponding to the target image can be fully viewed. If the eye is outside this area, the virtual image corresponding to the target image cannot be fully viewed. In the embodiment of the present application, the eyebox includes multiple viewing positions (i.e., positions where the eye is located), and the entire virtual image corresponding to the target image can be viewed from any viewing position within the eyebox.

[0079] In an embodiment of the present application, the position of the target image in the image displayed by the image generation unit corresponds to the viewing position in the eyebox of the virtual image display device, and the position of the target image in the image displayed by the image generation unit changes with changes in the viewing position. That is, different viewing positions in the eyebox of the virtual image display device correspond to different positions of the target image in the image displayed by the image generation unit. Because the image light output by the image generation unit corresponds to the image displayed by the image generation unit, and the virtual image is formed based on the image light, the position of the virtual image corresponding to the target image in the entire display area of ​​the virtual image display device will also change with changes in the viewing position. When the size of the virtual image corresponding to the target image, the optical aperture of the virtual image display device, and the viewing distance are constant, compared with the virtual image corresponding to the target image being directly imaged in the entire display area of ​​the virtual image display device, the user can see the complete virtual image corresponding to the target image over a larger range, thereby increasing the size of the eyebox and facilitating user use.

[0080] Here, the viewing distance may refer to the distance between the viewing position and the center of the virtual image in the direction of the main light in the central field of view of the virtual image display device, or it may refer to the distance between the viewing position and the center of the reference optical element, where the reference optical element is the optical element closest to the viewer in the optical path among the optical elements involved in imaging, such as the curved mirror closest to the viewer in a desktop display, or the windshield of a vehicle where the HUD is located.

[0081] Moreover, compared with the method of increasing the size of some optical elements in the optical system of the virtual image display device to increase the optical aperture and thus achieve the expansion of the eye box, the virtual image display device in the embodiment of the present application can adopt an optical system with a smaller optical aperture. Since the size of the optical aperture is usually related to the volume of the optical system, the larger the optical aperture, the larger the volume of the optical system, and conversely, the smaller the optical aperture, the smaller the volume of the optical system. Therefore, the embodiment of the present application is also conducive to obtaining a virtual image display device with a smaller volume.

[0082] In the embodiment of the present application, the position of the target image in the image displayed by the image generation unit refers to the position of the first image region where the target image is located within the entire image region of the image displayed by the image generation unit. For example, the position of the target image in the image displayed by the image generation unit can be represented by the coordinates of at least some pixels in the first image region.

[0083] In some examples, the position of the target image can be represented by the coordinates of a reference point and the size of the target image. Here, the reference point can be any point in the target image, for example, the center point of the target image, a vertex of the target image (such as the upper left vertex or lower left vertex of a rectangular target image, etc.), any point on the outer contour line of the target image, etc. The coordinates of the reference point are the coordinates of the pixel point corresponding to the reference point in the image to be displayed. The size of the target image includes the length, width, and area of ​​the target image, etc. The position of the target image in the image displayed by the image generation unit can be uniquely determined by combining the coordinates of the reference point and the size of the target image.

[0084] In other examples, the location of the target image can be represented by the coordinates of a reference point set, which includes at least two reference points. The coordinates of all reference points in the reference point set not only serve as positioning but also indicate the size of the target image. For example, the reference point set may include the four vertices of a rectangular target image, or two vertices on a diagonal of a rectangular target image, and so on. The location of the target image in the image to be displayed can be uniquely determined based on the coordinates of the reference point set.

[0085] The different positions of the two target images may mean that the pixels contained in the target images are at least partially different. Assuming that the first target image and the second target image are two target images with different positions, the different pixels contained in the first target image and the second target image include the following situations: the first target image and the second target image contain the same number of pixels, and the coordinates of some or all of the pixels are different (that is, the second target image can be obtained by translating the first target image); the number of pixels contained in the first target image is greater than the number of pixels contained in the second target image, and the pixels of the first target image contain the pixels of the second target image (that is, the second target image can be obtained by scaling the first target image); the number of pixels contained in the first target image is greater than the number of pixels contained in the second target image, and some of the pixels of the second target image are the same as some of the pixels of the first target image (that is, the second target image can be obtained by scaling and translating the first target image).

[0086] In an embodiment of the present application, the target image can be an image of any content. For example, when the virtual image display device is a desktop display system, the target image can be a photo, a video image, a text, and the like. For another example, when the virtual image display device is a HUD, the target image can include at least one of an augmented reality (AR) display image and a status display image. The augmented reality display image is used to display information such as indication information of external objects, navigation information, and the like. The indication information of external objects includes but is not limited to safe vehicle distance, surrounding obstacles, and reversing images. Navigation information includes but is not limited to direction arrows, distance, and driving time. The status display image is used to display the status information of the vehicle. Taking a car as an example, the status information of the vehicle includes but is not limited to information such as driving speed, mileage, fuel level, water temperature, and headlight status.

[0087] In some examples, the position of the target image corresponding to a portion of the viewing positions in the eyebox is the entire image area of ​​the image displayed by the image generation unit 51. That is, the image displayed by the image generation unit 51 only includes the target image. The position of the target image corresponding to another portion of the viewing positions in the eyebox is a portion of the image area displayed by the image generation unit 51. That is, the image displayed by the image generation unit 51 includes the target image and a background image, where the background image is the image other than the target image in the image to be displayed.

[0088] In other examples, the positions of the target images corresponding to all viewing positions in the eye box are all partial image areas of the image displayed by the image generation unit 51 .

[0089] In the embodiments of the present application, the background image is a static image or a dynamic image. For example, a static image can be a solid color image (such as green or black), a geometric pattern, or a picture or photo of any other content. A dynamic image can be a switching between multiple static images, or an image containing graphics that change over time.

[0090] This application does not limit the content of the background image. In some examples, the background image can be an image associated with the content of the target image. For example, the background image can be obtained by processing part or all of the target image, including but not limited to one or more methods such as blurring and defocusing. In other examples, the background image can be one or more images with fixed content.

[0091] In some examples, the virtual image corresponding to the target image is a three-dimensional image or a two-dimensional image. When the target image occupies a portion of the image area displayed by the image generation unit 51 , the virtual image corresponding to the target image is located in a portion of the display area of ​​the virtual image display device 51 .

[0092] In some examples, the virtual image corresponding to the background image does not exist, or is a three-dimensional image or a two-dimensional image. For example, when the background image is a black image, the image light output by the image generation unit 51 does not contain the light corresponding to the background image. In this case, the virtual image corresponding to the background image does not exist. If the surrounding environment of the imaging unit is black (such as a desktop monitor), the virtual image of the background image seen by the user at the viewing position is black. If the surrounding environment of the imaging unit is brighter (such as a HUD), the user cannot see the virtual image of the background image at the viewing position. For another example, when the background image is not a pure black image, the virtual image of the background image can be three-dimensional or two-dimensional.

[0093] Optionally, when the virtual image displayed by the virtual image display device 51 includes a virtual image corresponding to the target image and a virtual image corresponding to the background image, the virtual image corresponding to the background image is located outside the virtual image corresponding to the target image, or the virtual image corresponding to the target image covers at least a portion of the virtual image corresponding to the background image. Here, "the virtual image corresponding to the target image covers at least a portion of the virtual image corresponding to the background image" means that the virtual image corresponding to the target image is located in front of the virtual image corresponding to the background image and close to the viewer, so that at least a portion of the virtual image corresponding to the background image is obscured by the virtual image corresponding to the target image.

[0094] In the embodiments of the present application, the image generation unit 51 is also called an optical machine. In some examples, the image generation unit 51 may include a light source, a light modulator, and a driver. The light source is used to generate a light beam. The driver is used to drive the light modulator based on the image data of the image to be displayed. The light modulator is used to modulate the light beam generated by the light source under the drive of the driver to form image light. The light modulator includes but is not limited to a liquid crystal modulator, liquid crystal on silicon (LCoS), a micro-electro-mechanical system (MEMS), or a digital micromirror device (DMD). In other examples, the image generation unit 51 may also use a display screen, such as an organic light-emitting diode (OLED) display screen. The present application does not limit the structure of the image generation unit 51, as long as it can form image light carrying image information based on the acquired image data.

[0095] In the embodiment of the present application, the imaging unit 52 includes one or more optical elements. Optical elements include, but are not limited to, reflectors and lenses. Reflectors include, for example, plane reflectors, concave mirrors, and convex mirrors. Lenses include, for example, convex lenses, concave lenses, and free-form mirrors. The present application does not impose any restrictions on the structure of the imaging unit 52, as long as it can form a virtual image based on the image light output by the image generation unit 51.

[0096] Figure 6 is a schematic diagram of the structure of a virtual image display device provided by an embodiment of the present application. As shown in Figure 6, the imaging unit 52 includes a curved mirror 521, which is used to reflect the image light output by the image generation unit 51 to form a virtual image.

[0097] Optionally, the imaging unit 52 further includes a diffusion screen 522 , which is used to diffuse the image light output by the image generating unit 51 and then output the image light to the curved mirror 521 .

[0098] Exemplarily, the virtual image display device in FIG6 may be a desktop display system.

[0099] Figure 7 is a schematic diagram illustrating the relationship between viewing positions and reference points within an eyebox of a virtual image display device provided by an embodiment of the present application. As shown in Figure 7 , the dotted box represents the eyebox, which includes multiple viewing positions. The center of both eyes is considered a viewing position, and Figure 7 shows only one viewing position. When the virtual image V remains unchanged, the user's left eye A1 can see a portion of the right side of the virtual image V, namely, a portion of the right area X1. The user's right eye A2 can see a portion of the left side of the virtual image V, namely, a portion of the left area X2. There is an overlapping area X12 between the right area X1 and the left area X2.

[0100] This phenomenon of seeing different portions of the virtual image at different viewing positions is due to the characteristics of the optical system of the virtual image forming unit. The optical elements of this imaging unit include an aperture stop, which limits the maximum angle (also known as the aperture angle) of the marginal rays in the point imaging beam on the optical axis. The aperture of this aperture stop can be referred to as the optical aperture of the virtual image display device. Within the constraints of the aperture stop, users can see different portions of the virtual image V at different viewing positions.

[0101] Assume that the viewing distance corresponding to any viewing position in the eye box is the first viewing distance h1. Here, the viewing distance refers to the distance between the viewing position and the center of the virtual image in the direction of the main light of the central field of view of the virtual image display device. The main light of the central field of view of the virtual image display device refers to the light from the center of the virtual image to the center of the aperture stop, and the main light is the central light of the light beam participating in the imaging. For example, as shown by the straight line O in Figure 7. The vertex of the first field of view angle corresponding to the display area is taken as the reference point A, and the distance between the reference point A and the center of the virtual image is the reference distance h2. The first viewing distance h1 is greater than the reference distance h2. The reference point A is the intersection point of the first edge light of the display area under the restriction of the aperture stop, such as the intersection point of the two first edge light rays L1 in Figure 6, and the main light passes through the reference point A.

[0102] The reference point is the vertex of the first field of view of the display area. If one of the user's eyes is located at the reference point, the user can see the complete virtual image displayed in the display area. When the user's viewing position is defined by the center position of the binocular eyes, if the user's viewing position is far away from the reference point, the user can see less of the non-display area (i.e., the peripheral area of ​​the display area). If the user's viewing position is far enough away from the reference point, the user will see a virtual image in part of the display area and will not see the image in the non-display area at all. Therefore, when the viewing distance corresponding to any viewing position in the eye box of the virtual image display device is greater than the viewing distance corresponding to the reference point, the display area seen by the user accounts for a larger proportion of the user's entire viewing area, which is conducive to improving the user experience.

[0103] In an embodiment of the present application, the display area includes a first sub-display area for displaying a virtual image corresponding to a target image. When the center of the first sub-display area coincides with the center of the display area, the ratio of the first distance d1 to the second distance d2 is within a ratio range. The first distance d1 is the distance between the reference point A and the center point, and the second distance d2 is the distance between the center of the eye box and the center point. The eye box center is located on the principal ray O, and the center point is the intersection of the principal ray O and the plane P where the first marginal ray L1 and the second marginal ray L2 intersect. The first marginal ray L1 is the marginal ray corresponding to the first viewing angle, and the second marginal ray L2 is the marginal ray corresponding to the second viewing angle corresponding to the first sub-display area. The ratio range is [20%, 90%]. Here, there are countless first marginal rays L1 and second marginal rays L2, but only two of each are shown in the figure. Accordingly, there are countless intersections of the first marginal rays L1 and the second marginal rays L2, forming a roughly circular area. The radial dimension of this roughly circular area is the size of the optical aperture of the virtual image display device.

[0104] When the ratio of the first distance d1 to the second distance d2 is within this ratio range, most of the display area of ​​the virtual image display device can be seen at each viewing position in the eye box, and the first sub-display area for displaying the target image occupies a relatively large area in the entire display area, which is convenient for user viewing and provides a better user experience.

[0105] It should be noted that the aperture stop's aperture is determined by the area formed by the intersection of the first marginal ray L1 and the second marginal ray L2. This is because in some examples, the aperture stop does not exist in the direction of propagation of the light beam between the viewing position and the virtual image. For example, in a HUD, image light propagates from below the windshield to the windshield, then reflects off the windshield to the human eye. In this case, the aperture stop is not in the direction of propagation of the light beam between the human eye and the virtual image. In other examples, such as desktop display systems, the aperture stop is located in the direction of propagation of the light beam between the viewing position and the virtual image. In this case, the intersection of the first marginal ray L1 and the second marginal ray L2 represents the aperture edge of the aperture stop.

[0106] In some examples, the ratio of the first distance d1 to the second distance d2 is equal to the ratio of the difference between the optical aperture D of the virtual image display device and the reference pupil distance S to the sum of the optical aperture D and the reference pupil distance S. Here, the reference pupil distance is a set value that can be set according to actual needs. Exemplarily, the reference pupil distance can range from 55 mm to 75 mm, for example, 65 mm. That is, the ratio can be expressed using formula (1).

[0107] When the ratio of the first distance to the second distance satisfies formula (1), when the user views from the center of the eye box, the virtual image corresponding to the target image can be located in the middle of the display area of ​​the entire virtual image display device and in the overlapping area of ​​the display areas seen by both eyes, for example, the overlapping area X12 in Figure 7. In this way, a larger eye box can be obtained while maintaining a larger first sub-display area, thereby improving the user's viewing experience.

[0108] In some examples, within the eyebox, the area of ​​the target image corresponding to the first viewing position is larger than the area of ​​the target image corresponding to the second viewing position, and the viewing distance corresponding to the first viewing position is smaller than the viewing distance corresponding to the second viewing position. For example, the target image corresponding to the first viewing position is obtained by magnifying or reducing the target image corresponding to the second viewing position. In other words, the virtual image corresponding to the target image increases as the viewing distance corresponding to the viewing position decreases, or the virtual image corresponding to the target image decreases as the viewing distance corresponding to the viewing position increases.

[0109] In this way, when the user is viewing from a first viewing position closer to the virtual image, they can see a larger virtual image of the target image. When the user is viewing from a second viewing position farther from the virtual image, they can see a smaller virtual image of the target image. Due to the limitations of the aperture stop, the closer the user is to the virtual image, the larger the display area they can see, while the farther the user is from the virtual image, the smaller the display area they can see. Therefore, by enlarging the virtual image of the target image when the user is closer to the virtual image and reducing it when the user is farther away from the virtual image, the user can always see the entire virtual image of the target image, while seeing less of, or even no, the virtual image corresponding to the background image. This achieves an expansion of the eyebox size in the depth direction (i.e., the direction of the principal light ray).

[0110] In other examples, the virtual image corresponding to the target image translates as the viewing position shifts, and the translation direction of the virtual image corresponding to the target image is opposite to the translation direction of the viewing position. Therefore, in the eyebox, the virtual image of the target image corresponding to the third viewing position is offset in a first direction relative to the virtual image of the target image corresponding to the fourth viewing position, with reference to the chief ray of the central field of view of the virtual image display device. The third viewing position is offset in a second direction relative to the fourth viewing position, with reference to the chief ray. The first direction and the second direction are opposite directions relative to the chief ray.

[0111] Here, the third viewing position is offset relative to the fourth viewing position in the second direction based on the main light, which means that the third viewing position is farther from the main light than the fourth viewing position in the second direction, and the second direction is the radial outward direction centered on the main light.

[0112] Because the user's viewing position deviates perpendicularly from the principal ray due to the limitations of the aperture, the virtual image they see moves in a direction perpendicular to the principal ray and opposite to the direction of the deviation. Thus, when the user views from a third viewing position, closer to the principal ray, the center of the virtual image of the target image is closer to the principal ray. Conversely, when the user views from a fourth viewing position, farther from the principal ray, the center of the virtual image of the target image is farther from the principal ray. This ensures that the user always sees the entire virtual image of the target image and cannot see the virtual image corresponding to the background image.

[0113] FIG8 is a flow chart of a method for generating image data according to an embodiment of the present application. This method is used to provide image data for an image to be displayed to the virtual image display device shown in FIG5 or FIG6. This method can be executed by a processor. The processor can be integrated into the virtual image display device, or can be an external device to the virtual image display device. As shown in FIG8, the method includes:

[0114] S801: Determine the eye position of a viewer of the virtual image display device, where the eye position is in the eye box of the virtual image display device.

[0115] Here, the eye position of the viewer may be the center position of both eyes of the viewer or the position of a single eye of the viewer, etc.

[0116] In some examples, an eye image of the viewer can be obtained by an acquisition device, and the viewer's eye position can be determined based on the acquired eye image. The acquisition device can be a red-green-blue (RGB) camera, a binocular camera, an infrared camera, or a depth camera. This application does not limit the method for determining the eye position, and any method in the relevant technology can be used. The acquisition device can be located in front of the viewer to facilitate obtaining the viewer's eye image. For example, it is set on the top of the housing of a desktop display device or the top of the windshield of a vehicle.

[0117] S802: Determine position indication information according to the eye position.

[0118] The position indication information is used to indicate the position of the target image in the image to be displayed. The position indication information indicates different positions of the target image corresponding to different viewing positions in the eye box. The position of the target image changes as the viewer's eye position changes.

[0119] In one possible embodiment, the position indication information indirectly indicates the position of the target image within the image to be displayed. For example, the position indication information includes position information of a first sub-display area, where the position information of the first sub-display area is used to indicate the position of the first sub-display area within the display area of ​​the virtual image display device. The first sub-display area is used to display a virtual image corresponding to the target image. Because the virtual image is formed by magnifying the image to be displayed, the position of the target image within the image to be displayed and the position of the virtual image corresponding to the target image within the entire display area have a one-to-one correspondence. Therefore, the position information of the first sub-display area can indirectly reflect the position of the target image within the image to be displayed.

[0120] In this embodiment, S802 includes: determining the position information of the first sub-display area corresponding to the eye position according to the correspondence between the viewing position and the position information of the sub-display area.

[0121] The correspondence includes multiple viewing positions in the eye box and position information of the sub-display area corresponding to each viewing position. Exemplarily, the position information of the sub-display area includes geometric information of the sub-display area and the position of a reference point in the sub-display area, and the geometric information of the sub-display area is used to indicate the area and / or shape of the sub-display area. When the shape of the sub-display area is a default shape, the geometric information of the sub-display area may only indicate the area of ​​the sub-display area. For example, the shape of the sub-display area is a rectangle, and the geometric information of the sub-display area includes the length and width of the sub-display area. For another example, the shape of the sub-display area is a circle, and the geometric information of the sub-display area includes the diameter or radius of the sub-display area. The number of reference points can be one or more, for example, the reference point can be the center point of the sub-display area, or the two end points on the diagonal of a rectangular sub-display area, or the two end points of the diameter of a circular sub-display area, etc. The present application does not limit the method of the position information of the sub-display area, as long as the shape, size and position of the sub-display area in the entire display area can be determined.

[0122] In some examples, in the corresponding relationship, the area of ​​the sub-display region corresponding to the first viewing position is larger than the area of ​​the sub-display region corresponding to the second viewing position, and the viewing distance corresponding to the first viewing position is smaller than the viewing distance corresponding to the second viewing position.

[0123] For example, as shown in Figure 9, assuming A1 is the first viewing position and A2 is the second viewing position, the viewing distance corresponding to A1 is smaller than the viewing distance corresponding to A2. The area of ​​sub-display region X1 corresponding to A1 is larger than the area of ​​sub-display region X2 corresponding to X2. For another example, assuming A2 is the first viewing position and A3 is the second viewing position, the viewing distance corresponding to A2 is smaller than the viewing distance corresponding to A3. The area of ​​sub-display region X2 corresponding to A2 is larger than the area of ​​sub-display region X3 corresponding to A3.

[0124] Part (a) of Figure 10 is a schematic diagram showing the relationship between the first sub-display area corresponding to viewing position A2 and the entire display area. Part (b) of Figure 10 is a schematic diagram showing the relationship between the first sub-display area corresponding to viewing position A1 and the entire display area. Part (c) of Figure 10 is a schematic diagram showing the relationship between the first sub-display area corresponding to viewing position A3 and the entire display area. As can be seen from Figure 10, the first sub-display area corresponding to A1 is obtained by enlarging the first sub-display area corresponding to A2; the first sub-display area corresponding to A3 is obtained by reducing the first sub-display area corresponding to A2.

[0125] In other examples, in the corresponding relationship, the position of the sub-display area corresponding to the third viewing position (which can be the position of a reference point in the sub-display area) is offset in a first direction relative to the position of the sub-display area corresponding to the fourth viewing position, with the principal ray of the central field of view of the virtual image display device as a reference, and the third viewing position is offset in a second direction relative to the fourth viewing position with the principal ray as a reference, and the first direction and the second direction are two opposite directions with the principal ray as a reference.

[0126] For example, as shown in Figure 11, assuming A1 is the third viewing position and A2 is the fourth viewing position, and based on the chief ray O, A1 is offset to the left relative to A2, then the sub-display area X1 corresponding to A1 is offset to the right relative to the sub-display area X2 corresponding to A2. For another example, assuming A3 is the third viewing position and A2 is the fourth viewing position, and based on the chief ray O, A3 is offset to the right relative to A2, then the sub-display area X3 corresponding to A3 is offset to the left relative to the sub-display area X2 corresponding to A2.

[0127] Part (a) of Figure 12 is a schematic diagram showing the relationship between the first sub-display area corresponding to viewing position A2 and the entire display area. Part (b) of Figure 12 is a schematic diagram showing the relationship between the first sub-display area corresponding to viewing position A1 and the entire display area. Part (c) of Figure 12 is a schematic diagram showing the relationship between the first sub-display area corresponding to viewing position A3 and the entire display area. As can be seen from Figure 12, the first sub-display area corresponding to A1 is obtained by shifting the first sub-display area corresponding to A2 to the right; the first sub-display area corresponding to A3 is obtained by shifting the first sub-display area corresponding to A2 to the left.

[0128] In the embodiment of the present application, the correspondence between the viewing position and the position information of the sub-display area can be obtained based on the field of view of the optical system of the virtual image display device and the geometric relationship between each viewing position in the eye box.

[0129] In another possible implementation, the position indication information directly indicates the position of the target image in the image to be displayed. For example, the position indication information includes the position information of the target image.

[0130] In this embodiment, S802 includes: determining the position information of the first image area corresponding to the eye position according to the correspondence between the viewing position and the position information of the image area.

[0131] This correspondence includes multiple viewing positions within the eyebox and positional information of the image areas corresponding to each viewing position. Exemplarily, the positional information of the image areas includes geometric information of the image areas and the position of a reference point within the image areas. The geometric information of the image areas indicates the area and / or shape of the first sub-display area. The positional information of the image areas can be found in the aforementioned positional information of the sub-display areas, and a detailed description thereof is omitted here.

[0132] In this embodiment, the correspondence between the viewing position and the position information of the sub-display area can be obtained based on the field of view of the optical system of the virtual image display device and the geometric relationship between the various viewing positions in the eye box; and then the correspondence between the viewing position and the position information of the sub-display area can be converted into the correspondence between the viewing position and the position information of the image area.

[0133] S803: Generate image data of the image to be displayed according to the image data of the target image and the position indication information.

[0134] When the position indication information indicates that the target image is located in the entire image area of ​​the image to be displayed, that is, the image to be displayed only includes the target image, S803 includes: using image data of the target image as image data of the image to be displayed.

[0135] When the position indication information indicates that the target image is located in a partial image area of ​​the image to be displayed, that is, the image to be displayed includes the target image and a background image, S803 includes: combining the image data of the target image and the image data of the background image to obtain image data of the image to be displayed, with the target image located at the position indicated by the position indication information. In some examples, the background image is located in an area of ​​the image to be displayed outside the area where the target image is located. In other examples, the background image is located in the entire image area of ​​the image to be displayed, and the target image is located above the background image.

[0136] In some examples, if the content of the background image is associated with the content of the target image, then in S803, the image data of the background image is first processed to obtain the image data of the background image, and then the image data of the target image and the image data of the background image are combined. Here, the methods for processing the target image include but are not limited to blurring, blurring, and increasing transparency, and one or more of these methods can be used.

[0137] In other examples, if the content of the background image is unrelated to the content of the target image, then in S803 , the image data of the background image is first acquired, and then the image data of the target image and the image data of the background image are combined.

[0138] In the embodiment of the present application, the manner of combining the image data of the target image and the image data of the background image includes any of the following:

[0139] The first method is to obtain partial image data corresponding to a second image area in the image data of the background image, where the second image area is the area of ​​the image to be displayed excluding the first image area, and the first image area is the area where the target image is located; and to splice the partial image data with the image data of the target image to obtain image data of the image to be displayed, so that the target image is located at the position indicated by the position indication information.

[0140] The second method is to use the image data of the target image to replace the image data of the background image corresponding to the position indicated by the position indication information;

[0141] The third method is to superimpose and combine the image data of the target image and the image data of the background image so that the target image is located above the background image and at the position indicated by the position indication information.

[0142] Because the virtual image display device includes multiple optical elements, these optical elements can cause distortion in the virtual image displayed by the virtual image display device. Furthermore, the degree of distortion varies depending on the sub-display areas of the display area. Therefore, it is necessary to perform distortion correction on the target image based on the position of the first sub-display area used to display the virtual image of the target image. Specifically, the method may further include performing distortion correction processing on the image data of the target image based on the eye position or position indication information.

[0143] Exemplarily, distortion correction processing is performed on the image data of the target image based on the eye position or position indication information, including: obtaining distortion correction parameters corresponding to the eye position or position indication information, the distortion correction parameters being used to compensate for the distortion caused by the optical elements of the virtual image display device; and using the distortion correction parameters to perform distortion correction processing on the image data of the target image.

[0144] When distortion correction is performed on the target image data according to the eye position, a correspondence between the viewing position and the distortion correction parameters can be pre-established. Based on the correspondence between the viewing position and the distortion correction parameters, the distortion correction parameters corresponding to the eye position are determined.

[0145] In some examples, the correspondence between the viewing position and the distortion correction parameters includes two sub-correspondences: a correspondence between the viewing position and the position of a sub-display area, and a correspondence between the position of the sub-display area and the distortion correction parameters. Based on the correspondence between the viewing position and the position of the sub-display area, the position of the first sub-display area corresponding to the eye position can be determined. Then, based on the correspondence between the position of the sub-display area and the distortion correction parameters, the distortion correction parameters corresponding to the position of the first sub-display area are determined. The distortion correction parameters corresponding to the position of the first sub-display area are the distortion correction parameters corresponding to the eye position.

[0146] In other examples, the correspondence between the viewing position and the distortion correction parameter can be a direct correspondence between the viewing position and the distortion correction parameter, and the distortion correction parameter corresponding to the eye position is determined based on the direct correspondence. The direct correspondence can be generated based on the two sub-correspondences described above.

[0147] When performing distortion correction processing on the image data of the target image based on the position indication information, a correspondence between the position of the image region and the distortion correction parameters can be pre-established. Based on the correspondence between the position of the image region and the distortion correction parameters, the distortion correction parameters corresponding to the eye position are determined.

[0148] In some examples, the correspondence between the position of the image area and the distortion correction parameters includes two sub-correspondences: a correspondence between the position of the image area and the position of the sub-display area, and a correspondence between the position of the sub-display area and the distortion correction parameters. Based on the correspondence between the position of the image area and the position of the sub-display area, the position of the first sub-display area corresponding to the position indicated by the position indication information can be determined. Then, based on the correspondence between the position of the sub-display area and the distortion correction parameters, the distortion correction parameters corresponding to the position of the first sub-display area are determined. The distortion correction parameters corresponding to the position of the first sub-display area are the distortion correction parameters corresponding to the position indication information.

[0149] In other examples, the correspondence between the position of the image region and the distortion correction parameter may be a direct correspondence between the position of the image region and the distortion correction parameter, and the distortion correction parameter corresponding to the position indicated by the position indication information is determined based on the direct correspondence. The direct correspondence may be generated based on the two sub-correspondences described above.

[0150] Exemplarily, the correspondence between the position of the sub-display area and the distortion correction parameter can be obtained in the following manner.

[0151] First, the distortion grid parameters corresponding to the entire display area of ​​the virtual image display device are obtained through testing and / or mathematical simulation. For example, the distortion grid parameters corresponding to the entire display area of ​​the virtual image display device are determined based on the surface type (including but not limited to spherical, aspherical, free-form surface, etc.), surface parameters (including but not limited to curvature, ellipticity coefficient, asphericity coefficient, free-form surface polynomial coefficient, etc.) of each optical element in the optical system of the virtual image display device, and the size of each optical element. The distortion grid parameters are a distortion grid composed of an array of NxM points, where N and M are integers, and each point represents the actual position of the virtual image formed by a certain field of view light after passing through the optical system.

[0152] Then, based on the distortion grid parameters and the ideal grid parameters, the distortion correction grid parameters corresponding to the entire display area are determined. Here, the ideal grid parameters are a distortion grid composed of an NxM array of points, with the same number of points as the distortion grid, and a one-to-one correspondence between each point. Each point in the ideal grid represents the ideal position of the virtual image formed by a particular field of view ray after passing through the ideal optical system. For example, by subtracting the distortion grid parameters from the ideal grid parameters point by point, the distortion correction parameters corresponding to the display area can be obtained. These distortion correction parameters are for the distortion correction grid composed of an NxM array of points.

[0153] According to the position of the first sub-display area, the distortion correction parameters of some grid points are selected from the distortion correction grid, and the target image is corrected using the distortion correction parameters corresponding to the selected grid points, thereby achieving distortion correction for a certain human eye observation point position.

[0154] When the optical performance of the virtual image display device is relatively ideal, the distortion correction process can be omitted.

[0155] FIG13 is a schematic diagram of the structure of an image data generation device provided by an exemplary embodiment of the present application. This device can be implemented as part or all of a device through software, hardware, or a combination of both. The device provided by this embodiment of the present application can implement the process of FIG8 of the embodiment of the present application. As shown in FIG13 , the device 1300 includes an eye position determination module 1301, an information determination module 1302, and a generation module 1303. The eye position determination module 1301 is used to determine the eye position of a viewer of a virtual image display device, where the eye position is located in the eye box of the virtual image display device. The information determination module 1302 is used to determine position indication information based on the eye position. The position indication information is used to indicate the position of a target image in the image to be displayed, and the position of the target image changes with the viewer's eye position. The generation module 1303 is used to generate image data for the image to be displayed based on the image data of the target image and the position indication information. The image data of the image to be displayed is used by the virtual image display device to form a virtual image in the display area.

[0156] In one possible implementation, the position indication information includes position information of a first sub-display area, the position information of the first sub-display area being used to indicate a position of the first sub-display area within a display area of ​​the virtual image display device, the first sub-display area being used to display a virtual image corresponding to the target image. The information determination module 1302 is configured to determine the position information of the first sub-display area corresponding to the eye position based on a correspondence between the viewing position and the position information of the sub-display areas.

[0157] In another possible embodiment, the position indication information includes position information of a first image area, where the first image area is the area where the target image is located; the relationship acquisition submodule 13021 is used to determine the position information of the first image area corresponding to the eye position based on the correspondence between the viewing position and the position information of the image area.

[0158] In some examples, the generating module 1303 is configured to combine image data of a target image and image data of a background image to obtain image data of an image to be displayed, and the target image is located at a position indicated by the position indication information.

[0159] Optionally, the apparatus further includes: a correction module 1304. The correction module 1304 is configured to perform distortion correction processing on the image data of the target image according to the eye position or the position indication information.

[0160] Exemplarily, the correction module 1304 includes a parameter acquisition submodule 13041 and a correction submodule 13042. The parameter acquisition submodule 13041 is configured to obtain distortion correction parameters corresponding to the eye position or the position indication information, wherein the distortion correction parameters are used to compensate for distortion caused by optical elements of the virtual image display device. The correction submodule 13042 is configured to perform distortion correction processing on the image data of the target image using the distortion correction parameters.

[0161] It should be noted that the image data generation device provided in the above embodiments only uses the division of the above functional modules as an example to illustrate the generation of image data. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the image data generation device provided in the above embodiments and the image data generation method embodiment are based on the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0162] The division of modules in the embodiments of the present application is illustrative and is merely a logical functional division. In actual implementation, other division methods may be used. Furthermore, the functional modules in the various embodiments of the present application may be integrated into a single processor, or may exist physically separately, or two or more modules may be integrated into a single module. The aforementioned integrated modules may be implemented in the form of hardware or software functional modules.

[0163] If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including a number of instructions for enabling a terminal device (which can be a personal computer, mobile phone, or communication device, etc.) or a processor to execute all or part of the steps of the method of each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, and other media that can store program code.

[0164] The present application also provides a computer device. FIG14 exemplarily provides a possible architecture diagram of a computer device 1400.

[0165] The computer device 1400 includes a memory 1401 , a processor 1402 , a communication interface 1403 , and a bus 1404 . The memory 1401 , the processor 1402 , and the communication interface 1403 are connected to each other via the bus 1404 .

[0166] Memory 1401 can be a ROM, static storage device, dynamic storage device, or RAM. Memory 1401 can store programs. When the program stored in memory 1401 is executed by processor 1402, processor 1402 and communication interface 1403 are used to perform the device access method. Memory 1401 can also store data sets. For example, a portion of the storage resources in memory 1401 can be divided into a data storage module for storing image data, etc.

[0167] The processor 1402 may be a general-purpose CPU, a microprocessor, an application-specific integrated circuit (ASIC), a graphics processing unit (GPU), or one or more integrated circuits.

[0168] Processor 1402 may also be an integrated circuit chip with signal processing capabilities. During implementation, some or all of the functions of the signal processing device of the present application may be performed by hardware integrated logic circuits or software instructions in processor 1402. The aforementioned processor 1402 may also be a general-purpose processor, a digital signal processor (DSP), an ASIC, a field programmable gate array (FPGA), or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component. It may implement or execute the various methods disclosed in the above embodiments of the present application. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in conjunction with the embodiments of the present application may be directly implemented and executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software modules may be located in a storage medium well-known in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or the like. The storage medium is located in the memory 1401 , and the processor 1402 reads the information in the memory 1401 and completes part of the functions of the signal processing device of the embodiment of the present application in combination with its hardware.

[0169] The communication interface 1403 uses a transceiver module such as, but not limited to, a transceiver to implement communication between the computer device 1400 and other devices or a communication network. For example, image data can be received through the communication interface 1403.

[0170] The bus 1404 may include a path for transmitting information between various components of the computer device 1400 (eg, the memory 1401 , the processor 1402 , and the communication interface 1403 ).

[0171] The descriptions of the processes corresponding to the above figures have different focuses. For parts that are not described in detail in a certain process, please refer to the relevant descriptions of other processes.

[0172] In an embodiment of the present application, a computer-readable storage medium is also provided, which stores computer instructions. When the computer instructions stored in the computer-readable storage medium are executed by a computer device, the computer device executes the above-mentioned method for generating image data.

[0173] In an embodiment of the present application, a computer program product including instructions is further provided, which, when executed on a computer device, enables the computer device to execute the above-mentioned method for generating image data.

[0174] In an embodiment of the present application, a chip is also provided for executing the method for generating image data shown in FIG8 .

[0175] The present application also provides a display device including a processor and a virtual image display device, wherein the virtual image display device is any of the aforementioned virtual image display devices. The processor is configured to generate image data of an image to be displayed based on the method shown in FIG8 . The virtual image display device is configured to display a virtual image based on the image data.

[0176] Optionally, the display device further includes a power supply for supplying power to the processor and the PGU.

[0177] In some examples, the display device is a heads-up display device or a desktop display device.

[0178] Figure 15 is a circuit diagram of a display device provided by an embodiment of the present application. As shown in Figure 15, the circuit in the display device mainly includes a host processor (host CPU) 1501, an external memory interface 1502, an internal memory 1503, an audio module 1504, a video module 1505, a power module 1506, a wireless communication module 1507, an I / O interface 1508, a video interface 1509, a display circuit 1510 and a modulator 1511, etc. Wherein, the main processor 1501 and its peripheral components, such as an external memory interface 1502, an internal memory 1503, an audio module 1504, a video module 1505, a power module 1506, a wireless communication module 1507, an I / O interface 1508, a video interface 1509, a display circuit 1510 can be connected by a bus. The main processor 1501 can be called a front-end processor.

[0179] In addition, the circuit diagrams shown in the embodiments of the present application do not constitute specific limitations on the display device. In other embodiments of the present application, the display device may include more or fewer components than shown, or may combine or separate certain components, or arrange the components differently. The components shown in the diagrams may be implemented in hardware, software, or a combination of software and hardware.

[0180] The main processor 1501 includes one or more processing units. For example, the main processor 1501 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural network processor (NPU). Different processing units may be independent devices or integrated into one or more processors.

[0181] The main processor 1501 may also include a memory for storing instructions and data. In some embodiments, the memory in the main processor 1501 is a cache memory. This memory can store instructions or data that the main processor 1501 has just used or is reusing. If the main processor 1501 needs to use the instruction or data again, it can directly call it from the memory. This avoids duplicate accesses, reduces the waiting time of the main processor 1501, and thus improves system efficiency.

[0182] In some embodiments, the display device may further include multiple input / output (I / O) interfaces 1508 connected to the main processor 1501. The interfaces 1508 may include an I2C (Inter-Integrated Circuit) interface, an I2S (Inter-Integrated Circuit Sound) interface, a PCM (Pulse Code Modulation) interface, a UART (Universal Asynchronous Receiver / Transmitter) interface, a MIPI (Mobile Industry Processor Interface) interface, a GPIO (General-Purpose Input / Output) interface, a SIM (Subscriber Identity Module) interface, and / or a USB (Universal Serial Bus) interface. The I / O interfaces 1508 may be connected to devices such as a mouse, touchpad, keyboard, camera, speaker, microphone, etc., or to physical buttons on the display device (e.g., volume button, brightness adjustment button, power button, etc.).

[0183] The external memory interface 1502 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the display device. The external memory card communicates with the main processor 1501 through the external memory interface 1502 to implement data storage function.

[0184] The internal memory 1503 can be used to store computer executable program codes, and the executable program codes include instructions. The internal memory 1503 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a call function, a time setting function, etc.), etc. The data storage area may store data created during the use of the display device (such as a phone book, world time, etc.), etc. In addition, the internal memory 1503 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash memory (Universal Flash Storage, UFS), etc. The main processor 1501 executes various functional applications and data processing of the display device by running instructions stored in the internal memory 1503 and / or instructions stored in a memory provided in the main processor 1501.

[0185] The display device can implement audio functions such as music playback and calls through the audio module 1504 and the application processor.

[0186] The audio module 1504 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 1504 can also be used to encode and decode audio signals, such as for playing or recording. In some embodiments, the audio module 1504 can be provided in the processor 101, or some functional modules of the audio module 1504 can be provided in the processor 101.

[0187] The video interface 1509 can receive external audio and video signals, which can specifically be a High Definition Multimedia Interface (HDMI), a Digital Visual Interface (DVI), a Video Graphics Array (VGA), a Display Port (DP), etc. The video interface 1509 can also output video to the outside. When the display device is used as a head-up display, the video interface 1509 can receive speed signals and power signals input from peripheral devices, and can also receive external AR video signals. When the display device is used as a projector, the video interface 1509 can receive video signals input from an external computer or terminal device.

[0188] The video module 1505 can decode the video input from the video interface 1509, for example, by performing H.264 decoding. The video module can also encode the video captured by the display device, for example, by performing H.264 encoding on the video captured by an external camera. Furthermore, the main processor 1501 can also decode the video input from the video interface 1509 and output the decoded image signal to the display circuit 1510.

[0189] The display circuit 1510 and modulator 1511 are used to display corresponding images. In this embodiment, the video interface 1509 receives an external video source signal, which the video module 1505 decodes and / or digitizes before outputting one or more image signals to the display circuit 1510. The display circuit 1510 drives the modulator 1511 based on the input image signal to image the incident polarized light, thereby outputting at least two third light beams. Furthermore, the main processor 1501 may also output one or more image signals to the display circuit 1510.

[0190] In this embodiment, the display circuit 1510 and the modulator 1511 are electronic components in the modulation unit 230 , and the display circuit 1510 can be referred to as a driving circuit.

[0191] The power module 1506 is used to provide power to the main processor 1501 and the light source 151 based on input power (e.g., direct current). The power module 1506 may include a rechargeable battery, which can provide power to the main processor 1501 and the light source 151. The light emitted by the light source 151 can be transmitted to the modulator 1511 for imaging, thereby forming an image light signal.

[0192] The wireless communication module 1507 enables the display device to communicate wirelessly with the outside world. It can provide wireless local area networks (WLAN) (such as Wireless Fidelity (Wi-Fi) networks), Bluetooth (BT), Global Navigation Satellite System (GNSS), Frequency Modulation (FM), Near Field Communication (NFC), infrared (IR) and other wireless communication solutions. The wireless communication module 1507 can be one or more devices that integrate at least one communication processing module. The wireless communication module 1507 receives electromagnetic waves via an antenna, frequency modulates and filters the electromagnetic wave signals, and sends the processed signals to the main processor 1501. The wireless communication module 1507 can also receive signals to be sent from the main processor 1501, frequency modulate them, amplify them, and convert them into electromagnetic waves for radiation through the antenna.

[0193] In addition, in addition to being input through the video interface 1509, the video data decoded by the video module 1505 can also be received wirelessly through the wireless communication module 1507 or read from an external memory. For example, the display device can receive video data from a terminal device or an in-vehicle entertainment system through the wireless LAN in the vehicle, and the display device can also read audio and video data stored in an external memory.

[0194] The present application also provides a vehicle, which includes any of the aforementioned display devices. The aforementioned display device can be installed on the vehicle. Please refer to Figure 16, which is a schematic diagram of a possible functional framework of a vehicle provided by the present application.

[0195] As shown in FIG16 , the functional framework of a vehicle may include various subsystems, such as the illustrated sensor system 12, a control system 14, one or more peripheral devices 16 (one of which is shown as an example), a power supply 18, a computer system 20, and a head-up display system 32. Optionally, the vehicle may also include other functional systems, such as an engine system that provides power to the vehicle, etc., which are not limited in this application.

[0196] The sensor system 12 may include a plurality of detection devices that sense the information being measured and convert the sensed information into electrical signals or other required information outputs according to certain rules. As shown in the figure, these detection devices may include a global positioning system (GPS), a vehicle speed sensor, an inertial measurement unit (IMU), a radar unit, a laser rangefinder, a camera, a wheel speed sensor, a steering sensor, a gear position sensor, or other components for automatic detection, etc., and this application does not limit them.

[0197] The control system 14 may include several components, such as a steering unit, a braking unit, a lighting system, an autonomous driving system, a map navigation system, a network timing system, and an obstacle avoidance system, as shown. Optionally, the control system 14 may also include components such as a throttle controller and an engine controller for controlling vehicle speed, although this application does not limit this.

[0198] The peripheral devices 16 may include several components, such as the communication system shown in the figure, a touch screen, a user interface, a microphone, and a speaker. The communication system is used to enable network communication between the vehicle and other devices. In practical applications, the communication system may utilize wireless communication technology or wired communication technology to enable network communication between the vehicle and other devices. Wired communication technology may involve communication between the vehicle and other devices via network cables or optical fibers.

[0199] Power supply 18 represents a system that provides electrical power or energy to the vehicle, and may include, but is not limited to, rechargeable lithium batteries or lead-acid batteries. In practical applications, one or more battery components in the power supply are used to provide electrical energy or energy for starting the vehicle. The type and material of the power supply are not limited in this application.

[0200] Several functions of the vehicle are controlled and implemented by the computer system 20. The computer system 20 may include one or more processors 2001 (one processor is shown as an example) and a memory 2002 (also referred to as a storage device). In actual applications, the memory 2002 is also inside the computer system 20, or it can be outside the computer system 20, for example, as a cache in the vehicle, etc., which is not limited in this application.

[0201] The processor 2001 may include one or more general-purpose processors, such as a graphics processing unit (GPU). The processor 2001 may be used to run relevant programs or instructions corresponding to the programs stored in the memory 2002 to implement corresponding functions of the vehicle.

[0202] The memory 2002 may include a volatile memory, such as RAM; the memory may also include a non-volatile memory, such as ROM, flash memory, HDD or solid-state drive SSD; the memory 2002 may also include a combination of the above types of memory. The memory 2002 can be used to store a set of program codes or instructions corresponding to the program codes, so that the processor 2001 can call the program codes or instructions stored in the memory 2002 to implement the corresponding functions of the vehicle. The function includes but is not limited to some or all of the functions in the vehicle function framework diagram shown in Figure 16. In the present application, a set of program codes for vehicle control can be stored in the memory 2002, and the processor 2001 can call the program code to control the safe driving of the vehicle. How to achieve safe driving of the vehicle is described in detail below in this application.

[0203] Optionally, in addition to storing program code or instructions, memory 2002 may also store information such as road maps, driving routes, and sensor data. Computer system 20 may integrate with other components in the vehicle functional framework diagram, such as sensors and GPS in the sensor system, to implement relevant vehicle functions. For example, computer system 20 may control the vehicle's direction or speed based on data input from sensor system 12, although this application does not limit this.

[0204] The head-up display system 32 may include several components, such as a windshield, a controller, and a head-up display (i.e., the aforementioned head-up display device). The controller is used to generate images (e.g., images containing vehicle status such as speed, battery / fuel level, and augmented reality (AR) content) in response to user instructions and send these images to the head-up display for display. The head-up display may include an image generation unit and a reflective component. The windshield is used to cooperate with the head-up display to implement the optical path of the head-up display system, so that the target image is presented in front of the driver. It should be noted that the functions of some components in the head-up display system may also be implemented by other subsystems of the vehicle. For example, the controller may also be a component of the control system.

[0205] FIG16 of this application illustrates four subsystems: sensor system 12, control system 14, computer system 20, and head-up display system 32. These subsystems are merely illustrative and not limiting. In practice, a vehicle may combine several components according to different functions to create subsystems with corresponding functions. In practice, a vehicle may include more or fewer systems or components, and this application does not limit this.

[0206] The above-mentioned transportation vehicles can be cars, trucks, motorcycles, buses, ships, airplanes, helicopters, lawn mowers, recreational vehicles, amusement park vehicles, construction equipment, trams, golf carts, trains, and carts, etc., and the embodiments of the present application do not make special limitations.

[0207] Unless otherwise defined, the technical or scientific terms used herein shall have the usual meanings understood by persons of ordinary skill in the field to which the present disclosure belongs. The words “first”, “second”, “third” and similar terms used in the patent application specification and claims of the present disclosure do not indicate any order, quantity or importance, but are only used to distinguish different components. Similarly, words such as “one” or “a” do not indicate a quantity limitation, but rather indicate the presence of at least one. Words such as “include” or “comprising” and similar terms mean that the elements or objects appearing before “include” or “comprising” cover the elements or objects listed after “include” or “comprising” and their equivalents, and do not exclude other elements or objects.

[0208] The above description is only one embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made on the basis of the present application shall be included in the scope of protection of the present application.

Claims

1. A virtual image display device, characterized in that: comprising an image generating unit and an imaging unit, The image generating unit is used to acquire image data, display an image based on the image data, and output image light, the image displayed by the image generating unit includes a target image, the position of the target image in the displayed image corresponds to the viewing position in the eye box of the virtual image display device, and the position of the target image in the displayed image changes as the viewing position changes; The imaging unit is used to form a virtual image in a display area based on the image light.

2. The virtual image display device according to claim 1, characterized in that: The viewing distance corresponding to the viewing position in the eye box is greater than the viewing distance corresponding to the reference point, and the reference point is the vertex of the first viewing angle corresponding to the display area.

3. The virtual image display device according to claim 2, characterized in that: The display area includes a first sub-display area for displaying a virtual image corresponding to the target image; When the center of the first sub-display area coincides with the center of the display area, a ratio of the first distance to the second distance is within a ratio range, the first distance is the distance between the reference point and the center point, the second distance is the distance between the center position of the eye box and the center point, the center position of the eye box is located on the main ray of the central field of view of the virtual image display device, the center point is the intersection of the plane where the intersection of the first edge ray and the second edge ray is located and the main ray, the first edge ray is the edge ray corresponding to the first field of view angle, and the second edge ray is the edge ray corresponding to the second field of view angle corresponding to the first sub-display area; Wherein, the ratio range is [20%, 90%].

4. The virtual image display device according to any one of claims 1 to 3, characterized in that: In the eye box, the area of ​​the target image corresponding to the first viewing position is larger than the area of ​​the target image corresponding to the second viewing position, and the viewing distance corresponding to the first viewing position is smaller than the viewing distance corresponding to the second viewing position; and / or, In the eye box, the virtual image of the target image corresponding to the third viewing position is offset in a first direction relative to the virtual image of the target image corresponding to the fourth viewing position, with the principal ray of the central field of view of the virtual image display device as a reference. The third viewing position is offset in a second direction relative to the fourth viewing position, with the principal ray as a reference. The first direction and the second direction are two opposite directions with the principal ray as a reference.

5. The virtual image display device according to any one of claims 1 to 4, characterized in that: The virtual image corresponding to the target image increases as the viewing distance corresponding to the viewing position decreases, or the virtual image corresponding to the target image decreases as the viewing distance corresponding to the viewing position increases; and / or, The virtual image corresponding to the target image translates as the viewing position translates, and the translation direction of the virtual image corresponding to the target image is opposite to the translation direction of the viewing position.

6. The virtual image display device according to any one of claims 1 to 5, characterized in that: The displayed image also includes a background image, which is an image other than the target image in the displayed image, and is a static image or a dynamic image.

7. The virtual image display device according to any one of claims 1 to 6, characterized in that: The virtual image includes a virtual image corresponding to the target image and a virtual image corresponding to a background image. The virtual image corresponding to the background image is outside the virtual image corresponding to the target image or the virtual image corresponding to the target image covers at least a portion of the virtual image corresponding to the background image.

8. The virtual image display device according to any one of claims 1 to 7, characterized in that: The virtual image corresponding to the target image is a three-dimensional image or a two-dimensional image.

9. A method for generating image data, characterized in that: include: determining an eye position of a viewer of the virtual image display device, the eye position being located in an eye box of the virtual image display device; Determining position indication information according to the eye position, wherein the position indication information is used to indicate the position of a target image in an image to be displayed, and the position of the target image changes as the eye position of the viewer changes; Image data of the image to be displayed is generated according to the image data of the target image and the position indication information, and the image data of the image to be displayed is used for the virtual image display device to form a virtual image in a display area.

10. The method according to claim 9, characterized in that The position indication information includes position information of a first sub-display area, the position information of the first sub-display area is used to indicate the position of the first sub-display area in the display area of ​​the virtual image display device, and the first sub-display area is used to display the virtual image corresponding to the target image; Determining position indication information according to the eye position includes: The position information of the first sub-display area corresponding to the eye position is determined according to the correspondence between the viewing position and the position information of the sub-display area.

11. The method according to claim 10, characterized in that In the corresponding relationship, the area of ​​the sub-display region corresponding to the first viewing position is larger than the area of ​​the sub-display region corresponding to the second viewing position, and the viewing distance corresponding to the first viewing position is smaller than the viewing distance corresponding to the second viewing position; and / or, In the corresponding relationship, the position of the sub-display area corresponding to the third viewing position is offset in a first direction relative to the position of the sub-display area corresponding to the fourth viewing position, with the principal ray of the central field of view of the virtual image display device as a reference, and the third viewing position is offset in a second direction relative to the fourth viewing position with the principal ray as a reference, and the first direction and the second direction are two opposite directions with the principal ray as a reference.

12. The method according to claim 9, characterized in that The position indication information includes position information of a first image area, where the first image area is the area where the target image is located; Determining position indication information according to the eye position includes: According to the correspondence between the viewing position and the position information of the image area, the position information of the first image area corresponding to the eye position is determined.

13. The method according to any one of claims 9 to 12, characterized in that: The step of generating the image data of the image to be displayed according to the image data of the target image and the position indication information comprises: The image data of the target image and the image data of the background image are combined to obtain the image data of the image to be displayed, and the target image is located at the position indicated by the position indication information.

14. The method according to any one of claims 9 to 13, characterized in that: The method further comprises: According to the eye position or the position indication information, distortion correction processing is performed on the image data of the target image.

15. The method according to claim 14, characterized in that The step of performing distortion correction processing on the image data of the target image according to the eye position or the position indication information comprises: Obtaining a distortion correction parameter corresponding to the eye position or the position indication information, wherein the distortion correction parameter is used to compensate for distortion caused by an optical element of the virtual image display device; The distortion correction parameters are used to perform distortion correction processing on the image data of the target image.

16. A computer device, characterized in that: The computer device comprises a processor and a memory; the memory is used to store a software program, and the processor executes the software program stored in the memory so that the computer device implements the method according to any one of claims 9 to 15.

17. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions. When the computer instructions in the computer-readable storage medium are executed by a computer device, the computer device executes the method according to any one of claims 9 to 15.

18. A display device, characterized in that: include: A processor and a virtual image display device; The processor is used to generate image data of an image to be displayed based on the method according to any one of claims 9 to 15; The virtual image display device is connected to the processor, and is used to display a virtual image based on the image data. The virtual image display device is the device according to any one of claims 1 to 8.

19. The display device according to claim 18, characterized in that The display device is a head-up display device or a desktop display device.

20. A means of transport, characterized in that: The display device according to claim 18 or 19 is installed on the vehicle.

Images