Image display system, method and VR device having the same

Abstract

The present invention provides an image display system, a method and a VR device having the image display system. The image display system includes an image processing module, a display device and a lens assembly. After collecting an actual object, the image processing module records the collection perspective, renders a three-dimensional model based on the collection perspective, constructs a virtual camera at the collection perspective, so that the virtual camera reads and samples the rendered image within the rendered area of the three-dimensional model after rendering. The three-dimensional model has a spatial angular frequency f1, the virtual camera has a reading angular frequency f2 when reading the rendered image, and a sampling angular frequency f3 when sampling the rendered image. The image processing module calculates the least common multiple of the ripple intervals at each frequency and adjusts the frequency until the least common multiple is less than x times that of the display device to correct the actual image. After adopting the above technical solution, the problem of the change of the sampling angular frequency after the projection-type optical system corrects the distorted image can be solved, and the occurrence of adverse phenomena such as image blurring or moiré can be avoided.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to an image display system, method and VR device having the image display system. Background Art

[0002] Due to the design principles of the optical system of projection equipment, the refraction phenomenon in the light path will cause the final image distortion. The distortion is related to the distance and angle relationship of the projected object, the shape of the projected object itself and other factors. Therefore, the actual image obtained is often deformed. Therefore, distorted images are often used to solve this type of distortion problem.

[0003] However, since the pixel density of the image is fixed before image processing, the sampling angular frequency changes after the distorted image, which easily leads to undesirable phenomena such as image blur or moiré. Moiré pattern is a kind of high-frequency interference stripes that appear on the photosensitive element of a digital camera or scanner. It is a kind of high-frequency irregular stripes that make the image appear colorful.

[0004] Therefore, a new image display system is needed to remove the moiré pattern of the corrected image as much as possible under the condition that the distorted image can be corrected. Summary of the invention

[0005] In order to overcome the above-mentioned technical defects, the purpose of the present invention is to provide an image display system, method and VR device with the image display system, so as to solve the problem of change of sampling angular frequency after the projection optical system corrects the distorted image and avoid the occurrence of undesirable phenomena such as image blur or moiré.

[0006] The present invention discloses an image display system, comprising a camera device for collecting display objects, an image processing module and a display device connected to the camera device, and a lens assembly arranged between the display device and human eyes.

[0007] In an actual use environment, a camera device captures an actual object at a capture angle of view and forms an actual image, an image processing module records the capture angle of view, renders a three-dimensional model based on the capture angle of view, and constructs a virtual camera on the capture angle of view, so that the virtual camera reads and samples a rendered image in a rendering area of ​​the three-dimensional model after rendering, wherein the three-dimensional model has a spatial angular frequency f1, and the virtual camera has a reading angular frequency f2 corresponding to the capture angle of view when reading the rendered image, and has a sampling angular frequency f3 corresponding to the capture angle of view when sampling the rendered image.

[0008] The image processing module calculates a common multiple of any two of a first ripple interval d1 of the rendering area at the spatial angular frequency f1, a second ripple interval d2 of the rendering area at the reading angular frequency f2, a third ripple interval d3 of the rendering area at the sampling angular frequency f3, a fourth ripple interval d4 of the rendering area at the device angular frequency f4 of the display device, and a fifth ripple interval d5 of the rendering area at the imaging angular frequency f5 of the display light emitted by the display device after being imaged by the lens assembly;

[0009] The image processing module adjusts two of the spatial angular frequency f1, the reading angular frequency f2, the sampling angular frequency f3, the device angular frequency f4, and the imaging angular frequency f5 selected to form the common multiple until the common multiple is less than x times of the display device, and stores the result;

[0010] The image processing module corrects the actual image based on the adjusted spatial angular frequency f1, the reading angular frequency f2, the sampling angular frequency f3, the device angular frequency f4, and the imaging angular frequency f5.

[0011] Preferably, in a test environment, a camera device collects a test image of a test object, and an image processing module forms a distorted image for the test image based on an imaging relationship between a display device and a lens assembly;

[0012] The image processing module calculates and stores the mapping relationship between each pixel point in the test image and each corresponding pixel point in the distorted image.

[0013] Preferably, the image processing module obtains the simulation result of the test image and records it as a mapping relationship, or

[0014] The image processing module calculates the pixel angle deviation between the pixel points in the test image and the pixel points in the distorted image on both sides of the symmetry axis of the X axis and / or the Y axis, and records the pixel angle deviation as a mapping relationship;

[0015] The image processing module regards the distortion result of the actual image as the original image and the actual image as the processed image, and calculates the inverse mapping relationship of changing each pixel position in the original image to each pixel position in the processed image;

[0016] The image processing module stretches each pixel position of the actual image based on the inverse mapping relationship, and replaces the actual image in the above steps with the formed corrected image.

[0017] Preferably, the pixel points of the rendering area are distributed as m*n, where m and n are any integers between 100 and 1,000,000 respectively;

[0018] The image processing module corrects the test image based on the mapping relationship and detects the imaging effect of the test object.

[0019] Preferably, the imaging relationship includes:

[0020] The optical axis of the display device coincides with the optical axis of the lens assembly, or

[0021] The optical axis of the display device is parallel to the optical axis of the lens assembly, or

[0022] The optical axis of the display device forms an angle with the optical axis of the lens assembly, and the optical axis of the lens assembly passes through the optical center of the display device, or

[0023] The optical axis of the display device forms an angle with the optical axis of the lens assembly, and the optical axis of the lens assembly deviates from the optical center of the display device.

[0024] Preferably, the angle is 0.1°-60°;

[0025] When the optical axis of the display device is parallel to the optical axis of the lens assembly, the distance between the optical axis of the display device and the optical axis of the lens assembly is 0.1 mm-300 mm;

[0026] The display device is configured to have a number of pixels in a single display direction of 320-102400, and a size of the display device is between 1 inch and 3.4 inches;

[0027] The distortion map pixels of the distorted image account for 2%-50% of all pixels of the test image;

[0028] The lens assembly has a magnification of 3-3000;

[0029] x is 10 -5 ~10 5 .

[0030] The present invention also discloses an image display method, comprising the following steps:

[0031] The following image display system is configured, the image display system includes an image processing module and a display device, and a lens assembly disposed between the display device and a human eye;

[0032] In an actual use environment, an actual object is captured at a capture perspective and an actual image is formed. An image processing module records the capture perspective, a three-dimensional model is rendered based on the capture perspective, and a virtual camera is constructed on the capture perspective, so that the virtual camera reads and samples a rendered image in a rendering area of ​​the three-dimensional model after rendering, wherein the three-dimensional model has a spatial angular frequency f1, and the virtual camera has a reading angular frequency f2 when reading the rendered image, and has a sampling angular frequency f3 when sampling the rendered image.

[0033] The image processing module calculates a common multiple of any two of a first ripple interval d1 of the rendering area at the spatial angular frequency f1, a second ripple interval d2 of the rendering area at the reading angular frequency f2, a third ripple interval d3 of the rendering area at the sampling angular frequency f3, a fourth ripple interval d4 of the rendering area at the device angular frequency f4 of the display device, and a fifth ripple interval d5 of the rendering area at the imaging angular frequency f5 of the display light emitted by the display device after being imaged by the lens assembly;

[0034] The image processing module adjusts two of the spatial angular frequency f1, the reading angular frequency f2, the sampling angular frequency f3, the device angular frequency f4, and the imaging angular frequency f5 selected to form the common multiple until the common multiple is less than x times of the display device, and stores the result;

[0035] The image processing module corrects the actual image based on the adjusted spatial angular frequency f1, the reading angular frequency f2, the sampling angular frequency f3, the device angular frequency f4, and the imaging angular frequency f5.

[0036] Preferably, in an actual use environment, the camera device captures an actual object at a capture angle of view and forms an actual image, the image processing module records the capture angle of view, renders a three-dimensional model based on the capture angle of view, and constructs a virtual camera on the capture angle of view, so that the virtual camera reads and samples the rendered image in the rendering area after the three-dimensional model is rendered, and the step also includes:

[0037] In a test environment, a camera device collects a test image of a test object, and an image processing module forms a distorted image for the test image based on an imaging relationship between a display device and a lens assembly;

[0038] The image processing module calculates and stores the mapping relationship between each pixel point in the test image and each corresponding pixel point in the distorted image.

[0039] Preferably, the step of the image processing module calculating and storing the mapping relationship between each pixel point in the test image and each corresponding pixel point in the distorted image comprises:

[0040] The image processing module obtains the simulation results of the test image and records them as a mapping relationship, or

[0041] The image processing module calculates the pixel angle deviation between the pixel points in the test image and the pixel points in the distorted image on both sides of the symmetry axis of the X axis and / or the Y axis, and records the pixel angle deviation as a mapping relationship;

[0042] In an actual use environment, the camera device captures an actual object at a capture angle and forms an actual image, the image processing module records the capture angle, renders a three-dimensional model based on the capture angle, and constructs a virtual camera on the capture angle, so that the virtual camera reads and samples the rendered image in the rendering area after the three-dimensional model is rendered, and further includes:

[0043] The image processing module regards the distortion result of the actual image as the original image and the actual image as the processed image, and calculates the inverse mapping relationship of changing each pixel position in the original image to each pixel position in the processed image;

[0044] The image processing module stretches each pixel position of the actual image based on the inverse mapping relationship, and replaces the actual image in the above steps with the formed corrected image.

[0045] The present invention also discloses a VR device, comprising the image display system as described above.

[0046] Compared with the prior art, the above technical solution has the following beneficial effects:

[0047] 1. The correction of the distorted image still has the general visual effect that meets the user's needs;

[0048] 2. Improve the clarity of shooting and projection after correction and eliminate moiré. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 A schematic diagram of pixel positions of a test image and a distorted image in accordance with a preferred embodiment of the present invention;

[0050] Figure 2 A schematic diagram of a flow chart of an image display method in accordance with a preferred embodiment of the present invention;

[0051] Figure 3 A schematic diagram of imaging relationship in accordance with a preferred embodiment of the present invention;

[0052] Figure 4 A schematic diagram of imaging relationship in accordance with a preferred embodiment of the present invention;

[0053] Figure 5 This is a schematic diagram of imaging relationships in a preferred embodiment of the present invention.

[0054] Reference numerals:

[0055] 10-display device, 20-lens assembly, 30-human eye. DETAILED DESCRIPTION

[0056] The advantages of the present invention are further described below in conjunction with the accompanying drawings and specific embodiments.

[0057] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0058] The terms used in this disclosure are for the purpose of describing specific embodiments only and are not intended to limit the disclosure. The singular forms of "a", "said" and "the" used in this disclosure and the appended claims are also intended to include plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0059] It should be understood that although the terms first, second, third, etc. may be used in the present disclosure to describe various information, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present disclosure, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0060] In the description of the present invention, it is necessary to understand that the terms "longitudinal", "lateral", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc., indicating the orientation or position relationship, are based on the orientation or position relationship shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention.

[0061] In the description of the present invention, unless otherwise specified and limited, it should be noted that the terms "installed", "connected" and "connected" should be understood in a broad sense. For example, it can be a mechanical connection or an electrical connection, or it can be the internal connection between two components. It can be a direct connection or an indirect connection through an intermediate medium. For ordinary technicians in this field, the specific meanings of the above terms can be understood according to the specific circumstances.

[0062] In the following description, the suffixes such as "module", "component" or "unit" used to represent elements are only used to facilitate the description of the present invention, and have no specific meanings. Therefore, "module" and "component" can be used interchangeably.

[0063] The image display system in the embodiment of the present invention includes a camera device (optional), an image processing module, a display device and a lens assembly. When the camera device is working, it will capture the picture facing the camera device and the physical objects, constructed virtual objects, etc. included in the picture. The image processing module is connected to the camera device, so as to receive the camera picture captured by the camera device (the camera picture can be a dynamic video picture or a static image picture) and then perform software processing on the camera picture, further achieving effects such as correcting the camera picture and eliminating moiré patterns in the camera picture, and then outputting the processed image to the display device. Before the display device, that is, before the human eye receives the display content of the display device, the lens assembly is set to refract the display content to form an image, so as to enlarge the display content in the external large area to the interactive content that the human eye is accustomed to.

[0064] In order to eliminate the moiré pattern in the displayed content, when the image display system is in use (i.e., in the actual use environment), the camera device captures the actual object within a capture angle of view and forms an actual image. The capture angle of view means that when the image display system is carried in a wearable manner, when the user turns his head, in order to simulate the content actually viewed, the direction and angle of view pointed by his eyes will be defined as the capture angle of view, that is, the camera device will also capture the actual image that the human eye may see in a certain direction to form an actual image (i.e., the image display system simulates the interface and content seen by the human eye). After the actual image is sent to the image processing module, the image processing module records the capture angle of view at this time (for example, the horizontal or vertical direction). The image processing module constructs a three-dimensional stereoscopic figure formed by randomly superimposing at least one polygon, and renders the three-dimensional model based on the acquisition perspective. When the three-dimensional model is placed in front of the user, the image processing module renders the model with the same perspective as the acquisition perspective (for example, adding light, color, edges, etc.), and constructs a virtual camera in the direction of the acquisition perspective (that is, it does not actually exist, but it is assumed that there is a camera in this direction). The image processing module obtains the virtual camera to read the rendered image in the rendering area, and regards itself as a virtual camera that can read the rendered image formed by the brightness and color changes in the rendering area of ​​the three-dimensional model in this rendering mode. Since the above process is constructed by the image processing module, the following data can be obtained at the same time: the three-dimensional model has a spatial angular frequency f1 corresponding to the acquisition perspective (that is, the spatial arrangement resolution of the minimum grid constituting the three-dimensional model), the virtual camera has a reading angular frequency f2 when reading the rendered image (that is, the resolution under the reading interface when reading the rendered image), and has a sampling angular frequency f3 when sampling the rendered image (that is, the sampling resolution of the virtual camera when the virtual camera collects the rendered image).

[0065] After having the above-mentioned different frequencies, due to the display of the sampling angular frequency, the reading angular frequency, etc. on the display interface, stripes that may not be visible to the naked eye will be generated (stripes are constantly or flickering on the display interface, or they may be stripes of adjacent frames that are received and recognized together by the human eye due to the visual aftereffect), and there are intervals between these adjacent stripes. Therefore, the image processing module will calculate the first ripple interval d1 of the rendering area at the spatial angular frequency f1 (that is, the interval between adjacent ripples generated when only the spatial angular frequency f1 is used to render in the rendering area), the second ripple interval d2 of the rendering area at the reading angular frequency f2 (that is, the interval between adjacent ripples generated when only the reading angular frequency f2 is used to render in the rendering area), the third ripple interval d3 of the rendering area at the sampling angular frequency f3 (that is, the interval between adjacent ripples generated when only the sampling angular frequency f3 is used to render in the rendering area), and the device angular frequency f4 of the display device. The common multiple of any two of the fourth ripple interval d4 in the rendering area (i.e., the interval between adjacent ripples generated when only the device angular frequency f4 is used for rendering in the rendering area, and the device angular frequency f4 is the frequency of the display content in the rendering area), and the fifth ripple interval d5 in the rendering area at the imaging angular frequency f5 after the display light emitted by the display device is imaged by the lens component (i.e., the interval between adjacent ripples generated when only the imaging angular frequency f5 is used for rendering in the rendering area, and the imaging angular frequency f5 is the frequency of the imaged image after the lens component refracts, and the imaged image may be distorted or magnified). In other words, any two of the first ripple interval d1, the second ripple interval d2, the third ripple interval d3, the fourth ripple interval d4, and the fifth ripple interval d5 are first selected, and then the least common multiple [a, b] of the two selected ones is calculated, or the multiples of the least common multiple of the two selected ones are calculated.

[0066] Therefore, in order to adjust the moiré and reduce the overlap of high-frequency and low-frequency stripes at different frequencies, the parameters will be adjusted continuously (or directly selected by calculation), that is, the two selected (the two are two of the spatial angular frequency f1, reading angular frequency f2, sampling angular frequency f3, device angular frequency f4, and imaging angular frequency f5 selected to constitute the common multiple) are adjusted until the lowest common multiple or the multiple of the lowest common multiple is less than x times the angular frequency of the display device (preferably 1 times, the value of x directly affects the number of moirés in the rendering area, and the angular frequency of the display device is also the frequency corresponding to the maximum size of the display device). It can be understood that when the spatial angular frequency f1, the reading angular frequency f2, and the sampling angular frequency f3 are adjusted, that is, to deform the display content of the display device, and when the device angular frequency f4 is adjusted, it can be understood as causing the screen to move or deform, and when the imaging angular frequency f5 is adjusted, it can be understood as causing the lens assembly to move or deform. In other words, it can be applied even when the lens assembly, the display device, and the human eye do not have a normal and standard arrangement. Finally, the image processing module corrects the actual image based on the adjusted spatial angular frequency f1, reading angular frequency f2, sampling angular frequency f3, device angular frequency f4, and imaging angular frequency f5, so that the actual image will also remove moiré or minimize the generation of moiré after distortion correction.

[0067] In a preferred embodiment, before adjusting the actual image, the distorted image will also be preprocessed. Specifically, in a test environment, the camera device collects a test image of a test object (the test object can be any plane or three-dimensional image), and the image processing module forms a distorted image for the test image based on the imaging relationship between the display device and the lens assembly. For example, when the lens assembly refracts the content displayed by the display device to form a pillow shape or a barrel shape, the image processing module will also obtain a distorted image that does not actually conform to the human eye directly observing the test object. The post-image processing module calculates the mapping relationship between each pixel point in the test image and each corresponding pixel point in the distorted image and stores it. In this embodiment, the mapping relationship can be a corresponding relationship between coordinates, or it can be an expression of the distortion amount and angle of all pixel points on a curve (see Figure 1 , each pixel in a normal distorted image is symmetrical about the center point).

[0068] Furthermore, when the relationship between the lens assembly and the display device is standard, the image processing module obtains the simulation result of the test image (that is, the simulation software can directly simulate the result of the test image, and there is no need to experiment in the actual test environment) and records it as a mapping relationship, or when the relationship between the lens assembly and the display device is not standard, the image processing module calculates the pixel angle deviation between the pixel points in the test image and the pixel points in the distorted image on both sides of the symmetry axis of the X-axis and / or the Y-axis, and records the pixel angle deviation as a mapping relationship (in this state, the pixel points in the distorted image and the pixel points in the test image are not symmetrical about the horizontal and vertical directions, and the angle deviation can also be represented by the slope difference); when processing the distorted image, it can be Utilizing the reversible property of the optical path, the image processing module regards the distortion result of the actual image as the original image, and the actual image as the processed image (i.e., reverse operation). The image processing module calculates the inverse mapping relationship of changing each pixel position in the original image to each pixel position in the processed image, that is, the inverse mapping relationship is the angle, size or displacement required to move each pixel position in the original image (distorted image) to the pixel position in the processed image (actual image). The post-image processing module stretches each pixel position of the actual image based on the inverse mapping relationship, and replaces the actual image in the aforementioned steps with the formed corrected image, so that after the corrected image is refracted by the lens assembly, the image actually seen by the human eye is a normal image.

[0069] In a further preferred embodiment or an optional embodiment, the distribution of the pixels in the rendering area is m*n, where m and n are any integers between 100 and 1,000,000, respectively, that is, any rendering area with any pixel distribution is applicable. Before constructing the three-dimensional model, the image processing module can correct the test image based on the mapping relationship and observe the imaging effect of the test object to confirm the accuracy of the mapping relationship. If there is still distortion, the mapping relationship can be continuously adjusted until the test image is displayed normally.

[0070] In a preferred embodiment, see Figure 3 , Figure 4 and Figure 5 ( Figure 3 and Figure 4The uppermost part of the image is the magnified display device 10), and the imaging relationship includes: the optical axis of the display device 10 coincides with the optical axis of the lens assembly 20, that is, in the direction of the viewing angle of the human eye 30, and with the straight line where the center point of the human eye 30 is located as the axis, the lens assembly 20 is symmetrical left-right and up-down, and the display device 10 (display screen) is symmetrical left-right and up-down, or the optical axis of the display device 10 is parallel to the optical axis of the lens assembly 20, that is, in the direction of the viewing angle of the human eye 30, and with the straight line where the center point of the human eye 30 is located as the axis, the lens assembly 20 is symmetrical left-right and up-down, but the display device 10 (display screen) is not symmetrical left-right and up-down, on the contrary (the lens assembly 20 is not symmetrical left-right and up-down, but the display device 10 is symmetrical left-right and up-down,) or the optical axis of the display device 10 forms an angle with the optical axis of the lens assembly 20, and the optical axis of the lens assembly 20 passes through the optical center of the display device 10 (the optical center here refers to the average of the root mean square of the coordinate values ​​of the display area The coordinates represented by the value, and the surrounding p pixels, for example, the area within 100 pixels, the optical axis of the display device 10 refers to a virtual straight line perpendicular to the section plane of the optical center position, the optical axis of the lens assembly 20 refers to a virtual straight line where the light beam does not change direction after passing through the lens assembly 20 and the intersection with the lens assembly 20), that is, in the direction of the viewing angle of the human eye 30, and with the straight line where the center point of the human eye 30 is located as the axis, the lens assembly 20 is symmetrical left and right and up and down, and the display device 10 (display screen) is not symmetrical left and right and up and down, and the display device 10 is tilted, or the optical axis of the display device 10 forms an angle with the optical axis of the lens assembly 20, and the optical axis of the lens assembly 20 deviates from the optical center of the display device 10, that is, in the direction of the viewing angle of the human eye 30, and with the straight line where the center point of the human eye 30 is located as the axis, the lens assembly 20 is not symmetrical left and right and up and down, and the lens assembly 20 is tilted, and the display device 10 (display screen) is symmetrical left and right and up and down. In other words, when the display device 10 is tilted or the lens assembly 20 is tilted in order to reserve space for the human eye 30 and fit the human face, the present invention is still applicable and is no longer limited to the requirement that the display screen and the lens must be placed face to face in existing image display systems (such as VR glasses, AR glasses).

[0071] Understandably, Figure 3 and Figure 4 The intersection of the thin solid lines is the observation point, d is the angular interval, and d can be understood as the fourth ripple interval d4, and the device angular frequency f4 of the display device is 1 / d.

[0072] In a preferred embodiment, the angle is 0.1°-60°; when the optical axis of the display device is parallel to the optical axis of the lens assembly, the distance between the optical axis of the display device and the optical axis of the lens assembly is 0.1mm-300mm; the display device is configured to have 320-102400 pixels in a single display direction, and the size of the display device is 1 inch to 3.4 inches; the distortion mapping pixels of the distorted image account for 2%-50% of all pixels of the test image (the mapping relationship of the distortion mapping pixels referred to here can be the ratio of the number of points used to the total number of pixels); the magnification of the lens assembly is 3-3000, and x is 10 -5 ~10 5 , preferably x can be any integer such as 1, 2, 3, or the reciprocal of an integer.

[0073] It is understood that in the application of augmented reality or mixed reality, the actual captured image is constructed in the virtual space after distortion correction, or vice versa, the rendered image of the virtual model projected after modeling is superimposed on the actual image. Figure 2 The present invention also discloses an image display method, comprising the following steps: configuring the following image display system, the image display system comprising an image processing module and a display device, and a lens assembly disposed between the display device and a human eye; in an actual use environment, capturing an actual object at a capture angle of view and forming an actual image, the image processing module recording the capture angle of view, rendering a three-dimensional model based on the capture angle of view, constructing a virtual camera on the capture angle of view, so that the virtual camera reads and samples a rendered image in a rendering area after the three-dimensional model is rendered, wherein the three-dimensional model has a spatial angular frequency f1, the virtual camera has a reading angular frequency f2 when reading the rendered image, and has a sampling angular frequency f3 when sampling the rendered image, the image processing module calculates a first ripple interval d1 of the rendering area under the spatial angular frequency f1, and a first ripple interval d2 of the rendering area under the reading angular frequency f3, Take the common multiple of any two of the second ripple interval d2 of the rendering area under the angular frequency f2, the third ripple interval d3 of the rendering area under the sampling angular frequency f3, the fourth ripple interval d4 of the device angular frequency f4 of the display device in the rendering area, and the fifth ripple interval d5 of the imaging angular frequency f5 of the display light emitted by the display device after imaging by the lens assembly in the rendering area; the image processing module adjusts two of the spatial angular frequency f1, the reading angular frequency f2, the sampling angular frequency f3, the device angular frequency f4, and the imaging angular frequency f5 selected to constitute the common multiple until the common multiple is less than x times of the display device, and stores; the image processing module corrects the actual image based on the adjusted spatial angular frequency f1, the reading angular frequency f2, the sampling angular frequency f3, the device angular frequency f4, and the imaging angular frequency f5. The angular frequencies in the above embodiments can be initially arranged with reference to patent CN107003777A.

[0074] Preferably, in an actual usage environment, the camera device captures an actual object at a capture angle of view and forms an actual image, the image processing module records the capture angle of view, renders a three-dimensional model based on the capture angle of view, and constructs a virtual camera on the capture angle of view, so that the virtual camera reads and samples the rendered image within the rendering area of ​​the three-dimensional model after rendering, and the step also includes: in a test environment, the camera device captures a test image of a test object, the image processing module forms a distorted image for the test image based on the imaging relationship between the display device and the lens assembly; the image processing module calculates and stores the mapping relationship between each pixel point in the test image and each corresponding pixel point in the distorted image.

[0075] Preferably, the step in which the image processing module calculates and stores the mapping relationship between each pixel point in the test image and each corresponding pixel point in the distorted image includes: the image processing module obtains the simulation result of the test image and records it as a mapping relationship, or the image processing module calculates the pixel angle deviation between the pixel points in the test image and the pixel points in the distorted image on both sides of the symmetry axis of the X-axis and / or the Y-axis, and records the pixel angle deviation as a mapping relationship; in an actual use environment, the camera device captures the actual object at a capture angle and forms an actual image, the image processing module records the capture angle, renders a three-dimensional model based on the capture angle, and constructs a virtual camera on the capture angle so that the virtual camera reads and samples the rendered image in the rendering area after the three-dimensional model is rendered, and further includes: the image processing module regards the distortion result of the actual image as the original image, the actual image as the processed image, and the image processing module calculates the inverse mapping relationship of changing each pixel position in the original image to each pixel position in the processed image; the image processing module stretches each pixel position of the actual image based on the inverse mapping relationship, and replaces the actual image in the aforementioned step with the formed corrected image.

[0076] The present invention also discloses a VR device, comprising the image display system as described above, wherein the display device is a display screen of the VR device.

[0077] It should be noted that the embodiments of the present invention have better practicability and do not impose any form of limitation on the present invention. Any technician familiar with the field may use the technical content disclosed above to change or modify it into an equivalent effective embodiment. However, any modification or equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention are still within the scope of the technical solution of the present invention.

Claims

1. An image display system, comprising an image processing module and a display device, and a lens assembly disposed between the display device and a human eye, characterized in that: In an actual use environment, an actual object is captured at a capture perspective and an actual image is formed, the image processing module records the capture perspective, a three-dimensional model is rendered based on the capture perspective, a virtual camera is constructed on the capture perspective, so that the virtual camera reads and samples a rendered image in a rendering area after the three-dimensional model is rendered, wherein the three-dimensional model has a spatial angular frequency f1, the virtual camera has a reading angular frequency f2 when reading the rendered image, and has a sampling angular frequency f3 when sampling the rendered image, The image processing module calculates a common multiple of any two of a first ripple interval d1 of the rendering area at a spatial angular frequency f1, a second ripple interval d2 of the rendering area at a reading angular frequency f2, a third ripple interval d3 of the rendering area at a sampling angular frequency f3, a fourth ripple interval d4 of the rendering area at a device angular frequency f4 of the display device, and a fifth ripple interval d5 of the rendering area at an imaging angular frequency f5 of display light emitted by the display device after imaging by a lens assembly; The image processing module adjusts two of the spatial angular frequency f1, the reading angular frequency f2, the sampling angular frequency f3, the device angular frequency f4, and the imaging angular frequency f5 selected to constitute the common multiple until the common multiple is less than x times of the display device, and stores the result; The image processing module corrects the actual image based on the adjusted spatial angular frequency f1, the reading angular frequency f2, the sampling angular frequency f3, the device angular frequency f4, and the imaging angular frequency f5.

2. The image display system according to claim 1, wherein: The image display system also includes a camera device; In a test environment, the camera device collects a test image of a test object, and the image processing module forms a distorted image for the test image based on an imaging relationship between the display device and the lens assembly; The image processing module calculates and stores a mapping relationship between each pixel point in the test image and each corresponding pixel point in the distorted image.

3. The image display system according to claim 2, wherein: The image processing module obtains the simulation result of the test image and records it as the mapping relationship, or The image processing module calculates the pixel angle deviation between the pixel points in the test image and the pixel points in the distorted image on both sides of the symmetry axis of the X axis and / or the Y axis, and records the pixel angle deviation as the mapping relationship; The image processing module regards the distortion result of the actual image as the original image, and the actual image as the processed image, and the image processing module calculates the inverse mapping relationship of changing each pixel position in the original image to each pixel position in the processed image; The image processing module stretches each pixel position of the actual image based on the inverse mapping relationship, and replaces the actual image in the above steps with the formed corrected image.

4. The image display system according to claim 3, wherein: The pixel distribution of the rendering area is m*n, where m and n are any integers between 100 and 1,000,000 respectively; The image processing module corrects the test image based on the mapping relationship and detects the imaging effect of the test object.

5. The image display system according to claim 2, characterized in that: The imaging relationship includes: The optical axis of the display device coincides with the optical axis of the lens assembly, or The optical axis of the display device is parallel to the optical axis of the lens assembly, or The optical axis of the display device forms an angle with the optical axis of the lens assembly, and the optical axis of the lens assembly passes through the optical center of the display device, or The optical axis of the display device forms an angle with the optical axis of the lens assembly, and the optical axis of the lens assembly deviates from the optical center of the display device.

6. The image display system according to claim 5, characterized in that: The angle is 0.1°-60°; When the optical axis of the display device is parallel to the optical axis of the lens assembly, the distance between the optical axis of the display device and the optical axis of the lens assembly is 0.1 mm-300 mm; The display device is configured to have a pixel number of 320-102400 in a single display direction, and a size of the display device is between 0.1 inches and 3.4 inches; The distortion mapping pixels of the distorted image account for 2%-50% of all pixels of the test image; The magnification of the lens assembly is 3-3000; x is 10 -5 ~10 5 .

7. An image display method, characterized in that: The following steps are involved: The following image display system is configured, the image display system includes a camera device for collecting display objects, an image processing module and a display device connected to the camera device, and a lens assembly arranged between the display device and the human eye; In an actual use environment, the camera device captures an actual object at a capture angle of view and forms an actual image, the image processing module records the capture angle of view, renders a three-dimensional model based on the capture angle of view, and constructs a virtual camera on the capture angle of view, so that the virtual camera reads and samples a rendered image in a rendering area after the three-dimensional model is rendered, wherein the three-dimensional model has a spatial angular frequency f1, the virtual camera has a reading angular frequency f2 when reading the rendered image, and has a sampling angular frequency f3 when sampling the rendered image, The image processing module calculates a common multiple of any two of a first ripple interval of a rendering area at a spatial angular frequency f1, a second ripple interval of a rendering area at a reading angular frequency f2, a third ripple interval of a rendering area at a sampling angular frequency f3, a fourth ripple interval of a device angular frequency f4 of a display device in the rendering area, and a fifth ripple interval of a display light emitted by the display device in the rendering area at an imaging angular frequency f5 after the display light is imaged by a lens assembly; The image processing module adjusts two of the spatial angular frequency f1, the reading angular frequency f2, the sampling angular frequency f3, the device angular frequency f4, and the imaging angular frequency f5 selected to constitute the common multiple until the common multiple is less than x times of the display device, and stores the result; The image processing module corrects the actual image based on the adjusted spatial angular frequency f1, the reading angular frequency f2, the sampling angular frequency f3, the device angular frequency f4, and the imaging angular frequency f5.

8. The image display method according to claim 7, characterized in that: In an actual use environment, an actual object is captured at a capture perspective and an actual image is formed, the image processing module records the capture perspective, a three-dimensional model is rendered based on the capture perspective, a virtual camera is constructed on the capture perspective, and the step of enabling the virtual camera to read and sample a rendered image in a rendering area after the three-dimensional model is rendered also includes: In a test environment, a camera device collects a test image of a test object, and the image processing module forms a distorted image for the test image based on an imaging relationship between the display device and the lens assembly; The image processing module calculates and stores a mapping relationship between each pixel point in the test image and each corresponding pixel point in the distorted image.

9. The image display method according to claim 8, wherein: The step of the image processing module calculating and storing the mapping relationship between each pixel point in the test image and each corresponding pixel point in the distorted image comprises: The image processing module obtains the simulation result of the test image and records it as the mapping relationship, or The image processing module calculates the pixel angle deviation between the pixel points in the test image and the pixel points in the distorted image on both sides of the symmetry axis of the X axis and / or the Y axis, and records the pixel angle deviation as the mapping relationship; In an actual use environment, the camera device captures an actual object at a capture angle of view and forms an actual image, the image processing module records the capture angle of view, renders a three-dimensional model based on the capture angle of view, and constructs a virtual camera on the capture angle of view, so that the virtual camera reads and samples the rendered image in the rendering area after the three-dimensional model is rendered, and further includes: The image processing module regards the distortion result of the actual image as the original image, and the actual image as the processed image, and the image processing module calculates the inverse mapping relationship of changing each pixel position in the original image to each pixel position in the processed image; The image processing module stretches each pixel position of the actual image based on the inverse mapping relationship, and replaces the actual image in the above steps with the formed corrected image.

10. A VR device, characterized in that: The invention comprises an image display system as claimed in any one of claims 1 to 6.

Images