Image source encoding method, apparatus and near-eye display device

Abstract

The application relates to an image source coding method device and a near-eye display device, the method is used for the near-eye display device, the near-eye display device comprises a micro display and a light waveguide module, the method comprises the following steps: adjusting an original coding matrix of an image source according to a modulation matrix to obtain a first coding matrix; the modulation matrix is generated according to grating parameters of the light waveguide module, and the modulation matrix comprises modulation coefficients of pixel gray values of the image source; the image source is coded according to the first coding matrix; the coded image source is emitted through the micro display to form an emitted light field; and the emitted light field is propagated through the light waveguide module to output a display image. Through the application, the problem of brightness difference in the prior art near-eye display device is solved.

Description

Technical Field

[0001] This application relates to the field of near-eye display technology, and in particular to an image source encoding method, apparatus and near-eye display device. Background Technology

[0002] The optical waveguide module in near-eye display devices typically consists of an input grating, a pupil-expanding grating, and an output grating. Due to the generally high demands for portability and comfort in near-eye display devices, the waveguide module's thickness is generally less than 1mm. This extremely thin waveguide causes the light field to be incident on the grating region multiple times during transmission. These cascaded reflections result in a power-like attenuation of the light field energy. The light field energy attenuates along orthogonal one-dimensional directions in the pupil-expanding and output grating regions, respectively. Consequently, the brightness of the light field received by the human eye after passing through the output grating exhibits a diagonal attenuation distribution. Furthermore, commonly used grating structures exhibit angular and spectral selectivity, meaning they respond significantly differently to different incident angles and wavelengths. Therefore, image information from different pixels, after propagation and coupling out through the optical waveguide module, is received by the human eye, resulting in significant brightness differences. These brightness differences manifest as non-uniformity in the field of view and color.

[0003] There is currently no effective solution to the problem of brightness differences in existing near-eye display devices. Summary of the Invention

[0004] This embodiment provides an image source encoding method, apparatus, and near-eye display device to solve the problem of brightness differences in existing near-eye display devices.

[0005] In a first aspect, this embodiment provides an image source encoding method for a near-eye display device, the near-eye display device including a microdisplay and an optical waveguide module, the method comprising,

[0006] The original encoding matrix of the image source is adjusted according to the modulation matrix to obtain the first encoding matrix; the modulation matrix is ​​generated according to the grating parameters of the optical waveguide module, and the modulation matrix includes the modulation coefficients of the pixel grayscale values ​​of the image source;

[0007] The image source is encoded according to the first encoding matrix. The encoded image source is emitted through the microdisplay to form an emitted light field. The emitted light field is propagated through the optical waveguide module to output and display an image.

[0008] In some embodiments, the modulation matrix includes a first modulation sub-matrix, which includes first modulation coefficients of the grayscale values ​​of all pixels of the image source. The first modulation coefficients are obtained by normalizing the grayscale values ​​of the bias pixels based on the grayscale value of the center pixel of the microdisplay.

[0009] In some embodiments, the modulation matrix includes a second modulation sub-matrix and a third modulation sub-matrix. The second modulation sub-matrix includes second modulation coefficients for the R component grayscale values ​​of all pixels in the image source. The second modulation coefficients are obtained by normalizing the R component grayscale values ​​based on the G component grayscale values. The third modulation sub-matrix includes third modulation coefficients for the B component grayscale values ​​of all pixels in the image source. The third modulation coefficients are obtained by normalizing the B component grayscale values ​​based on the G component grayscale values.

[0010] In some embodiments, the optical waveguide module includes an input grating, a pupil grating, and an output grating, and the grating parameters include input grating parameters, pupil grating parameters, and output grating parameters.

[0011] In some embodiments, the values ​​obtained by the normalization process range from 0.5 to 1.5.

[0012] In some embodiments, the method further includes obtaining eye position information of a user wearing the near-eye display device, and selecting a modulation matrix corresponding to the eye position information based on the eye position information.

[0013] Secondly, this embodiment provides an image source encoding device for a near-eye display device, the near-eye display device including a microdisplay and an optical waveguide module, the device comprising,

[0014] The encoding matrix adjustment module is used to adjust the original encoding matrix of the image source according to the modulation matrix to obtain a first encoding matrix; the modulation matrix is ​​generated according to the grating parameters of the optical waveguide module, and the modulation matrix includes the modulation coefficients of the pixel grayscale values ​​of the image source;

[0015] The image encoding module is used to encode the image source according to the first encoding matrix. The encoded image source is emitted through the microdisplay to form an emitted light field. The emitted light field is propagated through the optical waveguide module to output a display image.

[0016] Thirdly, this embodiment provides a near-eye display device, which includes a main control board, a microdisplay, an optical waveguide module, and an eye-tracking module; the main control board is connected to the microdisplay and the eye-tracking module.

[0017] The eye-tracking module acquires the eye position information of the user wearing the near-eye display device;

[0018] The main control board generates a modulation matrix based on the grating parameters of the optical waveguide module. The modulation matrix includes the modulation coefficients of the pixel grayscale values ​​of the image source. Based on the human eye position information, the board selects the modulation matrix corresponding to the human eye position information.

[0019] The main control board adjusts the original encoding matrix of the image source according to the modulation matrix corresponding to the human eye position information to obtain a second encoding matrix;

[0020] The main control board encodes the image source according to the second encoding matrix. The encoded image source is emitted through the microdisplay to form an emitted light field. The emitted light field propagates through the optical waveguide module and outputs a display image.

[0021] Fourthly, this embodiment provides an electronic device including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the image source encoding method described in the first aspect above.

[0022] Fifthly, this embodiment provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the image source encoding method described in the first aspect above.

[0023] Compared with the prior art, the image source encoding method, apparatus and near-eye display device provided in this embodiment calculate a modulation matrix including the modulation coefficients of the pixel grayscale values ​​of the image source according to the grating parameters of the optical waveguide module, adjust the original encoding matrix of the image source according to the modulation coefficients, encode the image source using the adjusted encoding matrix, and emit the encoded image source through the microdisplay to form an emitted light field. The emitted light field propagates through the optical waveguide module to output the displayed image, which improves the uniformity of the brightness coupled out of the optical waveguide module and solves the problem of brightness difference in the near-eye display device in the prior art.

[0024] Details of one or more embodiments of this application are set forth in the following drawings and description to make other features, objects and advantages of this application more readily apparent. Attached Figure Description

[0025] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0026] Figure 1 This is a schematic diagram of the structure of a near-eye display device in this embodiment;

[0027] Figure 2This is a schematic diagram of another near-eye display device in this embodiment;

[0028] Figure 3 This is a flowchart of the image source encoding method in this embodiment;

[0029] Figure 4 This is a schematic diagram of the optical waveguide module in this embodiment;

[0030] Figure 5 This is a schematic diagram of the optical field propagation and energy of the optical waveguide module in this embodiment;

[0031] Figure 6(a) is a schematic diagram of the angle selectivity of a grating in this embodiment;

[0032] Figure 6(b) is a schematic diagram of another grating angle selectivity in this embodiment;

[0033] Figure 6(c) is a schematic diagram of another grating angle selectivity in this embodiment;

[0034] Figure 7 This is a schematic diagram of the optical field propagation of the optical waveguide module in this embodiment;

[0035] Figure 8(a) is a schematic diagram of the output brightness distribution of the optical waveguide module in this embodiment;

[0036] Figure 8(b) is a schematic diagram of another output brightness distribution of the optical waveguide module in this embodiment;

[0037] Figure 8(c) is a schematic diagram of another coupling brightness distribution of the optical waveguide module in this embodiment;

[0038] Figure 8(d) is a schematic diagram of another coupling brightness distribution of the optical waveguide module in this embodiment;

[0039] Figure 9 This is a schematic diagram of the spectral selectivity of the grating of the optical waveguide module in this embodiment;

[0040] Figure 10 This is a schematic diagram of the pupil position offset in this embodiment;

[0041] Figure 11 This is a structural block diagram of an image source encoding device according to this embodiment. Detailed Implementation

[0042] To better understand the purpose, technical solution, and advantages of this application, the application is described and illustrated below in conjunction with the accompanying drawings and embodiments.

[0043] Unless otherwise defined, the technical or scientific terms used in this application shall have the general meaning as understood by one of ordinary skill in the art to which this application pertains. Words such as “a,” “an,” “an,” “the,” “the,” and “these,” used in this application, do not indicate quantitative limitation and may be singular or plural. The terms “comprising,” “including,” “having,” and any variations thereof used in this application are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that comprises a series of steps or modules (units) is not limited to the listed steps or modules (units) but may include steps or modules (units) not listed, or may include other steps or modules (units) inherent to such processes, methods, products, or devices. The terms “connected,” “linked,” and “coupled,” used in this application, are not limited to physical or mechanical connections but may include electrical connections, whether direct or indirect. The term “multiple” used in this application refers to two or more. The "and / or" operator describes the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: A alone, A and B simultaneously, and B alone. Typically, the character " / " indicates that the objects before and after it are in an "or" relationship. The terms "first," "second," and "third," etc., used in this application are merely for distinguishing similar objects and do not represent a specific ordering of the objects.

[0044] Terminology Explanation:

[0045] Brightness uniformity: The image source signal of a near-eye display device is encoded and emitted as a light field through a microdisplay or display module. This light field propagates via a waveguide module in a TIR (total internal reflection)-out configuration before being received by the human eye. The image source consists of an array of pixels, which includes pixels and sub-pixels, carrying grayscale and spectral information. The brightness information received by the human eye includes field-of-view information (corresponding to pixels) and color information (corresponding to sub-pixels), corresponding to field-of-view uniformity and color uniformity, respectively. In other words, brightness uniformity includes both field-of-view uniformity and color uniformity.

[0046] Field uniformity: Different pixels of the image source signal represent different fields of view through the light field emitted by the microdisplay. The grayscale information of different pixels is transmitted through the optical waveguide module and received by the human eye, which corresponds to the field uniformity.

[0047] Color uniformity: For the same pixel in the image source signal, the light field emitted by the microdisplay carries RGB color information. The grayscale information of the RGB components of the same pixel is transmitted through the optical waveguide module and received by the human eye, which corresponds to color uniformity.

[0048] This embodiment provides a near-eye display device. Figure 1 This is a schematic diagram of the structure of a near-eye display device in this embodiment, as shown below. Figure 1 As shown, the near-eye display device includes a main control board 1, a microdisplay 2, an optical waveguide module 3, and an eye-tracking module 4.

[0049] The main control board 1, microdisplay 2, optical waveguide module 3, and eye-tracking module 4 are all mounted on the main circuit board. The main control board 1 is connected to the microdisplay 2, and the main control board 1 is also connected to the eye-tracking module 4.

[0050] The main control board 1 serves as the control and calculation unit for the near-eye display device. It controls the microdisplay 2 to display the encoded image source signal, controls the eye-tracking module 4 to track the user's eye position information while wearing the near-eye display device, receives the eye position information sent by the eye-tracking module 4, calculates the modulation matrix based on the grating parameters of the optical waveguide module 3, and selects the modulation matrix corresponding to the eye position information. This modulation matrix includes modulation coefficients for the pixel grayscale values ​​of the image source. These modulation coefficients include those that adjust the grayscale values ​​of all pixels in the image source, as well as those that adjust the RGB component grayscale values ​​of the same pixel across all pixels.

[0051] After obtaining the modulation matrix corresponding to the human eye position information, the main control board 1 adjusts the original encoding matrix of the image source according to the modulation matrix corresponding to the human eye position information to obtain the second encoding matrix. The main control board 1 encodes the image source according to the second encoding matrix. The encoded image source is emitted through the microdisplay 2 to form an emitted light field. The emitted light field propagates through the optical waveguide module 3 and outputs the displayed image.

[0052] The microdisplay 2 is connected to the main control board 1, receives the encoded image source sent by the main control board 1, and emits it to form an emitted light field.

[0053] The optical waveguide module 3 is mounted on the main circuit board. The optical waveguide module 3 propagates the outgoing light field formed by the microdisplay 2 and outputs the displayed graphics.

[0054] The core functional modules of the near-eye display device include a microdisplay 2 and an optical waveguide module 3. To obtain a pupil distribution with high brightness uniformity, certain requirements must be placed on the design of the optical waveguide module 3. The design results of the field uniformity and color uniformity of the optical waveguide module 3 are used as reverse input to re-encode the image source signal of the microdisplay 2. That is, the optical waveguide module 3 outputs a modulation matrix, and the pixels and sub-pixels (RGB components) of the image source signal are re-scaled according to the modulation matrix to obtain a new encoding method, and then the light field is output.

[0055] The near-eye display device provided in this embodiment calculates a modulation matrix including the modulation coefficients of the pixel grayscale values ​​of the image source based on the grating parameters of the optical waveguide module 3. The original encoding matrix of the image source is adjusted according to the modulation coefficients, and the image source is encoded using the adjusted encoding matrix. The encoded image source is emitted through the microdisplay 2 to form an emitted light field. The emitted light field propagates through the optical waveguide module 3 to output the displayed image, thereby improving the uniformity of the brightness coupled out of the optical waveguide module 3 and solving the problem of brightness differences in the prior art near-eye display devices.

[0056] This embodiment also provides a near-eye display device. Figure 2 This is a schematic diagram of another near-eye display device in this embodiment, as shown below. Figure 2 As shown, the near-eye display device also includes an eye-tracking module control board 5, which is mounted on the main circuit board.

[0057] The eye-tracking module control board 5 is connected to the main control board 1 and the eye-tracking module 4. The eye-tracking module control board 5 is used to control the eye-tracking module 4 to track the eye position information of the user wearing the near-eye display device and to receive the user's eye position information acquired by the eye-tracking module 4. The eye-tracking module control board 5 sends the acquired eye position information to the main control board 1.

[0058] The main control board 1 sends an eye position information acquisition signal to the eye-tracking module control board 5. Upon receiving this signal, the eye-tracking module control board 5 controls the eye-tracking module 4 to acquire the eye position information. The eye-tracking module 4 then sends the acquired eye position information back to the eye-tracking module control board 5, which in turn sends it back to the main control board 1. The main control board 1 receives the eye position information from the eye-tracking module control board 5, calculates a modulation matrix based on the grating parameters of the optical waveguide module 3, and selects the modulation matrix corresponding to the eye position information. This modulation matrix includes modulation coefficients for the pixel grayscale values ​​of the image source. These modulation coefficients include those that adjust the grayscale values ​​of all pixels in the image source, as well as those that adjust the grayscale values ​​of the RGB components of the same pixel across all pixels.

[0059] After obtaining the modulation matrix corresponding to the human eye position information, the main control board 1 adjusts the original encoding matrix of the image source according to the modulation matrix corresponding to the human eye position information to obtain the second encoding matrix. The main control board 1 encodes the image source according to the second encoding matrix. The encoded image source is emitted through the microdisplay 2 to form an emitted light field. The emitted light field propagates through the optical waveguide module 3 and outputs the displayed image.

[0060] The near-eye display device provided in this embodiment calculates a modulation matrix including the modulation coefficients of the pixel grayscale values ​​of the image source based on the grating parameters of the optical waveguide module 3. The original encoding matrix of the image source is adjusted according to the modulation coefficients, and the image source is encoded using the adjusted encoding matrix. The encoded image source is emitted through the microdisplay 2 to form an emitted light field. The emitted light field propagates through the optical waveguide module 3 to output the displayed image, thereby improving the uniformity of the brightness coupled out of the optical waveguide module 3 and solving the problem of brightness differences in the prior art near-eye display devices.

[0061] This embodiment provides an image source encoding method, which can be used in any of the near-eye display devices described in the above embodiments. Figure 3 This is a flowchart of the image source encoding method in this embodiment, as follows: Figure 3 As shown, the process includes the following steps:

[0062] Step S302: Adjust the original encoding matrix of the image source according to the modulation matrix to obtain the first encoding matrix; the modulation matrix is ​​generated according to the grating parameters of the optical waveguide module, and the modulation matrix includes the modulation coefficients of the pixel grayscale values ​​of the image source.

[0063] The main control board generates a modulation matrix based on the grating parameters of the optical waveguide module. The modulation matrix includes the modulation coefficients of the pixel grayscale values ​​of the image source. Based on the modulation matrix, the main control board adjusts the original encoding matrix of the image source to obtain the first encoding matrix.

[0064] Step S304: The image source is encoded according to the first encoding matrix. The encoded image source is emitted through the microdisplay to form an emitted light field. The emitted light field is propagated through the optical waveguide module to output the displayed image.

[0065] The main control board encodes the image source according to the first encoding matrix. The encoded image source is emitted through the microdisplay, forming an emitted light field. This emitted light field propagates through the optical waveguide module and outputs the displayed image. The first encoding matrix is ​​obtained by adjusting the original encoding matrix according to the modulation coefficients in the modulation matrix.

[0066] Through the above steps, based on the grating parameters of the optical waveguide module, a modulation matrix including the modulation coefficients of the pixel grayscale values ​​of the image source is calculated. The original encoding matrix of the image source is adjusted according to the modulation coefficients. The image source is then encoded using the adjusted encoding matrix. The encoded image source is emitted through the microdisplay to form an emitted light field. The emitted light field propagates through the optical waveguide module and outputs the displayed image. This improves the uniformity of the brightness coupled out of the optical waveguide module and solves the problem of brightness differences in existing near-eye display devices.

[0067] Taking a binary grating as an example of the grating structure of optical waveguide module 3, the angular selectivity and spectral selectivity of the grating are explained.

[0068] Figure 4 This is a schematic diagram of the optical waveguide module in this embodiment. The optical waveguide module 3 includes an input grating 41, a pupil-expanding grating 42, and an output grating 43. Since near-eye display devices generally require portability and comfort, the thickness of the optical waveguide module 3 is generally less than 1 mm. The extremely thin waveguide causes the light field to be incident on the grating region multiple times during transmission. The cascaded multiple reflections cause the light field energy to exhibit a power-like attenuation.

[0069] Figure 5 This is a schematic diagram of the optical field propagation and energy of the optical waveguide module in this embodiment, as shown below. Figure 5 As shown, the light field energy attenuates along orthogonal one-dimensional directions in the pupil expansion grating and the coupling grating regions, respectively. Therefore, the brightness of the light field received by the human eye after passing through the coupling grating will exhibit a distribution that attenuates along the diagonal.

[0070] Figure 6(a) is a schematic diagram of the angle selectivity of one type of grating in this embodiment. Figure 6(b) is a schematic diagram of the angle selectivity of another type of grating in this embodiment. Figure 6(c) is a schematic diagram of the angle selectivity of another type of grating in this embodiment. As shown in Figures 6(a), 6(b), and 6(c), the grating exhibits different responses, i.e., diffraction efficiencies, for different incident angles (i.e., field of view). Within a certain angular range, the grating exhibits a relatively high average diffraction efficiency. Once deviated from this range, the diffraction efficiency drops sharply. Figures 6(a), 6(b), and 6(c) show the different responses of the grating to different incident angles.

[0071] Figure 7 This is a schematic diagram of the optical field propagation of the optical waveguide module in this embodiment. (See diagram below.) Figure 1 , Figure 2 and Figure 7 As shown, the grating of the optical waveguide module 3 diffracts for pixels with different fields of view. The image source consists of an array of pixels, each containing a sub-pixel of RGB. Because each pixel has a different field of view, the diffraction efficiency of the grating of the optical waveguide module 3 varies, resulting in uneven brightness received by the human eye 6.

[0072] Figure 8(a) is a schematic diagram of one type of output brightness distribution of the optical waveguide module in this embodiment. Figure 8(b) is a schematic diagram of another type of output brightness distribution of the optical waveguide module in this embodiment. Figure 8(c) is a schematic diagram of another type of output brightness distribution of the optical waveguide module in this embodiment. Figure 8(d) is a schematic diagram of another type of output brightness distribution of the optical waveguide module in this embodiment. A typical waveguide module consists of three regions: input, pupil, and output grating. Assuming that all pixels in the image encoded by the image source have the same grayscale value, through the cascading design between regions, the output brightness distribution (or grayscale distribution) in different fields of view may have four cases as shown in Figure 8(a), Figure 8(b), Figure 8(c), and Figure 8(d). As shown in Figure 8(a), the output brightness is randomly distributed. As shown in Figure 8(b), the output brightness is radially distributed. As shown in Figure 8(c), the output brightness is fan-shaped distributed. As shown in Figure 8(d), the output brightness is quadrant distributed. The grayscale distribution corresponds one-to-one with the array pixels of the microdisplay 2. The center pixel corresponds to the grayscale value of the center field of view, and the offset pixel corresponds to the grayscale value of the off-axis field of view. The center field of view is the field of view corresponding to the center of the microdisplay 2, and the other pixels relative to the center pixel are offset pixels.

[0073] In some embodiments, the modulation matrix includes a grayscale modulation matrix M1, which includes grayscale modulation coefficients Q1 for the grayscale values ​​of all pixels in the image source. The grayscale modulation coefficients Q1 are obtained by normalizing the grayscale values ​​of the bias pixels based on the grayscale values ​​of the center pixels of the center field of view corresponding to the center of the microdisplay 2.

[0074] Specifically, based on the grating parameters of the optical waveguide module 3, a fixed value is set so that the grayscale values ​​of all pixels in the image source are the same. Data simulation is used to obtain the grayscale values ​​H1 of the center pixel and H2 of the offset pixels propagating through the optical waveguide module 3. The grayscale value H1 of the center pixel is used to normalize the grayscale value H2 of the offset pixel, resulting in a normalized grayscale value N1 for the offset pixel. After normalization, the grayscale value of the center pixel is 1, and the grayscale value of the offset pixel is N1. A grayscale modulation coefficient Q1 is generated based on the normalized grayscale value N1 of the offset pixel. The grayscale modulation coefficient Q1 forms the grayscale modulation matrix M1. For example, the grayscale modulation coefficient Q1 is generated based on N1*Q1 = 1. By selecting the grating parameters of the optical waveguide module 3, N1 is set between 0.5 and 1.5. If N1 is too small or too large, the grayscale modulation matrix M1 output by the waveguide design will be used for re-encoding of the image source pixels, causing a decrease in the overall emitted brightness of the microdisplay 2.

[0075] In some embodiments, when the coupled-out brightness exhibits an irregular random distribution, as shown in Figure 8(a), the image source pixels and the grayscale modulation matrix M1 are in one-to-one correspondence (point correspondence). Interpolation can be used to adjust the original encoding matrix of the image source, improving the efficiency of image source re-encoding while ensuring a certain level of brightness uniformity accuracy. When the coupled-out brightness exhibits a regular distribution, as shown in Figures 8(b), 8(c), and 8(d), the image source pixels and the grayscale modulation matrix M1 are in patch correspondence, meaning that the grayscale modulation coefficients Q1 of the grayscale modulation matrix M1 within a certain patch are equal or differ by less than a certain threshold. Patch processing can be used to adjust the original encoding matrix of the image source, greatly improving encoding efficiency.

[0076] Figure 9 This is a schematic diagram of the spectral selectivity of the grating in the optical waveguide module of this embodiment, as shown below. Figure 9 As shown, the grating exhibits different responses to different incident wavelengths (RGB), and the RGB diffraction efficiency depends on the designed grating parameters. The grayscale value information of a specific pixel in the coupled brightness distribution is extracted, consisting of the R component grayscale value, the G component grayscale value, and the B component grayscale value.

[0077] In some embodiments, the modulation matrix includes an R-component color modulation matrix M2 and a B-component color modulation matrix M3. M2 includes R-component modulation coefficients Q2 for the R-component gray values ​​of all pixels in the image source, which are obtained by normalizing the R-component gray values ​​based on the G-component gray values. M3 includes B-component modulation coefficients Q3 for the B-component gray values ​​of all pixels in the image source, which are obtained by normalizing the B-component gray values ​​based on the G-component gray values.

[0078] Specifically, based on the grating parameters of the optical waveguide module 3, the grayscale value ratio of the RGB components of each pixel in the original image source signal is set to 1:1:1. Data simulation yields the grayscale values ​​H3 (R component), H4 (G component), and H5 (B component) of each pixel in the image source propagated through the optical waveguide module 3. The grayscale value H3 of the R component is normalized using the grayscale value H4 of the G component to obtain the normalized grayscale value N2 of the R component; similarly, the grayscale value H5 of the B component is normalized using the grayscale value H4 of the G component to obtain the normalized grayscale value N3 of the B component. After coupling out through the waveguide module, the RGB grayscale value ratio of the pixel is transformed to N2:1:N3. The R component color modulation coefficient Q2 is generated based on the normalized grayscale value N2 of the R component. The R component color modulation coefficient Q2 forms the R component color modulation matrix M2. For example, the grayscale modulation coefficient Q2 is generated based on N2*Q2 = 1. The B-component color modulation coefficient Q3 is generated based on the normalized grayscale value N3 of the B-component. The B-component color modulation coefficient Q3 forms the B-component color modulation matrix M3. For example, the grayscale modulation coefficient Q3 is generated based on N3*Q3 = 1. By selecting the grating parameters of the optical waveguide module 3, N2 and N3 are set between 0.5 and 1.5. If N2 and N3 are too small or too large, the color modulation matrices M2 and M3 output by the waveguide design, used for re-encoding the image source sub-pixels, may still exhibit color casts in different coupling regions.

[0079] In some of these embodiments, the original encoding matrix I1 of the image source is adjusted according to the grayscale modulation matrix M1, the R component color modulation matrix M2, and the B component color modulation matrix M3 to obtain the first encoding matrix I2. The image source is then encoded according to the first encoding matrix I2. The encoded image source is emitted through the microdisplay 2 to form an emitted light field. The emitted light field propagates through the optical waveguide module 3 to output the displayed image.

[0080] Specifically, the new luminance value of each pixel is obtained by dot product of matrices I1 and M1. Then, according to matrices M2 and M3, the RGB ratio in the new luminance value is adjusted, and the first encoding matrix I2 is obtained based on the new luminance value and the RGB ratio.

[0081] The grayscale modulation matrix M1, designed based on the grating parameters of the optical waveguide module 3, corresponds one-to-one with the pixels of the image source array. It can be used for pixel brightness re-encoding, improving the overall uniformity of the waveguide-coupled brightness, i.e., field-of-view uniformity. The R-component color modulation matrix M2 and the B-component color modulation matrix M3, designed based on the grating parameters of the optical waveguide module 3, are normalized using the most sensitive wavelength of human eye, G light. They can be used for RGB sub-pixel brightness re-encoding, improving the white balance of the waveguide-coupled brightness, i.e., color uniformity. Combining modulation matrices M1, M2, and M3, the overall brightness uniformity of the near-eye display device can be significantly improved, while simultaneously ensuring both field-of-view uniformity and color uniformity.

[0082] In some embodiments, the eye position information of a user wearing a near-eye display device is obtained, a modulation matrix is ​​generated based on the grating parameters of the optical waveguide module, and a modulation matrix corresponding to the eye position information is selected based on the eye position information.

[0083] Specifically, Figure 10 This is a schematic diagram of the pupil position offset in this embodiment. To ensure the universality of near-eye display devices for different user groups, the coupling pupil of the optical waveguide module 3 needs to be expanded to a certain width, such as... Figure 10 As shown, the position of the extended pupil region is represented by x. The brightness received by the human eye 6 at different positions within the extended pupil region varies. Position information is added to the modulation matrix calculated based on the grating parameters of the optical waveguide module 3 to form new modulation matrices M1(x), M2(x), and M3(x). Modulation matrices M1(x), M2(x), and M3(x) at different positions x within the extended pupil region are obtained through data simulation. By giving a step size Δx, the modulation matrices at all positions within the coupled pupil range are covered, and a lookup table of positions and modulation matrices is calculated. The human eye position information fed back by the eye-tracking module 4 corresponds to the position of the extended pupil region. The corresponding modulation matrix is ​​obtained by looking up the table based on this human eye position information, enabling adaptive adjustment of the brightness uniformity at different pupil positions.

[0084] This embodiment also provides an image source encoding device for implementing the above embodiments and preferred embodiments; details already described will not be repeated. The terms "module," "unit," "subunit," etc., used below refer to combinations of software and / or hardware that perform a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0085] Figure 11 This is a structural block diagram of an image source encoding device according to this embodiment. The device is used in a near-eye display device, which includes a microdisplay 2 and an optical waveguide module 3, such as... Figure 11As shown, the device includes:

[0086] The encoding matrix adjustment module 10 is used to adjust the original encoding matrix of the image source according to the modulation matrix to obtain the first encoding matrix; the modulation matrix is ​​generated according to the grating parameters of the optical waveguide module 3, and the modulation matrix includes the modulation coefficients of the pixel gray values ​​of the image source.

[0087] The image encoding module 20 is used to encode the image source according to the first encoding matrix. The encoded image source is emitted through the microdisplay 2 to form an emitted light field. The emitted light field is propagated through the optical waveguide module 3 to output the displayed image.

[0088] It should be noted that the above modules can be functional modules or program modules, and can be implemented through software or hardware. For modules implemented through hardware, the above modules can reside in the same processor; or the above modules can be located in different processors in any combination.

[0089] This embodiment also provides an electronic device including a memory and a processor, the memory storing a computer program and the processor being configured to run the computer program to perform the steps in any of the above method embodiments.

[0090] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor and the input / output device is connected to the processor.

[0091] Optionally, in this embodiment, the processor can be configured to perform the following steps via a computer program:

[0092] S1, Based on the modulation matrix, the original encoding matrix of the image source is adjusted to obtain the first encoding matrix. This modulation matrix is ​​generated based on the grating parameters of the optical waveguide module 3, and includes the modulation coefficients of the pixel grayscale values ​​of the image source.

[0093] S2, the image source is encoded according to the first encoding matrix, and the encoded image source is emitted through the microdisplay 2 to form an emitted light field. The emitted light field is propagated through the optical waveguide module 3 to output the displayed image.

[0094] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementations, and will not be repeated in this embodiment.

[0095] Furthermore, in conjunction with the image source encoding methods provided in the above embodiments, this embodiment can also provide a storage medium for implementation. This storage medium stores a computer program; when executed by a processor, the computer program implements the steps of any of the image source encoding methods in the above embodiments.

[0096] It should be understood that the specific embodiments described herein are merely illustrative of the application and not intended to limit it. All other embodiments derived by those skilled in the art based on the embodiments provided in this application without inventive effort are within the scope of protection of this application.

[0097] Obviously, the accompanying drawings are merely some examples or embodiments of this application. Those skilled in the art can apply this application to other similar situations based on these drawings without any creative effort. Furthermore, it is understood that although the work done in this development process may be complex and lengthy, for those skilled in the art, certain design, manufacturing, or production modifications made based on the technical content disclosed in this application are merely conventional technical means and should not be considered as insufficient disclosure of this application.

[0098] The term "embodiment" in this application refers to a specific feature, structure, or characteristic described in connection with an embodiment that may be included in at least one embodiment of this application. The appearance of this phrase in various places in the specification does not necessarily imply the same embodiment, nor does it imply that it is mutually exclusive with or independent of other embodiments. It will be clearly or implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments without conflict.

[0099] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of patent protection. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the appended claims.

Claims

1. An image source coding method characterized by, The method is used for a near-eye display device including a micro display and a light waveguide module, and the method includes, The original encoding matrix of the image source is adjusted according to a modulation matrix to obtain a first encoding matrix; The modulation matrix is generated according to grating parameters of the light waveguide module, and the modulation matrix includes modulation coefficients of pixel gray values of the image source; wherein the grating parameters are used to perform field of view uniformity and color uniformity simulation of the light waveguide module to obtain corresponding design results; the design results are used to calculate the modulation coefficients; The image source is encoded according to the first encoding matrix, and the encoded image source is emitted through the micro display to form an emitted light field, and the emitted light field is propagated through the light waveguide module to output a display image.

2. The image source coding method according to claim 1, characterized by, The modulation matrix includes a first modulation sub-matrix, and the first modulation sub-matrix includes first modulation coefficients of gray values of all pixels of the image source, and the first modulation coefficients are obtained by normalizing gray values of offset pixels according to a gray value of a center pixel of the micro display.

3. The image source coding method of claim 1, characterized by, The modulation matrix includes a second modulation sub-matrix and a third modulation sub-matrix, the second modulation sub-matrix includes second modulation coefficients of R component gray values of all pixels of the image source, and the second modulation coefficients are obtained by normalizing the R component gray values according to G component gray values; and the third modulation sub-matrix includes third modulation coefficients of B component gray values of all pixels of the image source, and the third modulation coefficients are obtained by normalizing the B component gray values according to the G component gray values.

4. The image source coding method according to any one of claims 1 to 3, characterized by, The light waveguide module includes a coupling-in grating, a pupil expanding grating and a coupling-out grating, and the grating parameters include coupling-in grating parameters, pupil expanding grating parameters and coupling-out grating parameters.

5. The image source coding method according to any one of claims 2 or 3, characterized by, The value range of the value obtained by the normalization processing is 0.5 to 1.

5.

6. The image source coding method according to any one of claims 1 to 3, characterized by, The method further includes obtaining human eye position information of a user wearing the near-eye display device, and selecting a modulation matrix corresponding to the human eye position information according to the human eye position information.

7. An image source coding apparatus characterized by comprising: The device is used for a near-eye display device including a micro display and a light waveguide module, and the device includes, An encoding matrix adjustment module is configured to adjust an original encoding matrix of the image source according to a modulation matrix to obtain a first encoding matrix; The modulation matrix is generated according to grating parameters of the light waveguide module, and the modulation matrix includes modulation coefficients of pixel gray values of the image source; wherein the grating parameters are used to perform field of view uniformity and color uniformity simulation of the light waveguide module to obtain corresponding design results; the design results are used to calculate the modulation coefficients; An image encoding module is configured to encode the image source according to the first encoding matrix, and the encoded image source is emitted through the micro display to form an emitted light field, and the emitted light field is propagated through the light waveguide module to output a display image.

8. A near-eye display device, comprising: The device comprises a main control board, a micro display, an optical waveguide module and an eye tracking module; the main control board is connected with the micro display, and the main control board is connected with the eye tracking module; The eye tracking module acquires human eye position information of a user wearing the near-eye display device; The main control board generates a modulation matrix according to grating parameters of the optical waveguide module, and the modulation matrix comprises modulation coefficients of pixel gray scale values of an image source; According to the human eye position information, a modulation matrix corresponding to the human eye position information is selected; wherein the grating parameters are used for field of view uniformity and color uniformity simulation of the optical waveguide module to obtain corresponding design results; the design results are used for calculation of the modulation coefficients; The main control board adjusts an original encoding matrix of the image source according to the modulation matrix corresponding to the human eye position information to obtain a second encoding matrix; The main control board encodes the image source according to the second encoding matrix, and the encoded image source is emitted through the micro display to form an emitted light field; the emitted light field is propagated through the optical waveguide module to output a display image. 9.An electronic device comprising a memory and a processor, the electronic device characterized by, The memory stores a computer program, and the processor is configured to run the computer program to execute the image source encoding method in any one of claims 1 to 6.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the image source encoding method in any one of claims 1 to 6.

Images