A display method, device, apparatus, medium and product of a near-eye display device

Abstract

The application discloses a display method, device, equipment, medium and product of a near-eye display device, and the method comprises the following steps: acquiring the current eye movement state and individual refractive parameters of the wearer of the near-eye display device; determining a micro area set corresponding to a to-be-compensated aberration area in a waveguide exit light field according to the current eye movement state and the spatial distribution information of the current display scene; determining the target compensation wavefront amount required by each micro area according to the micro area set, the individual refractive parameters and a preset wavefront error mapping database; determining the target driving signal of each micro area according to the target compensation wavefront amount and a pre-established micro area response matrix; and driving each micro area in the micro area set to perform local phase modulation according to the target driving signal, so as to obtain a compensated display picture. By driving the micro area array to perform local phase modulation, targeted compensation of aberration is realized, and the problems of edge blur, uneven brightness and individual vision adaptation of the near-eye display device are effectively improved.

Description

Technical Field

[0001] This application relates to the field of near-eye display technology, and in particular to a display method, apparatus, device, medium and product for a near-eye display device. Background Technology

[0002] Near-eye display devices (such as AR glasses) transmit image light to the human eye through waveguides, enabling the superposition of virtual images with the real environment. Their imaging quality directly affects the user's visual experience and comfort.

[0003] Traditional waveguides transmit light through total internal reflection, and the light intensity distribution at their exit pupil is naturally non-uniform (exhibiting a Gaussian distribution or having bright spots). To ensure uniformity, existing technologies often require the introduction of homogenizing structures (such as optimized diffraction grating design or microlens arrays) to improve the brightness uniformity and expand the field within the eyebox.

[0004] However, the above solutions cannot compensate for the inherent residual aberrations such as field curvature and distortion of the waveguide system itself, nor can they adaptively correct for the refractive differences of different users (such as myopia or astigmatism). As a result, the final display image observed by the user still has aberration problems such as blurred edges and distortion, which affects the clarity and comfort of viewing. Summary of the Invention

[0005] This application provides a display method, apparatus, device, medium, and product for near-eye display devices to achieve targeted aberration compensation and effectively improve issues such as edge blurring, uneven brightness, and individual visual acuity adaptation in near-eye display devices.

[0006] According to a first aspect of the embodiments of this application, a display method for a near-eye display device is provided, comprising: To acquire the current eye movement status and individual refractive parameters of the wearer of the near-eye display device; Based on the current eye movement state and the spatial distribution information of the current display scene, determine the set of micro-regions corresponding to the aberration regions to be compensated in the waveguide emitted light field; Based on the set of micro-regions, the individual refractive parameters, and the preset wavefront error mapping database, determine the target compensation wavefront amount required for each micro-region; Based on the target compensation wavefront amount and the pre-established micro-region response matrix, the target driving signal of each micro-region is determined; Based on the target driving signal, each micro-region in the micro-region set is driven to perform local phase modulation to obtain a compensated display image.

[0007] According to a second aspect of the embodiments of this application, a display device for a near-eye display device is provided, comprising: The information acquisition module is used to acquire the current eye movement status and individual refractive parameters of the wearer of the near-eye display device; The micro-area determination module is used to determine the set of micro-areas corresponding to the aberrations to be compensated in the waveguide emitted light field based on the current eye movement state and the spatial distribution information of the current display scene. The compensation determination module is used to determine the target compensation wavefront amount required for each micro-region based on the micro-region set, the individual refractive parameters, and a preset wavefront error mapping database. The signal determination module is used to determine the target driving signal of each micro-region based on the target compensation wavefront amount and the pre-established micro-region response matrix; The screen display module is used to drive each micro-region in the micro-region set to perform local phase modulation according to the target driving signal, so as to obtain the compensated display screen.

[0008] According to a third aspect of the embodiments of this application, a near-eye display device is provided, the near-eye display device comprising: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the display method of the near-eye display device according to any embodiment of this application.

[0009] According to a fourth aspect of the embodiments of this application, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions, the computer instructions being configured to cause a processor to execute and implement the display method of the near-eye display device according to any embodiment of this application.

[0010] According to a fifth aspect of the embodiments of this application, a computer program product is also provided, the computer program product including a computer program, which, when executed by a processor, implements the display method of the near-eye display device of any embodiment of this application.

[0011] The technical solution of this application embodiment obtains the current eye movement state and individual refractive parameters of the wearer of the near-eye display device; based on the current eye movement state and the spatial distribution information of the current display scene, it determines the set of micro-areas corresponding to the aberrations to be compensated in the wavefront emitted light field; based on the micro-area set, individual refractive parameters, and a preset wavefront error mapping database, it determines the target compensation wavefront amount required for each micro-area; based on the target compensation wavefront amount and a pre-established micro-area response matrix, it determines the target driving signal for each micro-area; based on the target driving signal, it drives each micro-area in the micro-area set to perform local phase modulation, thereby obtaining the compensated display image. By driving the micro-area array to perform local phase modulation, targeted aberration compensation is achieved, effectively improving the problems of edge blurring, uneven brightness, and individual visual adaptation in near-eye display devices.

[0012] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this application, nor is it intended to limit the scope of this application. Other features of this application will become readily apparent from the following description. Attached Figure Description

[0013] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0014] Figure 1 This is a flowchart of a display method for a near-eye display device according to Embodiment 1 of this application; Figure 2 This is an example diagram of the central viewing micro-region weighting of a display method for a near-eye display device according to Embodiment 1 of this application; Figure 3 This is an example diagram of the right-side viewing micro-region weighting of a display method for a near-eye display device according to Embodiment 1 of this application; Figure 4 This is a flowchart of a display method for a near-eye display device according to Embodiment 2 of this application; Figure 5 This is a schematic diagram of the conversion in a display method for a near-eye display device according to Embodiment 2 of this application; Figure 6 This is a schematic diagram of visual region division in a display method for a near-eye display device according to Embodiment 2 of this application; Figure 7 This is a schematic diagram of the device structure in a display method for a near-eye display device according to Embodiment 2 of this application; Figure 8This is a comparison diagram of the edge field curves of a display method for a near-eye display device according to Embodiment 2 of this application; Figure 9 This is a schematic diagram of the structure of a display device for a near-eye display device according to Embodiment 3 of this application; Figure 10 This is a schematic diagram of the structure of a near-eye display device that implements an embodiment of this application. Detailed Implementation

[0015] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0016] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0017] Example 1 Figure 1 This application provides a flowchart of a display method for a near-eye display device according to Embodiment 1. This embodiment is applicable to situations where dynamic aberration compensation and brightness uniformity of the emitted light field of a near-eye display device are jointly controlled. This method can be executed by the display device of the near-eye display device, which can be implemented in hardware and / or software and can be configured within the near-eye display device. Figure 1 As shown, the method includes: S110. Obtain the current eye movement status and individual refractive parameters of the wearer of the near-eye display device.

[0018] In this embodiment, the near-eye display device refers to a display device that can be worn on the head by a user, including augmented reality glasses, smart head-mounted displays, etc., which includes a waveguide sheet and an image source for presenting virtual image information to the user. The wearer refers to the individual user who wears and uses the near-eye display device, i.e., the user corresponding to the human eye. The current eye movement state refers to the user's eye movements and fixation information at the current moment, including at least the pupil center coordinates, line of sight direction, and the current eye movement range. Individual refractive parameters refer to the optical refractive characteristics of the user's individual eyeball, including spherical power (myopia / hyperopia), cylindrical power (astigmatism), astigmatic axis, and higher-order aberration parameters.

[0019] Specifically, an eye-tracking camera, fixedly installed inside the frame of the near-eye display device, can capture real-time images of the user's eyes. After processing by an eye-tracking algorithm, the processor can obtain the user's current pupil center coordinates, gaze direction, and the spatial position of the current eye movement range. This information serves as a basis for subsequently determining which region of the waveguide emitted light field the user is focusing on. The processor can obtain the wearer's individual refractive parameters through user initial calibration. For example, during the initial wear, the system uses an interactive vision calibration process, such as focusing on the central cross, gradually adjusting the image clarity, and allowing the user to select a clearer image, to gradually approximate the user's true refractive state. The final calibration result is then stored as the user's individual refractive parameters in the terminal device. Alternatively, the processor can acquire information through the eye-tracking camera. It can use the camera to collect eye feature information (such as reflected light information) and reconstruct the wavefront distribution of the user's eyeball based on a preset optical model, thereby calculating the user's individual refractive parameters, including spherical power, cylindrical power, astigmatism axis, and higher-order aberration parameters.

[0020] S120. Based on the current eye movement state and the spatial distribution information of the current display scene, determine the set of micro-regions corresponding to the aberration regions to be compensated in the waveguide emitted light field.

[0021] In this embodiment, the spatial distribution information of the currently displayed scene refers to the spatial distribution information of the currently presented image content within the field of view (FOV). For example, if the user is looking at a navigation arrow, the FOV is located in the upper right corner. The waveguide emitted light field refers to the light field distribution that is emitted from the waveguide coupling area and projected onto the user's eye movement range after the image light propagates through total internal reflection inside the waveguide sheet. The aberration region to be compensated refers to the area in the waveguide emitted light field where the image quality is degraded due to optical system aberrations or individual user refractive aberrations, and wavefront compensation is required. A micro-area set refers to a collection of one or more micro-areas corresponding to the aberration to be compensated, formed by patterning a dynamic light field modulation layer on the waveguide coupling region into multiple independent driving units. The modulation layer (such as liquid crystal) can be patterned into multiple independently controllable micro-areas (Pixels / Tiles). Each micro-area corresponds to a sub-region or a field of view range of the waveguide pupil. Instead of uniformly modulating the entire waveguide, a partitionable dynamic light field modulation layer is set in the waveguide coupling region. Local light field reconstruction is achieved through multiple independently controllable micro-area units. Each micro-area corresponds to a specific eyebox region and field of view range, thereby achieving dynamic aberration compensation and illumination equalization control for local fields of view.

[0022] It should be noted that the dynamic light field modulation layer can be implemented using liquid crystal materials, electro-optic crystal materials, or other electrically controllable tunable refractive index materials, with liquid crystal materials being preferred. Those skilled in the art should understand that any material capable of changing the local refractive index under the action of an external driving signal, thereby applying phase modulation to the transmitted light wave, is applicable to this application.

[0023] Specifically, the processor can determine the intersection point between the current line of sight and the waveguide coupling region based on the current eye movement state; simultaneously, based on the spatial distribution information of the current display scene, it can determine the key areas that the user may focus on in the current display image and use these key areas as the predicted line of sight focus areas; combining the intersection point position with the predicted line of sight focus areas, it can determine the waveguide emission area corresponding to the current optical path and the waveguide emission area that needs to be compensated based on the scene content; then, based on the preset mapping relationship between the eye movement range coordinates and the waveguide coupling region coordinates, it can map the determined waveguide emission area to the corresponding micro-regions among the multiple independent micro-regions patterned by the dynamic light field modulation layer on the waveguide coupling region, forming a set of micro-regions to be driven.

[0024] S130. Based on the micro-region set, individual refractive parameters, and the preset wavefront error mapping database, determine the target compensation wavefront amount required for each micro-region.

[0025] In this embodiment, the wavefront error mapping database refers to the mapping data that is pre-established through waveguide optical system modeling and offline optical simulation, reflecting the distribution of wavefront errors of the waveguide system at different field of view angles. The target compensation wavefront amount refers to the amount of wavefront compensation required for each micro-region to make the actual wavefront approximate the ideal wavefront.

[0026] Individual refractive parameters include: spherical power, cylindrical power, astigmatic axis, and higher-order aberration parameters.

[0027] In this embodiment, spherical power measures the difference between the overall refractive power of the eye (i.e., the ability to converge or diverge light) and that of an emmetropic eye (a normal eye that can form a clear image without any correction), reflecting the degree of myopia or hyperopia. The unit is diopters (D). Spherical power determines the overall degree of defocus in the user's eye and can be converted into a corresponding wavefront error. This wavefront error is then used as a component of the target compensation wavefront amount, driving the corresponding micro-area to generate reverse phase compensation, allowing the user to see virtual images clearly without wearing additional glasses. Cylindrical power measures the degree of difference in refractive power along different meridians of the eye, reflecting the severity of astigmatism. The unit is diopters (D). Cylindrical power characterizes the direction-related astigmatic component in aberrations and can be incorporated into the wavefront error model, causing differentiated phase modulation amounts in micro-areas of different directions, thereby eliminating directional imaging blur. The astigmatic axis (Axis) indicates the principal direction corresponding to the cylindrical lens power, i.e., the reference direction angle required for astigmatism correction, measured in degrees (°). Its value typically ranges from 0° to 180°. The astigmatic axis, together with the cylindrical lens power, determines the spatial directional distribution of astigmatic aberrations. The axis direction can be used to determine the phase modulation distribution of each micro-area in the micro-array in different directions, enabling the compensation wavefront to accurately match the spatial directional characteristics of the user's astigmatism and achieve targeted directional aberration correction. Higher-order aberrations refer to the more complex wavefront distortion components existing in the ocular optical system, in addition to the lower-order aberrations (defocus and astigmatism) corresponding to the spherical and cylindrical lenses. Higher-order aberrations are typically represented mathematically using Zernike polynomials, including but not limited to: coma (out-of-axis point imaging produces a comet-like tail), spherical aberration (the focal positions of edge rays differ from those of the central rays), trefoil, etc. Higher-order aberration parameters incorporate complex and detailed wavefront distortion information into the error model. They can be superimposed with lower-order aberrations to form a complete actual wavefront, allowing the calculation of the target compensation wavefront amount containing higher-order components.

[0028] Specifically, the processor can acquire individual refractive parameters of the user, including spherical power, cylindrical power, astigmatism axis, and higher-order aberration parameters; simultaneously, based on the current eye movement position, it retrieves the system aberration parameters corresponding to the current optical path from a pre-established waveguide system field of view and wavefront error mapping database; it superimposes the individual refractive aberration of the user with the system aberration to obtain the actual wavefront of the current optical path; it calculates the difference between the ideal wavefront and the actual wavefront as the wavefront error that needs to be compensated; and then, based on the spatial position and field of view range of each micro-region in the micro-region set, it distributes the wavefront error to each micro-region, determining the target compensation wavefront amount that each micro-region needs to bear.

[0029] S140. Determine the target driving signal for each micro-region based on the target compensation wavefront amount and the pre-established micro-region response matrix.

[0030] In this embodiment, the micro-area response matrix refers to matrix data obtained through offline calibration, used to characterize the mapping relationship between the driving signal and wavefront response of each micro-area, where each column represents the wavefront response characteristics generated by the unit driving signal of the corresponding micro-area. The target driving signal refers to the optimal driving signal of each micro-area determined after optimization to make the phase change generated by the modulation layer approximate the target compensation wavefront amount.

[0031] Specifically, the processor can optimize the solution by minimizing the difference between the target compensation wavefront amount required for each micro-region and the actual phase change that the modulation layer can generate, based on the pre-established micro-region response matrix. At the same time, the solution is constrained by the constraint that the deviation between the actual brightness distribution of the compensated light field within the eye-tracking range and the preset target uniform brightness distribution meets the preset condition. The target driving signal for each micro-region is then determined based on the optimization results.

[0032] S150. Based on the target driving signal, drive each micro-region in the micro-region set to perform local phase modulation to obtain the compensated display screen.

[0033] In this embodiment, local phase modulation refers to changing the local refractive index of each micro-region by applying a corresponding driving signal to each micro-region, thereby causing a phase change in the emitted light from the corresponding local area of ​​the micro-region and achieving wavefront reshaping.

[0034] Specifically, the processor can apply the target driving signal to each micro-region in the micro-region set. By changing the local refractive index of each micro-region, the corresponding phase change is generated in each micro-region. The phase changes of each micro-region act together on the outgoing light field to realize the wavefront reshaping of the outgoing light field. After local phase modulation, the aberration of the outgoing light field is compensated and the brightness distribution tends to be uniform, forming a compensated display image output to the human eye.

[0035] The technical solution of this application embodiment obtains the current eye movement state and individual refractive parameters of the wearer of the near-eye display device; based on the current eye movement state and the spatial distribution information of the current display scene, it determines the set of micro-areas corresponding to the aberrations to be compensated in the wavefront emitted light field; based on the micro-area set, individual refractive parameters, and a preset wavefront error mapping database, it determines the target compensation wavefront amount required for each micro-area; based on the target compensation wavefront amount and a pre-established micro-area response matrix, it determines the target driving signal for each micro-area; based on the target driving signal, it drives each micro-area in the micro-area set to perform local phase modulation, thereby obtaining the compensated display image. By driving the micro-area array to perform local phase modulation, targeted aberration compensation is achieved, effectively improving the problems of edge blurring, uneven brightness, and individual visual adaptation in near-eye display devices.

[0036] As a first optional embodiment of this embodiment, based on the above embodiments, it further includes: When the wearer's gaze is detected to be turning toward the edge of the waveguide coupling region, the weight coefficients of each micro-region in the current focus area are adjusted in the optimization solution based on the region division threshold.

[0037] In this embodiment, the waveguide coupling region refers to the area on the waveguide sheet used to gradually diffract the image light propagating from internal total internal reflection to the user's eye movement range, and is usually equipped with an output coupling grating. The region division threshold refers to a preset spatial boundary condition used to distinguish between the central region and the edge region. The key focus region refers to the key display area that the user may focus on or is about to focus on, determined based on the spatial distribution information of the image content in the current display scene, such as the field of view area where navigation arrows, interactive objects, etc., are located. The weight coefficient refers to the priority or importance quantification parameter assigned to different micro-regions or different regions during the optimization process. The larger the weight coefficient, the higher the priority given to the micro-region in the optimization process, and the higher the corresponding aberration compensation accuracy requirement.

[0038] Specifically, the processor continuously tracks the user's pupil center coordinates and gaze direction using an eye-tracking camera fixed inside the frame. When the processor detects a change in the user's gaze direction, it obtains the intersection of the user's gaze and the waveguide plane as the gaze center. Based on the spatial distribution information of the current display scene, the processor identifies the field of view of key content (such as navigation arrows or interactive objects) in the currently displayed image. Combined with the gaze area determined by the current eye tracking, the processor jointly determines the key attention area that requires aberration compensation. The distance between each position within the key attention area and the gaze center is calculated. When the distance is less than the first region division threshold, it is defined as the central region; when the distance is greater than the second region division threshold, it is defined as the edge region, with the area in between being a transition region. During the optimization process, the processor can set the weight coefficients for the edge and central regions separately. The closer to the gaze center, the higher the base weight. The processor can increase the weight coefficients of these micro-regions belonging to the edge region by 1.2-1.5 times based on the base weights, giving them a higher priority in the optimization objective function; for micro-regions corresponding to non-key attention areas, their weight coefficients are maintained or reduced. This weighting adjustment prioritizes aberration compensation accuracy in key areas of interest during the optimization process. Based on the adjusted weighting coefficients, the processor combines the target compensation wavefront amount required for each micro-region with a pre-established micro-region response matrix. The optimization objective is to minimize the difference between the target compensation wavefront amount and the actual phase change that the modulation layer can produce, with brightness uniformity as a constraint. This determines the optimal driving signal for each micro-region. The processor then uses these optimal driving signals to drive local phase modulation in the key areas of interest and other micro-regions. Because the key areas of interest have higher weighting coefficients, their aberration compensation accuracy is prioritized, ensuring that the edge image remains clear and has uniform brightness even when the user's gaze shifts to the edge areas.

[0039] Based on the above embodiments, the steps for adjusting the weight coefficients of each micro-region in the current key area of ​​focus during the optimization solution based on the region division threshold can be refined as follows: Based on the region division threshold, the current micro-regions in the current key focus area are classified into types to obtain the micro-region type of each current micro-region; the weight coefficient of each current micro-region in the optimization solution is adjusted according to the micro-region type.

[0040] In this embodiment, the current focus area refers to the gaze area determined based on the current eye tracking. The current micro-region refers to one or more independent, controllable micro-region units within the dynamic light field modulation layer set on the waveguide coupling region, corresponding to the current display area to be processed. The micro-region type refers to the category attribute determined after classification based on the spatial location of the current micro-region in the waveguide coupling region or its corresponding field of view, combined with a region division threshold; this includes central region micro-regions and edge region micro-regions. The weighting coefficient refers to the priority or importance quantification parameter assigned to different micro-region types during the optimization process, used to adjust the contribution of different micro-region types to the optimization objective function.

[0041] Specifically, the processor can determine the user's current gaze position based on the current eye movement state, identify the user's current focus area, and recognize the specific location range and field of view range on the waveguide coupling area corresponding to the focus area. Within the dynamic light field modulation layer set on the waveguide coupling area, it identifies the micro-regions corresponding to the current focus area as the current micro-regions. The processor can use the intersection of the user's gaze and the waveguide plane as the gaze center and calculate the distance between each location within the focus area and the gaze center. By comparing the distance with a region segmentation threshold, it determines the type of micro-region corresponding to the distance. The processor can assign different base weight coefficients to different micro-region types based on their priority requirements in aberration compensation. For example, when the eye moves towards the edge of the field of view, the compensation weight for the corresponding edge region is increased by n times compared to its default weight. The processor can optimize the solution based on the adjusted weight coefficients, with the goal of minimizing the difference between the target compensation wavefront amount and the actual phase change that the modulation layer can produce, and with brightness uniformity as the constraint. The solution results enable higher priority micro-area types to obtain more accurate wavefront compensation. Based on the solution results, the processor determines the driving signal for each micro-area and drives each micro-area to perform local phase modulation, outputting aberrations to obtain a prioritized compensated display image.

[0042] For example, the peripheral field of view refers to the outer region of the current field of view (FOV) that is far from the user's current gaze center. The processor can first obtain the current gaze point through eye tracking, and then calculate the distance between each location and the gaze point: d= Wherein, the coordinates of the current gaze center are (x, y), and the coordinates of other positions are (x, y). g ,y g ).

[0043] The region segmentation thresholds include two parts: the central region threshold R1 and the peripheral region threshold R2. d≤R1: central region; R1<d<R2: transition region; d≥R2: edge field of view region.

[0044] Therefore, the edge field of view is not fixed, but dynamically determined relative to the current gaze point.

[0045] For example, a specific example can be used to illustrate the weight corresponding to the line of sight. Figure 2 An example diagram of the central viewing micro-region weighting of a display method for a near-eye display device provided in Embodiment 1 of this application is shown, as follows: Figure 2 As shown, Figure 2 This diagram illustrates the weight coefficient distribution of each micro-region when the user's gaze is at the center. The grayscale / color depth is positively correlated with the weight coefficient, with the central region having the highest weight. Therefore, the lighter the color, the lower the weight coefficient as the color decreases outwards, resulting in darker colors in the diagram. It can be seen that when the user's gaze is at the center, the formula above can be used to divide the micro-regions within the area of ​​focus into edge and central regions. By correlating the brightness and darkness with the weight coefficients of these micro-regions, it can be observed that the central region has the highest weight coefficient, gradually decreasing outwards, with the edge regions having the lowest weight coefficient. Figure 3 This application provides an example diagram of the right-side viewing micro-region weighting for a display method of a near-eye display device according to Embodiment 1. Figure 3 The diagram shows the corresponding shift of the weight peak after the user's gaze moves to the right. The weight change is continuous and gradual, ensuring a uniform transition of brightness within the eye movement range. It can be seen that when the user shifts to the right, the central and peripheral regions also change. The original peripheral region now becomes the central region, and the corresponding weight coefficient increases by n times. The weight peak shifts to the right with the position of the human eye, and the weight change is continuous and gradual, ensuring the uniformity of brightness in the new exit pupil region.

[0046] An optional embodiment of this application, in which the type of each micro-region is dynamically identified and the corresponding weight coefficient is dynamically adjusted as the user's gaze moves, achieves a dynamic balance between aberration compensation and brightness uniformity, prioritizes ensuring the imaging clarity of the gaze area, and at the same time takes brightness uniformity as a constraint to ensure the consistency of brightness across the entire field of view.

[0047] Example 2 Figure 4 This is a flowchart illustrating a display method for a near-eye display device according to Embodiment 2 of this application. This embodiment is a further refinement of the above embodiment. Figure 4 As shown, the method includes: S201. Obtain the current eye movement status and individual refractive parameters of the wearer of the near-eye display device.

[0048] S202. Obtain the pupil center coordinates, gaze direction, and current eye movement range position in the current eye movement state.

[0049] In this embodiment, the pupil center coordinates refer to the three-dimensional spatial coordinates of the user's pupil center in the eye-tracking camera coordinate system or a preset eye coordinate system, used to determine the current spatial position of the user's eyeball. The gaze direction refers to the gaze vector extending outward from the center of the user's eyeball through the pupil center, used to determine the direction the user is currently looking in. The current eye movement range position refers to the specific position of the user's pupil within the three-dimensional spatial region of the eye movement range (Eyebox), which refers to the area within the frame where the human eye can see the entire image.

[0050] Specifically, the processor can acquire real-time images of the user's eyes through an eye-tracking camera fixed inside the frame. After processing by the eye-tracking algorithm, it can obtain the user's current pupil center coordinates, gaze direction, and the spatial position of the current eye movement range.

[0051] S203. Based on the spatial distribution information of the current display scene, determine the key areas of interest and area information in the current display image.

[0052] In this embodiment, the key focus area refers to the display area in the currently displayed image that the user may pay attention to or is about to pay attention to, determined based on the spatial distribution information of the image content, such as the field of view area where navigation arrows, interactive objects, etc. are located. Area information refers to the spatial attribute data such as the position, range, and boundaries of the key focus area within the displayed image or field of view.

[0053] Specifically, the processor can obtain the currently presented image content through the scene rendering module, and based on the spatial distribution information of the image content, identify the key areas of interest in the currently displayed image, such as navigation arrows, interactive objects, prompts, and other key display areas that the user may or will pay attention to. At the same time, it can obtain information such as the position, range, and boundaries of the key areas of interest within the display image or field of view.

[0054] S204. Transform the pupil center coordinates and line of sight direction to the waveguide coordinate system, and combine the region information to determine the intersection of the line of sight and the waveguide coupling region and the aberration region to be compensated.

[0055] In this embodiment, the waveguide coordinate system refers to a three-dimensional spatial coordinate system established with the waveguide sheet as a reference, used to describe the spatial positional relationship between light rays, line of sight, and waveguide structure. The intersection point of the line of sight and the waveguide coupling region refers to the spatial position point where the user's line of sight direction vector intersects the plane where the waveguide coupling region is located on the propagation path. This intersection point is used to determine which local position in the waveguide coupling region the user is currently viewing.

[0056] Specifically, the processor can transform the pupil center coordinates and the line of sight direction into the waveguide coordinate system based on the preset transformation relationships between the eye coordinate system and the lens frame coordinate system, as well as between the lens frame coordinate system and the waveguide coordinate system. In the waveguide coordinate system, it calculates the intersection point of the line of sight vector and the plane containing the waveguide coupling region. Based on the field of view corresponding to the current line of sight direction, and combined with the waveguide system's field of view and wavefront error mapping database, the processor can determine the aberration regions in the current optical path that cause image quality degradation due to system aberrations.

[0057] S205. Based on the intersection point location and the aberration area to be compensated, combined with the preset mapping relationship, determine the waveguide emission area corresponding to the current optical path, and the set of micro-areas corresponding to the waveguide emission area.

[0058] The mapping relationship includes the correspondence between the spatial coordinates of the eye movement range and the coordinates of the waveguide coupling area, as well as the correspondence between the coordinates of the waveguide coupling area and each micro-region. In this embodiment, the mapping relationship refers to the correspondence between the spatial coordinates of the eye movement range and the coordinates of the waveguide coupling area, as well as the correspondence between the coordinates of the waveguide coupling area and the micro-region, established and stored in the lookup table during the factory calibration stage. The waveguide emission area refers to the local area in the waveguide coupling area that corresponds to the current optical path and where the image light is actually emitted into the current user's eye movement range.

[0059] Specifically, the processor can determine the waveguide emission region corresponding to the current optical path based on the intersection of the line of sight and the waveguide coupling region, as well as the aberration region to be compensated, combined with the correspondence between the eye movement range coordinates and the waveguide coupling region coordinates in the preset lookup table; then, based on the correspondence between the waveguide coupling region coordinates and the micro-regions established by the factory calibration, the determined waveguide emission region is mapped to the corresponding micro-regions in the dynamic optical field modulation layer, forming a set of micro-regions to be driven for subsequent aberration compensation and optical field modulation.

[0060] For example, a specific example can be used to demonstrate the conversion of eye-tracking information into the visual field. Figure 5 This is a schematic diagram illustrating the conversion process in a display method for a near-eye display device provided in Embodiment 2 of this application, as shown below. Figure 5 As shown, the pupil position and gaze vector can be obtained using an eye-tracking camera, thereby determining the pupil center position (x, y) in the eye-tracking coordinate system. e y e , z e ) and line-of-sight vector (g x g y g z After transformations to both the frame coordinate system and the waveguide coordinate system, the intersection point P(x) of the line of sight and the waveguide emitting surface can be determined. p y pThen, based on the intersection point P of the planes, the visual region is determined, such as... Figure 6 As shown, Figure 6 This is a schematic diagram of visual region division in a display method for a near-eye display device provided in Embodiment 2 of this application. Figure 6 As can be seen, the entire waveguide emission surface can be divided into 9 regions, with the central region being V0, and so on. The coordinate range corresponding to each region can be set, thereby determining the visual region where the plane intersection point P is located.

[0061] S206. Based on the current eye movement state and the preset wavefront error mapping database, determine the system aberration corresponding to the current gaze.

[0062] In this embodiment, the wavefront error mapping database refers to the mapping data that is pre-established through waveguide optical system modeling and offline optical simulation, reflecting the distribution of wavefront errors of the waveguide system under different field of view angles. It is used to query the corresponding system aberration parameters according to the current line of sight or field of view angle position. System aberration refers to the aberrations inherent in the waveguide optical system itself. For example, aberrations can be characterized using Zernike polynomials, including low-order and high-order aberration terms such as defocus, astigmatism, coma, and spherical aberration.

[0063] Specifically, the processor can query a pre-defined wavefront error mapping database based on the current eye movement's gaze direction or field of view. This database can be pre-established through waveguide optical system modeling and offline optical simulation, recording the wavefront error distribution data corresponding to the waveguide system at different field of view angles. The processor can then use the current gaze direction or field of view angle as an index to retrieve the corresponding system aberration parameters from this database.

[0064] S207. Determine the individual refractive aberration based on the individual refractive parameters, and superimpose the individual refractive aberration with the system aberration to obtain the actual wavefront.

[0065] In this embodiment, individual refractive aberration refers to the wavefront error introduced by the refractive errors (myopia, hyperopia, astigmatism, and higher-order aberrations) of the user's individual eyes. The actual wavefront refers to the actual outgoing wavefront shape after the image light propagates through the waveguide system and is coupled out, with the waveguide system aberrations and the user's individual refractive aberrations superimposed; that is, the true wavefront that includes all aberration factors.

[0066] Specifically, the processor can obtain the aberrations corresponding to the individual user's refractive parameters (i.e., individual refractive aberrations) and superimpose them with the waveguide system aberrations. The two aberrations work together on the outgoing light field to form the actual wavefront containing all aberration factors.

[0067] For example, the actual wavefront It can be determined using the following formula: + in, For system aberrations, This refers to individual refractive aberrations.

[0068] S208. The difference between the ideal wavefront and the actual wavefront is used as the target compensation wavefront quantity.

[0069] In this embodiment, the ideal wavefront refers to the wavefront shape that the emitted light field should exhibit under ideal optical system conditions without any aberrations.

[0070] Specifically, the processor can use an ideal wavefront as a reference to calculate the difference between the ideal wavefront and the actual wavefront. This difference is the wavefront error that needs to be compensated and serves as the target compensation wavefront amount. The processor then calculates the corresponding micro-area driving voltage distribution based on the target compensation wavefront amount, causing the micro-area modulation layer to generate a reverse phase distribution corresponding to the target compensation wavefront amount, thereby achieving real-time wavefront compensation and aberration correction.

[0071] For example, the current target compensation wavefront amount can be determined using the following formula. : - in, For the ideal wavefront.

[0072] S209. Based on the pre-established micro-area response matrix, the optimization objective is to minimize the difference between the target compensation wavefront amount and the actual phase change that the modulation layer can produce.

[0073] In this embodiment, the micro-area response matrix refers to matrix data obtained through offline calibration, which characterizes the mapping relationship between the driving signal and wavefront response of each micro-area. This can be obtained through offline calibration. During the calibration process, a preset driving voltage is applied sequentially to each micro-area, and the corresponding wavefront change is measured using a wavefront detection device, thereby establishing the mapping relationship between the driving signal and the wavefront response. For a modulation layer containing N micro-area units, the driving vector V=[V1,V2,…,VN] represents the driving voltage of each micro-area; each column in the response matrix M represents the wavefront response characteristics generated by the unit driving voltage of the corresponding micro-area. Multiplying the response matrix M by the driving vector V yields the total phase change MV generated by the modulation layer.

[0074] In this embodiment, the modulation layer refers to a dynamic optical field modulation layer disposed in the coupling region of the waveguide sheet. It is patterned into multiple independently controlled micro-regions. Each micro-region changes its local refractive index by applying different driving signals, generating a corresponding phase change, thereby modulating the outgoing light field on the wavefront. The optimization problem refers to finding the optimal combination of driving signals to achieve the best aberration compensation effect while ensuring brightness uniformity.

[0075] Specifically, the processor can use the target compensated wavefront amount as the optimization objective. A pre-established micro-area response matrix is ​​obtained; the product of the micro-area response matrix and the driving vector represents the total phase change that the modulation layer can actually produce under a given driving signal. The processor can optimize by minimizing the difference between the target compensated wavefront amount and the actual achievable phase change. That is, it aims to make the phase change produced by the modulation layer as close as possible to the target compensated wavefront amount. Physically, this means making the modulated actual wavefront as close as possible to the ideal wavefront, thereby minimizing aberrations.

[0076] For example, the optimization problem can be represented by the following formula: Min ΔW-MV The optimal driving vector V is obtained so that the phase change MV generated by the modulation layer is as close as possible to the target compensation wavefront ΔW, thereby realizing dynamic aberration compensation and optical field reconstruction.

[0077] Furthermore, based on the above embodiments, the calibration steps of the micro-area response matrix can be refined as follows: Determine the initial wavefront of the emitted light field of each micro-region when no power is applied; apply a preset driving signal to each micro-region and measure the corresponding wavefront change; establish the signal wavefront mapping relationship between the preset driving signal and the wavefront change of each micro-region, and combine the signal wavefront mapping relationships into a micro-region response matrix.

[0078] In this embodiment, the waveguide emitted light field refers to the light field distribution emitted from the waveguide coupling region and projected onto the user's eye movement range after the image light propagates through total internal reflection within the waveguide sheet. It includes the inherent aberration information of the waveguide system itself. The initial wavefront refers to the original wavefront shape of the waveguide emitted light field when all micro-regions are unpowered and without any phase modulation. The preset drive signal refers to the drive voltage value or drive signal parameter preset during the calibration process to measure the response characteristics of each micro-region. The wavefront change refers to the amount of phase change that occurs in the actual wavefront of the waveguide emitted light field relative to the initial wavefront (unpowered state) after a drive signal is applied to a certain micro-region. The signal wavefront mapping relationship refers to the one-to-one correspondence formed by pairing the preset drive signal (input) with the wavefront change (output) measured under that drive signal. This mapping relationship quantifies the wavefront modulation effect that a specific micro-region can produce under a unit drive signal.

[0079] Specifically, the processor can place all micro-regions in the dynamic optical field modulation layer set on the waveguide coupling region into a non-powered state. The processor can use a wavefront detection device (such as a Hartmann-Shack wavefront sensor) set in the waveguide output optical path to measure the wavefront morphology of the output optical field and obtain the initial wavefront in the non-powered state. This initial wavefront contains the inherent aberrations of the waveguide system itself and serves as a reference for calculating the wavefront changes of each micro-region in subsequent calibration measurements. The processor can traverse from micro-region 1 to micro-region N sequentially according to a preset calibration order. When calibrating a target micro-region, a preset driving signal (e.g., driving vector V=[V1,V2,…,VN]) is applied only to that micro-region, while all other micro-regions remain unpowered. The processor can use a wavefront detection device to measure the emitted wavefront shape after the driving signal is applied, and perform differential calculation between this wavefront and the measured initial wavefront to obtain the wavefront change generated by the micro-region under the preset driving signal. After completing the measurement of one micro-region, the system returns it to the unpowered state, and then performs the same operation on the next micro-region until all micro-regions are calibrated. The processor can pair the preset driving signal of each micro-region with the measured wavefront change (output) under that driving signal to establish a one-to-one signal wavefront mapping relationship for each micro-region. The processor can combine the signal wavefront mapping relationships of all micro-regions in the order of micro-region numbering to form a complete micro-region response matrix. Each column in the matrix corresponds to a micro-region, and the data in each column represents the wavefront response eigenvector generated by that micro-region under a unit driving signal. Once constructed, this matrix is ​​stored in the terminal device and used in subsequent optimization solutions, enabling the processor to quickly solve for the optimal driving signal combination for each micro-region based on the target compensation wavefront amount.

[0080] S210, with the constraint that the deviation between the actual brightness distribution of the compensated light field within the eye-tracking range and the uniform brightness distribution of the target is less than a preset threshold.

[0081] In this embodiment, the actual brightness distribution refers to the actual illuminance distribution formed within the eye-moving range after the driving signal is applied to the micro-area modulation layer, propagated through the waveguide, and denoted as I(V). Once the driving signal is determined, each micro-area generates corresponding phase modulation. The modulated light field propagates through the waveguide and is superimposed within the eye-moving range to form the actual brightness distribution. The target uniform brightness distribution refers to a preset target brightness distribution benchmark within the eye-moving range, used as a brightness uniformity evaluation standard, denoted as I0. This target distribution is typically a uniform distribution with consistent brightness values ​​across the entire eye-moving range. The preset threshold refers to the maximum allowable deviation between the actual brightness distribution and the target uniform brightness distribution, denoted as ε. Based on the Weber-Fechner law of the human eye and AR waveguide industry uniformity standards, the value of this threshold ranges from 0.01 to 0.10 in normalized brightness units, preferably from 0.01 to 0.05. When the deviation is less than this threshold, the human eye cannot perceive the brightness non-uniformity.

[0082] Specifically, since the micro-modulation layer alters the local light energy distribution while achieving wavefront compensation, optimizing only the wavefront difference may result in uneven brightness, such as darkening of the edge region and brightening of the center region. Therefore, a constraint is introduced: the deviation between the actual brightness distribution I(V) of the compensated light field within the eye-tracking range and the target uniform brightness distribution I0 must be less than a preset threshold ε. This constraint ensures that the light field after aberration compensation still maintains the predetermined uniformity.

[0083] For example, the constraint can be expressed as: I0-I(V) <ε S211. The optimization problem is obtained based on the optimization objective and constraints.

[0084] Specifically, the processor can combine the above optimization objectives with constraints to construct a complete optimization problem. The essence of this optimization problem is a constrained multi-objective optimization problem, in which aberration compensation accuracy and brightness uniformity constitute two optimization dimensions that need to be considered.

[0085] S212. Solve the optimization problem to obtain the target driving signal for each micro-region.

[0086] Specifically, the processor can solve the optimization problem according to a preset optimization algorithm, searching for the optimal solution that minimizes wavefront differences within the feasible region that satisfies the brightness uniformity constraint. The optimal driving signal combination V={V1,V2,V3,.....,Vn} obtained by the solution is the target driving signal for each micro-region, corresponding to the driving voltage of micro-regions 1 to n, respectively. This driving signal combination ensures that the modulation layer achieves optimal aberration compensation while maintaining the preset uniformity requirement for brightness distribution within the eye-tracking range. The processor can output the solution result to the driving circuit for subsequent independent driving control of each micro-region.

[0087] S213. Based on the target driving signal, drive each micro-region in the micro-region set to perform local phase modulation to obtain the compensated display screen.

[0088] The technical solution of this application combines eye tracking with scene rendering, performing targeted modulation only in the user's current gaze area and the predicted area of ​​focus. This avoids the problem of insufficient aberration correction caused by uniform processing across the entire screen, effectively compensating for field curvature, distortion, and individual refractive aberrations at the edges of the field of view. It does not increase optical thickness. By integrating the liquid crystal modulation layer onto or inside the waveguide surface, no additional lens group is required, keeping the device thin and light, and ensuring that brightness uniformity is not affected. Brightness uniformity is used as a constraint in the optimization solution to ensure consistent brightness distribution across the entire eyebox after aberration compensation.

[0089] For example, the overall device can be shown with a specific example. Figure 7 This is a schematic diagram of the device structure in a display method for a near-eye display device provided in Embodiment 2 of this application, as shown below. Figure 7 The diagram illustrates the complete optical path of image light emitted from the display screen (optical engine), transmitted through the waveguide, modulated by a dynamic liquid crystal modulation layer in the coupling region, and then entering the human eye. The two design schemes for the liquid crystal modulation layer can be summarized as follows: Scheme 1 involves a liquid crystal micro-area layer attached to the surface of the waveguide sheet, or Scheme 2 involves an integrated liquid crystal layer inside the waveguide sheet. The diagram also shows an eye-tracking camera fixed inside the frame, used to collect real-time data on the user's eye state. The display screen refers to the image source in the near-eye display system (such as a micro-projector, optical engine, or micro-display screen), used to generate a beam carrying image information and direct the image light into the input coupling end of the waveguide. The emitted image light contains the virtual image content to be displayed. The coupling grating refers to the input coupling grating set at the input end of the waveguide, used to couple the image light emitted from the display screen into the interior of the waveguide sheet through diffraction, deflecting the propagation direction of the light and satisfying the condition for total internal reflection propagation inside the waveguide. A waveguide is a transparent, flat optical waveguide element (usually made of glass or resin) used to propagate coupled image light forward in a zigzag path within the waveguide sheet via total internal reflection, transporting the image light from the incident end to the output end while maintaining the transmission of ambient light, thus achieving virtual-real fusion display. An output grating is a coupling grating (emission grating / output grating) located at the output end of the waveguide, used to gradually diffract and couple the image light propagating inside the waveguide to the outside, directing it towards the user's eye movement range. This output coupling grating extends to cover the waveguide output area, achieving exit pupil replication and enlargement. A micro-area liquid crystal modulation layer (dynamic modulation layer) is a dynamic light field modulation layer located in the waveguide output area (usually above or adjacent to the output coupling grating), patterned into multiple independently driven liquid crystal micro-area units. Each micro-area can independently change its local refractive index under the action of a driving signal, applying phase modulation to the passing light wave, thereby achieving local wavefront reshaping and aberration compensation. An eye-tracking camera refers to an eye-tracking camera fixedly installed inside the frame of a pair of glasses. It is used to acquire real-time images of the user's eyes. After processing by an eye-tracking algorithm, the coordinates of the user's pupil center, the direction of gaze, and the current eye movement range are obtained, which serve as the basis for determining the aberration area to be compensated and the corresponding micro-area. The image light propagation path can be represented as follows: the image light emitted from the display screen enters the waveguide through the left-side coupling grating (introduction), propagates forward in a total internal reflection manner inside the waveguide, reaches the right-side coupling area, and is gradually diffracted and coupled out through the exit coupling grating. It then passes through the micro-area liquid crystal modulation layer (dynamic modulation layer) and enters the human eye after being locally phase modulated according to this application.

[0090] For example, to characterize the optimization effect of this application, the modulation transfer function (MTF) can be used as a comparison method to demonstrate the effect. The MTF is a core evaluation index of the imaging quality of an optical system, used to measure the ability of an optical system to transfer the contrast of an object to the image plane. The MTF value is between 0 and 1. The higher the value, the stronger the system's ability to resolve details at that spatial frequency, and the clearer the image. Figure 8 This is a comparison diagram of the edge field-of-view curves of a display method for a near-eye display device provided in Embodiment 2 of this application, as shown below. Figure 8 As shown in the figure, the vertical axis represents the MTF value, and the horizontal axis represents the spatial frequency (unit: lp / mm, i.e., line pairs per millimeter). The red dashed line represents the image before optimization, and the blue solid line represents the image after optimization. The higher the spatial frequency, the finer the image details. The diffraction limit refers to the optimal standard of MTF for a given field of view in the AR glasses industry or design specifications. Across the entire frequency range, the optimized MTF value is significantly higher than that before optimization, meaning that the optimized edge field of view MTF is closer to the diffraction limit than before optimization. This proves that this application effectively compensates for edge aberrations, improves edge resolution, and meets the design specifications for edge imaging clarity, ensuring that users do not experience blurriness when viewing the edge of the image.

[0091] Example 3 Figure 9 This is a schematic diagram of the structure of a display device for a near-eye display device provided in Embodiment 3 of this application. Figure 9 As shown, the device includes: Information acquisition module 31 is used to acquire the current eye movement state and individual refractive parameters of the wearer of the near-eye display device; The micro-area determination module 32 is used to determine the set of micro-areas corresponding to the aberrations to be compensated in the waveguide emitted light field based on the current eye movement state and the spatial distribution information of the current display scene. The compensation determination module 33 is used to determine the target compensation wavefront amount required for each micro-region based on the micro-region set, the individual refractive parameters, and the preset wavefront error mapping database. The signal determination module 34 is used to determine the target driving signal of each micro-region based on the target compensation wavefront amount and the pre-established micro-region response matrix; The screen display module 35 is used to drive each micro-region in the micro-region set to perform local phase modulation according to the target driving signal, so as to obtain the compensated display screen.

[0092] The individual refractive parameters include: spherical power, cylindrical power, astigmatic axis, and higher-order aberration parameters.

[0093] Furthermore, the micro-region determination module 32 is specifically used for: Obtain the pupil center coordinates, gaze direction, and current eye movement range position in the current eye movement state; Based on the spatial distribution information of the current display scene, determine the key areas of interest and their information in the currently displayed image; The pupil center coordinates and the line of sight direction are transformed into the waveguide coordinate system. Combined with the region information, the intersection of the line of sight and the waveguide coupling region and the aberration region to be compensated are determined. Based on the intersection point location and the aberration region to be compensated, combined with a preset mapping relationship, the waveguide emission region corresponding to the current optical path and the set of micro-regions corresponding to the waveguide emission region are determined; wherein, the mapping relationship includes the correspondence between the spatial coordinates of the eye movement range and the coordinates of the waveguide coupling region, and the correspondence between the coordinates of the waveguide coupling region and each micro-region.

[0094] Furthermore, the compensation determination module 33 is specifically used for: Based on the current eye movement state and the preset wavefront error mapping database, determine the system aberration corresponding to the current gaze. The individual refractive aberration is determined based on the individual refractive parameters, and the individual refractive aberration is superimposed with the system aberration to obtain the actual wavefront; The difference between the ideal wavefront and the actual wavefront is used as the target compensation wavefront amount.

[0095] Furthermore, the signal determination module 34 is specifically used for: Based on the pre-established micro-area response matrix, the optimization objective is to minimize the difference between the target compensation wavefront amount and the actual phase change that the modulation layer can produce. The constraint condition is that the deviation between the actual brightness distribution of the compensated light field within the eye movement range and the uniform brightness distribution of the target is less than a preset threshold. The optimization problem is derived based on the optimization objective and the constraints. Solve the optimization problem to obtain the target driving signal for each micro-region.

[0096] Optionally, the device further includes a micro-area determination module, specifically used for: The calibration steps for the micro-region response matrix include: Determine the initial wavefront of the emitted light field of each micro-region when no power is applied; A preset driving signal is applied to each of the micro-regions, and the wavefront change corresponding to each micro-region is measured. Establish a signal wavefront mapping relationship between the preset driving signal and the wavefront change in each micro-region, and combine the signal wavefront mapping relationships into the micro-region response matrix.

[0097] Optionally, the device further includes: The weight determination module is used to adjust the weight coefficients of each micro-region in the current focus area in the optimization solution based on the region division threshold when the wearer's line of sight is detected to be rotating towards the edge of the waveguide coupling region.

[0098] Furthermore, the weight determination module is specifically used for: The adjustment of the weight coefficients of each micro-region in the optimization solution based on the region division threshold includes: Based on the region division threshold, the current micro-regions in the current key focus region are classified by type to obtain the micro-region type of each current micro-region; Adjust the weight coefficients of each current microregion in the optimization solution according to the microregion type.

[0099] The display device of the near-eye display device provided in the embodiments of this application can execute the display method of the near-eye display device provided in any embodiment of this application, and has the corresponding functional modules and beneficial effects of executing the method.

[0100] Example 4 Figure 10 A schematic diagram of a near-eye display device 40, which can be used to implement embodiments of this application, is shown. The near-eye display device is intended to represent various forms of wearable display devices, such as augmented reality glasses, virtual reality headsets, mixed reality helmets, smart glasses, head-mounted displays, helmet displays, smart goggles, and other near-eye display devices suitable for wearing on a user's head and capable of projecting display images to the user's eyes. The near-eye display device can also represent a display module integrated into other portable electronic devices, such as near-eye display components integrated into smart helmets, helmets, goggles, smartwatches, head-mounted communication devices, and other similar wearable electronic devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the application described and / or claimed herein.

[0101] like Figure 10As shown, the near-eye display device 40 includes at least one processor 41 and a memory, such as a read-only memory (ROM) 42 and a random access memory (RAM) 43, communicatively connected to the at least one processor 41. The memory stores computer programs executable by the at least one processor. The processor 41 can perform various appropriate actions and processes based on the computer program stored in the ROM 42 or loaded from storage unit 48 into the RAM 43. The RAM 43 may also store various programs and data required for the operation of the near-eye display device 40. The processor 41, ROM 42, and RAM 43 are interconnected via a bus 44. An input / output (I / O) interface 45 is also connected to the bus 44.

[0102] Multiple components in the near-eye display device 40 are connected to the I / O interface 45, including: an input unit 46, such as a keyboard, mouse, etc.; an output unit 47, such as various types of monitors, speakers, etc.; a storage unit 48, such as a disk, optical disk, etc.; and a communication unit 49, such as a network card, modem, wireless transceiver, etc. The communication unit 49 allows the near-eye display device 40 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0103] Processor 41 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 41 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 41 performs the various methods and processes described above, such as display methods for near-eye display devices.

[0104] In some embodiments, the display method of the near-eye display device may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 48. In some embodiments, part or all of the computer program may be loaded and / or installed on the near-eye display device 40 via ROM 42 and / or communication unit 49. When the computer program is loaded into RAM 43 and executed by processor 41, one or more steps of the display method of the near-eye display device described above may be performed. Alternatively, in other embodiments, processor 41 may be configured to perform the display method of the near-eye display device by any other suitable means (e.g., by means of firmware).

[0105] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0106] Computer programs used to implement the methods of this application may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0107] In the context of this application, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium can be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0108] To provide interaction with a user, the systems and techniques described herein can be implemented on a near-eye display device having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the near-eye display device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0109] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0110] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0111] In one embodiment, the present application also includes a computer program product, which includes a computer program that, when executed by a processor, implements the display method of the near-eye display device according to any embodiment of the present application.

[0112] In the implementation of the computer program product, computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof. Programming languages ​​include object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0113] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this application can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this application can be achieved, and this is not limited herein.

[0114] The specific embodiments described above do not constitute a limitation on the scope of protection of this application. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A display method for a near-eye display device, characterized in that, include: To acquire the current eye movement status and individual refractive parameters of the wearer of the near-eye display device; Based on the current eye movement state and the spatial distribution information of the current display scene, determine the set of micro-regions corresponding to the aberration regions to be compensated in the waveguide emitted light field; Based on the set of micro-regions, the individual refractive parameters, and the preset wavefront error mapping database, determine the target compensation wavefront amount required for each micro-region; Based on the target compensation wavefront amount and the pre-established micro-region response matrix, the target driving signal of each micro-region is determined; Based on the target driving signal, each micro-region in the micro-region set is driven to perform local phase modulation to obtain a compensated display image.

2. The method according to claim 1, characterized in that, The individual refractive parameters include: spherical power, cylindrical power, astigmatism axis, and higher-order aberration parameters.

3. The method according to claim 1, characterized in that, The step of determining the set of micro-regions corresponding to the aberration region to be compensated in the waveguide emitted light field based on the current eye movement state and the spatial distribution information of the current display scene includes: Obtain the pupil center coordinates, gaze direction, and current eye movement range position in the current eye movement state; Based on the spatial distribution information of the current display scene, determine the key areas of interest and their information in the currently displayed image; The pupil center coordinates and the line of sight direction are transformed to the waveguide coordinate system. Combined with the region information, the intersection of the line of sight and the waveguide coupling region and the aberration region to be compensated are determined. Based on the intersection point location and the aberration region to be compensated, combined with a preset mapping relationship, the waveguide emission region corresponding to the current optical path and the set of micro-regions corresponding to the waveguide emission region are determined; wherein, the mapping relationship includes the correspondence between the spatial coordinates of the eye movement range and the coordinates of the waveguide coupling region, and the correspondence between the coordinates of the waveguide coupling region and each micro-region.

4. The method according to claim 1, characterized in that, The step of determining the target compensation wavefront amount required for each micro-region based on the micro-region set, the individual refractive parameters, and a preset wavefront error mapping database includes: Based on the current eye movement state and the preset wavefront error mapping database, determine the system aberration corresponding to the current gaze. The individual refractive aberration is determined based on the individual refractive parameters, and the individual refractive aberration is superimposed with the system aberration to obtain the actual wavefront; The difference between the ideal wavefront and the actual wavefront is used as the target compensation wavefront amount.

5. The method according to claim 1, characterized in that, The step of determining the target driving signal for each micro-region based on the target compensation wavefront amount and the pre-established micro-region response matrix includes: Based on the pre-established micro-area response matrix, the optimization objective is to minimize the difference between the target compensation wavefront amount and the actual phase change that the modulation layer can produce. The constraint condition is that the deviation between the actual brightness distribution of the compensated light field within the eye movement range and the uniform brightness distribution of the target is less than a preset threshold. The optimization problem is derived based on the optimization objective and the constraints. Solve the optimization problem to obtain the target driving signal for each micro-region.

6. The method according to claim 1, characterized in that, The calibration steps for the micro-region response matrix include: Determine the initial wavefront of the emitted light field of each micro-region when no power is applied; A preset driving signal is applied to each of the micro-regions, and the wavefront change corresponding to each micro-region is measured. Establish a signal wavefront mapping relationship between the preset driving signal and the wavefront change in each micro-region, and combine the signal wavefront mapping relationships into the micro-region response matrix.

7. The method according to claim 1, characterized in that, Also includes: When the wearer's gaze is detected to be turning toward the edge of the waveguide coupling region, the weight coefficients of each micro-region in the current focus area are adjusted in the optimization solution based on the region division threshold.

8. The method according to claim 7, characterized in that, The adjustment of the weight coefficients of each micro-region in the optimization solution based on the region division threshold includes: Based on the region division threshold, the current micro-regions in the current key focus region are classified by type to obtain the micro-region type of each current micro-region; Adjust the weight coefficients of each current microregion in the optimization solution according to the microregion type.

9. A display device for a near-eye display system, characterized in that, include: The information acquisition module is used to acquire the current eye movement status and individual refractive parameters of the wearer of the near-eye display device; The micro-area determination module is used to determine the set of micro-areas corresponding to the aberrations to be compensated in the waveguide emitted light field based on the current eye movement state and the spatial distribution information of the current display scene. The compensation determination module is used to determine the target compensation wavefront amount required for each micro-region based on the micro-region set, the individual refractive parameters, and a preset wavefront error mapping database. The signal determination module is used to determine the target driving signal of each micro-region based on the target compensation wavefront amount and the pre-established micro-region response matrix; The screen display module is used to drive each micro-region in the micro-region set to perform local phase modulation according to the target driving signal, so as to obtain the compensated display screen.

10. A near-eye display device, characterized in that, The near-eye display device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the display method of the near-eye display device according to any one of claims 1-8.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that, when executed by a processor, implement the display method of the near-eye display device according to any one of claims 1-8.

12. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the display method of the near-eye display device according to any one of claims 1-8.

Images