An Adaptive Multi-Focal Surface AR Imaging System and Method

By employing a multi-layered cascaded electrically controlled adjustable phase modulation layer and polarization optical module in AR glasses, the depth of the virtual image is matched with the depth of human eye accommodation in real time, solving the problem of mismatch between the depth of the virtual image and the depth of the real object in existing AR glasses, thus improving the user experience and imaging quality of AR glasses.

CN123113147APending Publication Date: 2026-11-17SHENZHEN YINGMU TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611183846.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-08-05
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

When using a single-focal-plane imaging scheme with optical waveguides, the depth of the virtual image cannot be synchronized with the depth of the real object observed by the human eye, resulting in convergence-accommodation conflict. Prolonged wear can cause problems such as dizziness, dry eyes, and visual fatigue. Mechanical zoom schemes are bulky and have high response delays, while static layered multi-display panel schemes have huge computing power costs and severe crosstalk between light fields. The decoupling of virtual and real depths leads to a disconnect between virtual objects and the real scene.

Method used

A multi-layer cascaded electrically controlled adjustable phase modulation layer is used as the core focusing element. The phase distribution of each phase modulation layer is controlled by the driving signal to generate multiple discrete or continuous adjustable virtual focal planes. Combined with eye tracking and depth sensor, the virtual image depth is matched with the human eye's adjustment depth in real time. The polarization optical module is used to suppress light field crosstalk to achieve multi-focal plane imaging without mechanical displacement.

Benefits of technology

It achieves synchronous matching between the depth of virtual images and the depth of human eye adjustment, reduces device power consumption and heat generation, improves the viewing experience in dynamic scenes, eliminates the misalignment and ghosting of virtual and real images, reduces device latency and computing power requirements, and enhances the realism of the fusion between virtual and real scenes.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123113147A_ABST
    Figure CN123113147A_ABST
Patent Text Reader

Abstract

This invention provides an adaptive multi-focal plane AR imaging system and method, belonging to the field of near-eye augmented reality display technology. The system includes a micro-display module, a polarization-multiplexed dynamic phase-focusing optical module, and a waveguide-based perspective imaging module arranged sequentially along the optical path, as well as a multi-source sensing and acquisition unit and a central driving and rendering control unit. The focusing optical module contains at least two electrically controllable phase modulation layers, each independently receiving driving signals to change its phase distribution, superimposing to form multiple adjustable virtual focal planes without mechanical displacement. The system cyclically executes sensing, solving, rendering, driving, imaging, and feedback correction in a closed-loop manner, achieving real-time matching between virtual depth and human eye accommodation and real-world depth, eliminating convergence-accommodation conflicts.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of near-eye augmented reality display technology, and in particular to an adaptive multifocal AR imaging system and method. Background Technology

[0002] Currently, most mainstream AR glasses adopt a single-focal-plane imaging scheme using optical waveguides. All virtual images can only be fixed at a single virtual image distance. When the human eye observes a real object nearby, the lens naturally adjusts, but the depth of the virtual image cannot be matched synchronously, resulting in severe convergence-accommodation conflict (VAC). Wearing these glasses for a long time can easily cause problems such as dizziness, dry eyes, and visual fatigue.

[0003] The shortcomings of existing multi-focal plane AR solutions are represented as follows: 1. Mechanical zoom solutions rely on motors to drive the lens group to shift and zoom, resulting in a large structural volume, mechanical vibration, and high zoom response delay, which cannot meet the lightweight and low-latency requirements of wearable devices. 2. The static layered multi-display panel time-sharing imaging scheme requires synchronous rendering of multiple layers of depth images, which consumes a lot of computing power. The crosstalk between multiple light fields can easily produce ghosting, and it does not take into account the human eye's gaze state for dynamic scheduling. 3. Decoupling of virtual and real depth: The focal plane is generated only based on the layered virtual 3D model without collecting the depth information of the real environment. The depth of field and occlusion relationship of virtual objects are separated from the real scene, resulting in poor realism of virtual-real fusion.

[0004] Therefore, there is a need to provide an adaptive multi-focal plane AR imaging system and method to solve the above problems. Summary of the Invention

[0005] The purpose of this invention is to provide an adaptive multi-focal plane AR imaging system and method, which uses a multi-layer cascaded electrically controlled adjustable phase modulation layer as the core focusing element. The phase distribution of each phase modulation layer is adjusted by driving signals, and multiple discrete or continuous adjustable virtual focal planes are superimposed to generate multiple virtual focal planes. The entire process does not require motor drive or mechanical displacement of lens group.

[0006] To achieve the above objectives, the present invention provides an adaptive multi-focal plane AR imaging system, comprising a micro-display module, a polarization multiplexing dynamic phase focusing optical module, and an optical waveguide perspective imaging module arranged sequentially along the imaging optical path; as well as a multi-source sensing acquisition unit and a central driving and rendering control unit; The polarization-multiplexed dynamic phase-focusing optical module is positioned between the light-emitting side of the micro-display module and the light-incident side of the optical waveguide perspective imaging module; the signal output terminal of the multi-source sensing and acquisition unit is connected to the signal input terminal of the central drive and rendering control unit, the image signal output terminal of the central drive and rendering control unit is connected to the image signal input terminal of the micro-display module, and the drive signal output terminal of the central drive and rendering control unit is connected to the drive signal input terminal of the polarization-multiplexed dynamic phase-focusing optical module.

[0007] Preferably, the light-incident surface of the polarization multiplexing dynamic phase-focusing optical module is connected to the light-outcident surface of the micro-display module, and the light-outcident surface of the polarization multiplexing dynamic phase-focusing optical module is coupled to the light-incident surface of the optical waveguide perspective imaging module. The polarization multiplexing dynamic phase focusing optical module contains at least two electrically controlled adjustable phase modulation layers. Each layer independently receives driving signals to change the phase distribution, and after being superimposed, they form multiple adjustable virtual focal planes without mechanical displacement.

[0008] Preferably, the multi-source sensing and acquisition unit specifically includes an eye-tracking camera, a depth sensor, and an inertial measurement unit; the eye-tracking camera is positioned facing the human eye, the depth sensor is positioned facing the external scene, and the inertial measurement unit is fixed on the housing of the imaging system.

[0009] Preferably, the central drive and rendering control unit specifically includes an image processor, a phase drive chip, and a timing synchronization controller; the image processor generates a layered rendering image, the phase drive chip calculates the drive signal according to the target focal plane depth, and the timing synchronization controller coordinates the image refresh of the micro-display module and the focal plane switching timing of the polarization multiplexing dynamic phase focusing optical module.

[0010] Preferably, the micro-display module is a spatial light modulator that supports time-sequential sliced ​​and layered image output with a refresh rate of not less than 120Hz.

[0011] An adaptive multi-focal plane AR imaging control method, applied to an adaptive multi-focal plane AR imaging system, includes the following iterative execution steps: S1: The multi-source sensing acquisition unit collects sensing data, which specifically includes human eye state data, environmental depth data, and posture data. It calculates binocular disparity to solve for human eye accommodation depth, and transmits the sensing data to the central drive and rendering control unit after preprocessing. S2: Acquire a 3D virtual scene and extract the depth distribution data of virtual objects in the scene; based on the depth adjustment of the human eye, the depth distribution data of virtual objects and the real-world depth map, execute a layered matching strategy, select the main focal plane and at least one auxiliary focal plane, and output the target multi-focal plane set; S3: Perform depth slicing and layered rendering of the same 3D virtual scene according to the target multi-focal plane set, apply depth blur to each layered image, perform parallax compensation according to the pose data, and output the layered images to the micro display module. S4: Output the corresponding driving signal to the polarization multiplexing dynamic phase focusing optical module according to the target focal plane depth, switch the phase distribution of each adjustable phase modulation layer, so that the virtual focal plane formed after the superposition of each layer corresponds to the depth of each focal plane in the target multi-focal plane set. S5: The layered image light output by the micro-display module is modulated by the polarization multiplexing dynamic phase focusing optical module to form a multi-layer beam corresponding to different virtual image distances. After being fused with the external ambient light by the optical waveguide perspective imaging module, it is projected to the human eye. S6: Repeat steps S1 to S5 in the next sampling cycle, and update the target multi-focal surface set in real time based on the newly acquired sensing data to achieve continuous closed-loop adaptive control.

[0012] Preferably, the hierarchical matching strategy in S2 specifically includes: Primary focal plane matching: The virtual object depth with the smallest difference from the human eye's accommodation depth is selected as the primary focal plane. Allocate the highest image brightness and prioritize the use of the phase modulation channel; Auxiliary multi-layer focal plane selection: Select distant and near virtual objects whose depth difference from the main focal plane is greater than a preset threshold ΔD as the auxiliary focal plane set. Limit the number of focal plane layers to avoid multiple layers overlapping and creating ghosting; Virtual-real depth constraint correction: If there is a real object at close range in the area of ​​the gaze, the brightness of the virtual image at the same depth is automatically reduced to enhance the depth-of-field effect of virtual-real occlusion. Output target multifocal plane set The depth of each focal plane target is transmitted to the phase drive chip.

[0013] Preferably, the depth slicing and layering rendering of the 3D virtual scene in S3 specifically includes the following steps: S31: Based on the target focal plane set The 3D virtual scene is sliced ​​at depth, and each layer retains only the model and texture data of the corresponding depth range; S32: Add optical depth blur to layered images, apply Gaussian defocus blur to pixels that are off-focus, and simulate the defocusing imaging characteristics of the human eye. S33: Perform disparity distortion compensation on each layer of images based on pose data, and apply a disparity distortion compensation matrix to each layer of images; S34: Pack the multi-layer image frames processed by the above steps into time sequence and output them to the micro-display module for time-sharing synchronous refresh.

[0014] Preferably, the driving signal in S4 is a driving voltage, and the mapping relationship between the target focal plane depth and the driving voltage of each layer is determined by the following formula: The equivalent refractive power of a single layer is calculated using the following formula: ; The depth of a single-layer imaging layer is calculated using the following formula: ; In the formula, For single-layer driving voltage, , , These are the fitting calibration coefficients for a single-layer metasurface; The total refractive power of multi-layer cascaded lenses is calculated using the following formula: ; The total imaging depth of the multi-layer cascade is calculated using the following formula: ; in, This represents the total number of metasurface layers. This is the driving voltage for the nth layer. , , These are the independent calibration fitting coefficients for the nth metasurface layer; Drive voltage during focal plane switching U ( t Changes according to a linear slope: U ( t )= + · t ,0≤ t ≤Δ T ; Where, Δ T This refers to the transition time for focal plane switching. and These are the switching start voltage and target voltage, respectively. This is for switching time.

[0015] Therefore, the present invention employs the above-described adaptive multi-focal plane AR imaging system and method, and the beneficial technical effects are as follows: (1) This invention uses a high-speed infrared eye-tracking camera to collect the coordinates of both pupils in real time, calculates the current natural accommodation depth of the human eye, and dynamically matches and generates the main focal plane based on this by the central drive and rendering control unit, so that the depth of the virtual image is synchronized with the natural focusing state of the human eye lens. At the same time, combined with the three-dimensional depth map of the real scene collected by the depth sensor, the depth blur and occlusion relationship of the virtual image are corrected in real scene, realizing the linkage matching of the depth of the virtual image, the accommodation depth of the human eye and the depth of the real environment.

[0016] (2) The present invention uses a multi-layer cascaded electrically controlled adjustable phase modulation layer as the core focusing element. The phase distribution of each phase modulation layer is controlled by the driving signal, and multiple discrete or continuous adjustable virtual focal planes are generated by superposition. The whole process does not require motor drive or mechanical displacement of lens group.

[0017] (3) The present invention constructs a complete closed-loop control architecture, realizes instantaneous tracking of changes in human eye gaze, smooth and flicker-free switching of focal plane, and keeps the virtual and real images aligned when the head moves rapidly, which greatly improves the viewing experience in dynamic scenes.

[0018] (4) This invention simultaneously acquires a 3D depth map of the real scene and introduces a real-scene depth constraint during the target focal plane solution process: when there is a real object at close range in the area of ​​gaze, the brightness of the virtual image at the same depth is automatically reduced and the depth blur is enhanced, so that the perspective relationship, occlusion layer and defocus effect of the virtual object are highly consistent with the real scene. At the same time, a parallax distortion compensation matrix is ​​applied to each layer of images based on IMU head pose data to eliminate the misalignment between the virtual image and the real scene caused by head movement.

[0019] (5) This invention synergistically suppresses multi-layer light field crosstalk in two dimensions: the optical layer and the rendering layer. A polarization beam splitting multiplexing layer is set in the optical waveguide perspective imaging module to isolate beams from different focal layers using polarization states. During the rendering stage, Gaussian defocus blur is actively applied to pixels that deviate from the current focal plane to simulate the defocus imaging characteristics of the human eye. During the driving stage, a cancellation phase is applied to the non-currently active phase modulation layer to further reduce the interference of the inactive layer on the imaging light field. The triple approach effectively solves the inherent problem of ghosting caused by crosstalk between multiple light fields in the static layered multi-display panel scheme, ensuring that the images of each depth layer are independent and clear, and the imaging quality is high.

[0020] (6) The present invention adopts a focal point-driven primary focal plane priority strategy: only the human eye's focal depth is set as the primary clear focal plane and allocated the highest image brightness and rendering precision. The other auxiliary focal planes only need to be moderately defocused for rendering. There is no need to perform full-resolution fine rendering on all depth layers. This greatly reduces the computing power requirement of multi-focal plane layered rendering of embedded GPU, reduces device power consumption and heat generation, and helps to extend the battery life of AR glasses and adapt to the energy consumption constraints of mobile wearable scenarios. Attached Figure Description

[0021] Figure 1 This is an organizational structure diagram of an adaptive multi-focal plane AR imaging system according to the present invention; Figure 2 This is a flowchart of an adaptive multifocal plane AR imaging method. Detailed Implementation

[0022] The technical solution of the present invention will be further described below with reference to the accompanying drawings and embodiments.

[0023] Unless otherwise defined, the technical or scientific terms used in this invention shall have the ordinary meaning as understood by one of ordinary skill in the art to which this invention pertains.

[0024] Example 1 like Figure 1 As shown, this invention provides an adaptive multi-focal plane AR imaging system with an end-to-end delay of no more than 8ms. It includes a micro-display module, a polarization-multiplexed dynamic phase-focusing optical module, and a waveguide-based perspective imaging module arranged sequentially along the imaging optical path. All three are fixed within the AR glasses frame by a precision optical bracket. It also includes a multi-source sensing and acquisition unit and a central driving and rendering control unit. The micro-display module uses independently configured LCoS microdisplays for the left and right eyes, with its light-emitting surface facing the light-incident surface of the polarization-multiplexed dynamic phase-focusing optical module. The two are optically coupled to ensure that all polarized image light output from the micro-display module enters the focusing module.

[0025] A polarization-multiplexed dynamic phase-focusing optical module is positioned between the light-emitting side of the micro-display module and the light-incident side of the waveguide perspective imaging module. The light-incident surface of the polarization-multiplexed dynamic phase-focusing optical module is aligned with the light-emitting surface of the micro-display module, and the light-emitting surface of the polarization-multiplexed dynamic phase-focusing optical module is coupled with the light-incident surface of the waveguide perspective imaging module. The waveguide perspective imaging module also has an ambient light transmission channel, which allows external ambient light and the beam modulated by the polarization-multiplexed dynamic phase-focusing optical module to be projected onto the human eye along the same path. The signal output terminal of the multi-source sensing and acquisition unit is connected to the signal input terminal of the central driving and rendering control unit, which is used to transmit at least one type of sensing data, including human eye state, environmental depth, and system attitude, to the central driving and rendering control unit.

[0026] The image signal output terminal of the central drive and rendering control unit is connected to the image signal input terminal of the micro-display module, and is used to output time-series layered image data to the micro-display module; the drive signal output terminal of the central drive and rendering control unit is connected to the drive signal input terminal of the polarization multiplexed dynamic phase focusing optical module, and is used to output focal plane switching drive signals to the polarization multiplexed dynamic phase focusing optical module.

[0027] The polarization multiplexing dynamic phase focusing optical module contains at least two electrically controlled adjustable phase modulation layers. Each layer independently receives driving signals to change the phase distribution, and after being superimposed, they form multiple adjustable virtual focal planes without mechanical displacement.

[0028] The polarization multiplexing dynamic phase focusing optical module specifically includes a multi-layer cascaded geometric phase liquid crystal metasurface, a chromatic aberration corrected freeform lens group, and a multi-channel analog voltage driving array; the multi-channel analog voltage driving array receives the driving signal output by the central driving and rendering control unit and outputs an independent driving voltage to each layer of liquid crystal metasurface.

[0029] In this embodiment, the multi-layer cascaded geometric phase liquid crystal metasurface is configured as three layers, which are arranged sequentially along the optical axis. Each layer independently receives a single DC driving voltage, with a driving voltage range of 0 to 15V, a voltage resolution of 0.01V, and a phase modulation response time of no more than 0.3ms. The radial phase distribution is controlled by changing the rotation angle of the liquid crystal molecules. After the phases of each layer are superimposed, multiple adjustable virtual focal planes without mechanical displacement are formed.

[0030] The optical waveguide perspective imaging module employs a diffraction grating waveguide structure, comprising an input coupling grating, a beam expander grating, and an exit pupil grating. Multi-layered focal plane polarized beams, modulated by the focusing module, are coupled into the optical waveguide through the input grating, expanded, and projected onto the viewer's eye through the exit pupil grating. External ambient light directly passes through the waveguide substrate into the viewer's eye, achieving perspective fusion of virtual multi-layered images and the real-world scene along the same optical path. A polarization beam splitting multiplexing layer is incorporated within the waveguide substrate, utilizing polarization states to isolate beams from different focal planes and suppress crosstalk and ghosting across multiple optical fields.

[0031] The collaborative working modes of multi-layer cascaded metasurfaces include single-focal-plane mode, multi-focal-plane parallel mode, and infinity-based real-world matching mode, with each mode corresponding to different combinations of interlayer driving voltages.

[0032] The central drive and rendering control unit synchronously controls the image output of the micro-display module and the focal plane depth switching of the polarization multiplexing dynamic phase focusing optical module based on the sensing data collected by the multi-source sensing acquisition unit, so that the depth of the virtual image matches the current sensing state.

[0033] The multi-source sensing and acquisition unit specifically includes an eye-tracking camera, a depth sensor, and an inertial measurement unit. The eye-tracking camera is positioned facing the human eye and acquires the coordinates of both pupils and corneal reflection points in real time at a sampling frame rate of 240Hz. The depth sensor is positioned facing the external scene and has an acquisition period of 8ms to obtain a 3D depth map of the external real scene. The inertial measurement unit is fixed on the housing of the imaging system and outputs the head posture angular displacement in real time to generate a parallax compensation matrix.

[0034] The data output terminals of the three sensors are electrically connected to the corresponding data input terminals of the perception data fusion processor in the central drive and rendering control unit via flexible circuit boards.

[0035] The central drive and rendering control unit specifically includes an image processor, a phase drive chip, and a timing synchronization controller. The image processor generates layered rendered images, the phase drive chip calculates the drive signal based on the target focal plane depth, and the timing synchronization controller coordinates the image refresh of the micro-display module and the focal plane switching timing of the polarization multiplexing dynamic phase focusing optical module.

[0036] The micro-display module is a spatial light modulator that supports time-sequential, sliced, and layered image output with a refresh rate of no less than 120Hz.

[0037] like Figure 2 As shown, an adaptive multi-focal plane AR imaging control method is applied to an adaptive multi-focal plane AR imaging system, comprising the following iterative execution steps: S1: Sensing data is acquired through a multi-source sensing acquisition unit. The sensing data specifically includes human eye state data, environmental depth data, and posture data. The binocular disparity is calculated to solve for the human eye accommodation depth. =0.55m, and after preprocessing the sensing data, it is transmitted to the central drive and rendering control unit; In this embodiment, a 3×3 neighborhood mean smoothing filter is used to remove depth noise, resulting in a current environmental depth distribution between 0.3m and 5.0m.

[0038] S2: Acquire a 3D virtual scene and extract the depth distribution data of virtual objects within that scene: distant panel 1.5m, mid-range model 0.55m, close-up button 0.28m. Adjust the depth based on human eye perspective, the depth distribution data of virtual objects, and the real-world depth map. Execute a hierarchical matching strategy, select the primary focal plane and at least one auxiliary focal plane, and output the target multi-focal plane set; The hierarchical matching strategy in S2 specifically includes: Main fixation focal plane matching: Selected to match the depth of human eye accommodation The virtual object depth with the smallest difference of 0.55m is used as the principal focal plane. =0.55m, allocate the highest image brightness of 100%, and prioritize the use of the second-layer phase modulation channel; select the virtual object depth with the smallest difference from the natural accommodation depth of the human eye to ensure that the virtual image depth is synchronized with the lens accommodation.

[0039] Auxiliary multi-layer focal plane selection: Select virtual objects such as the distant panel and the close-up button whose depth difference from the main focal plane is greater than a preset threshold ΔD=0.3m as the auxiliary focal plane set. ={1.5m,0.28m}, limiting the number of focal plane layers to 2; only the main focal plane is allocated the highest rendering precision, while the auxiliary focal planes are moderately defocused to reduce the rendering load.

[0040] Virtual-real depth constraint correction: If there is a real object at close range in the area of ​​gaze, the brightness of the virtual image at the same depth is automatically reduced to enhance the depth of field effect of virtual-real occlusion; real-scene depth constraint is introduced, and the brightness and blur of the virtual image at the same depth are automatically adjusted according to the real-scene depth of the gaze area so that the occlusion relationship is consistent with the real-scene height.

[0041] In this embodiment, the real-world depth value in the query real-world depth map is 0.32, corresponding to a book on a real desktop. This real-world depth (0.32m) is close to the focal plane (0.28m) of the near-field virtual button. The system automatically reduces the brightness of the virtual image at the same depth from 80% to 55% and increases its Gaussian blur kernel radius by 2 times, making the virtual button visually "yield" to the real book, thus enhancing the depth-of-field effect of virtual and real occlusion.

[0042] Output target multifocal plane set The depth of each focal plane target is transmitted to the phase drive chip.

[0043] S3: Perform depth slicing and layered rendering of the same 3D virtual scene according to the target multi-focal plane set, apply depth blur to each layer image, perform parallax compensation based on pose data, and output the layered images to the micro-display module; the specific steps of depth slicing and layered rendering of the 3D virtual scene in S3 include: S31: Based on the target focal plane set The 3D virtual scene is sliced ​​at depth, and each layer retains only the model and texture data of the corresponding depth range; Centered on the depth of each target focal plane, a half-bandwidth δ=0.05m is set. The far-field layer retains virtual panel data within the depth range of [1.45m, 1.55m], the mid-field layer retains virtual stereo model data within the range of [0.50m, 0.60m], and the near-field layer retains virtual button data within the range of [0.23m, 0.33m]. Data in other depth ranges are discarded or masked.

[0044] S32: Add optical depth blur to layered images, apply Gaussian defocus blur to pixels that are off-focus, and simulate the defocusing imaging characteristics of the human eye. Gaussian defocus blur is applied to pixels in each layer of the image that deviate from the focal plane depth of that layer. Specifically, for the mid-range focal plane (0.55m), the pixels of the distant panel deviate from 0.95m, and the blur kernel radius is set to 12 pixels; the pixels of the near-field button deviate from 0.27m, and the blur kernel radius is set to 5 pixels. The Gaussian kernel radius is positively correlated with the depth deviation distance, simulating the defocus imaging characteristics of the human eye.

[0045] S33: Perform parallax distortion compensation on each layer of images based on posture data, and apply a parallax distortion compensation matrix to each layer of images; eliminate the misalignment between virtual images and real-world scenes caused by head movements.

[0046] S34: Pack the multi-layer image frames processed by the above steps into a time sequence. Each frame contains 3 layers of depth slice images and is output to the micro-display module for time-division synchronous refresh at a refresh rate of 120Hz.

[0047] S4: Output the corresponding driving signal to the polarization multiplexing dynamic phase focusing optical module according to the target focal plane depth, switch the phase distribution of each adjustable phase modulation layer, so that the virtual focal plane formed after the superposition of each layer corresponds to the depth of each focal plane in the target multi-focal plane set. In S4, the driving signal is the driving voltage, and the mapping relationship between the target focal plane depth and the driving voltage of each layer is determined by the following formula: The equivalent refractive power of a single layer is calculated using the following formula: ; The depth of a single-layer imaging layer is calculated using the following formula: ; In the formula, For single-layer driving voltage, , , These are the fitting calibration coefficients for a single-layer metasurface; in this embodiment... =-0.2D, =0.45D / V, =-0.012D / V 2 ; L ( U )= ; The total refractive power of multi-layer cascaded lenses is calculated using the following formula: ; The total imaging depth of the multi-layer cascade is calculated using the following formula: ; in, This represents the total number of metasurface layers. This is the driving voltage for the nth layer. , , These are the independent calibration fitting coefficients for the nth metasurface layer; In this embodiment The value is 3. ( , , )= The driving voltage of each layer is calculated by inverting the above formula, so that the total imaging depth of the three layers matches the depths of the three targets. The calculated cooperative working combination of the three metasurface layers is as follows: =2V (distant layer) =5V (mid-range main focal layer). =8V (close-up layer).

[0048] Layer 1: D1 = -0.2 + 0.45 × 2 - 0.012 × 4 = 0.652D; Layer 2: D2 = -0.2 + 0.45 × 5 - 0.012 × 25 = 1.75D; Layer 3: D3 = -0.2 + 0.45 × 8 - 0.012 × 64 = 2.632D; Total refractive power: =0.652+1.75+2.632=5.034D; After the three layers are stacked, the system simultaneously generates three virtual image depths: L1=1 / 0.652≈1.53m, L2=1 / 1.75≈0.57m, L3=1 / 2.632≈0.38m, which are basically matched with the target F={1.5m,0.55m,0.28m} (error ≤5%, determined by the accuracy of nonlinear fitting).

[0049] The phase drive chip uses a linear ramp-gradient voltage output to achieve smooth focal plane switching. Taking the second layer switching from 0V to 5V as an example: U ( t )= + · t =5 t, 0≤ t ≤1ms; Where, Δ T This refers to the transition time for focal plane switching. and These are the switching start voltage and target voltage, respectively. The switching time is 1ms. The switching transition duration is 5V / ms to ensure no abrupt changes or flickering during the focal plane transition. A canceling phase is applied to non-currently active focal layers (such as unused layers in single-focal-plane mode) (drive voltage set to 0V, phase delay 0) to suppress multilayer optical field crosstalk.

[0050] S5: The layered image light output from the microdisplay module is modulated by the polarization-multiplexed dynamic phase-focusing optical module to form multi-layered beams corresponding to different virtual image distances. After being fused with the external ambient light by the optical waveguide perspective imaging module, it is projected onto the human eye; the radial distribution of the single-layer liquid crystal metasurface is calculated by the following formula: ; In the formula: radial coordinates Drive voltage The corresponding phase delay; The radial coordinates of the beam; This is the center wavelength of the microdisplay.

[0051] In this embodiment, the three-layer polarized image light output by the micro-display module (corresponding to the distant panel, mid-range model, and close-up button) enters the polarization multiplexing dynamic phase focusing optical module.

[0052] The three-layer liquid crystal metasurface generates a corresponding radial phase distribution based on the driving voltage (U1=2V, U2=5V, U3=8V): = ; = ; = ; The three beams are modulated and focused at different virtual image distances: approximately 1.53m for the distant panel, approximately 0.57m for the mid-range model, and approximately 0.38m for the close-up button. A freeform lens group for chromatic aberration correction synchronously corrects diffraction chromatic aberration and spherical aberration, compressing the total optical path length.

[0053] The modulated, three-layer focused beam is coupled into the optical waveguide through an input coupling grating. Inside the waveguide, the beam is uniformly expanded by a beam-expanding grating and then projected onto the eye through an exit pupil grating. A polarization-splitting multiplexing layer inside the waveguide uses polarization states to isolate beams from different focal layers: the distant layer is P-polarized, the mid-range layer is S-polarized, and the near-field layer is circularly polarized, suppressing crosstalk and ghosting across multiple optical fields.

[0054] Meanwhile, ambient light directly enters the human eye through the optical waveguide substrate, containing depth information such as 0.32m for the real book, 0.5m for the desktop, and 5.0m for the wall. Thus, the three layers of virtual images are naturally superimposed on the retina—the distant virtual panel is coplanar with the real wall, the mid-range virtual model floats in front of the desktop, and the near-view virtual button is slightly in front of the real book. Each depth layer is independent and clear, resulting in a natural and realistic fusion of virtual and real elements.

[0055] S6: After the next sampling period of 8ms, repeat steps S1 to S5 to update the target multi-focal surface set in real time based on the newly acquired sensing data to achieve continuous closed-loop adaptive control.

[0056] The sensory data fusion processing unit will adjust the newly acquired human eye depth. Compared with the depth of the principal focal plane in the target focal plane set of the previous frame (0.55m), the depth deviation is 0.23m, which exceeds the preset threshold of 0.1m, and the target focal plane set is immediately updated.

[0057] At this point, the real-world depth map shows that the user's gaze has shifted to the real book in the foreground. The system then re-executes S2: switching the main focal plane to 0.32m, adjusting the auxiliary focal planes to 1.5m for the distant view and 0.60m for the mid-range view, and re-executes S3 rendering and S4 drive voltage calculation (the three layers of voltage are redistributed as U1=2V for the distant view, U2=6.5V for the new main focal plane, and U3=8V for the foreground view).

[0058] When eye-tracking data detects rapid eye saccades (angular velocity 650° / s, exceeding the threshold of 500° / s), the system activates the focal plane prediction interpolation algorithm: based on the saccade direction and speed, it predicts the eye fixation depth of the next frame to be 0.45m, outputs the phase drive voltage to an intermediate transition value in advance, and then fine-tunes it to the precise value after the actual fixation is confirmed, further reducing the adjustment delay.

[0059] When the system detects that the human eye has been staring at the same depth (0.55m mid-range model) for a long time (4.2 seconds, exceeding the 3-second threshold), it automatically reduces the brightness of the adjacent auxiliary focal planes (1.5m for distant view and 0.28m for close view) from 60% to 52% to alleviate visual fatigue.

[0060] The system continuously executes S1 to S5 in a cycle of one frame, achieving adaptive multi-focal plane dynamic control without human intervention throughout the process.

[0061] It is worth noting that all contents not described in detail in this invention are existing technologies and are well known to those skilled in the art.

[0062] Therefore, the present invention adopts the above-mentioned adaptive multi-focal plane AR imaging system and method, which uses a multi-layer cascaded electrically controlled adjustable phase modulation layer as the core focusing element. The phase distribution of each phase modulation layer is adjusted by driving signal, and multiple discrete or continuous adjustable virtual focal planes are superimposed to generate multiple virtual focal planes. The entire process does not require motor drive or mechanical displacement of lens group.

[0063] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the technical solutions of the present invention, and these modifications or equivalent substitutions cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of the present invention.

Claims

1. An adaptive multi-focal-plane (AR) imaging system, comprising: It includes a micro-display module, a polarization multiplexing dynamic phase focusing optical module, and an optical waveguide perspective imaging module arranged sequentially along the imaging optical path; as well as a multi-source sensing and acquisition unit and a central driving and rendering control unit; The polarization-multiplexed dynamic phase-focusing optical module is positioned between the light-emitting side of the micro-display module and the light-incident side of the optical waveguide perspective imaging module; the signal output terminal of the multi-source sensing and acquisition unit is connected to the signal input terminal of the central drive and rendering control unit, the image signal output terminal of the central drive and rendering control unit is connected to the image signal input terminal of the micro-display module, and the drive signal output terminal of the central drive and rendering control unit is connected to the drive signal input terminal of the polarization-multiplexed dynamic phase-focusing optical module.

2. The adaptive multi-focal-plane AR imaging system according to claim 1, wherein, The light-incident surface of the polarization multiplexing dynamic phase-focusing optical module is connected to the light-outcident surface of the micro-display module, and the light-outcident surface of the polarization multiplexing dynamic phase-focusing optical module is coupled to the light-incident surface of the optical waveguide perspective imaging module. The polarization multiplexing dynamic phase focusing optical module contains at least two electrically controlled adjustable phase modulation layers. Each layer independently receives driving signals to change the phase distribution, and after being superimposed, they form multiple adjustable virtual focal planes without mechanical displacement.

3. The adaptive multi-focal-plane AR imaging system according to claim 1, wherein, The multi-source sensing and acquisition unit specifically includes an eye-tracking camera, a depth sensor, and an inertial measurement unit; the eye-tracking camera is positioned facing the human eye, the depth sensor is positioned facing the external scene, and the inertial measurement unit is fixed on the housing of the imaging system.

4. The adaptive multi-focal plane AR imaging system according to claim 1, characterized in that, The central drive and rendering control unit specifically includes an image processor, a phase drive chip, and a timing synchronization controller. The image processor generates layered rendered images, the phase drive chip calculates the drive signal based on the target focal plane depth, and the timing synchronization controller coordinates the image refresh of the micro-display module and the focal plane switching timing of the polarization multiplexing dynamic phase focusing optical module.

5. An adaptive multi-focal plane AR imaging system according to claim 1, characterized in that, The micro-display module is a spatial light modulator that supports time-sequential, sliced, and layered image output with a refresh rate of no less than 120Hz.

6. An adaptive multi-focal plane AR imaging control method, applied to an adaptive multi-focal plane AR imaging system according to any one of claims 1-5, characterized in that, Includes the following iterative execution steps: S1: The multi-source sensing acquisition unit collects sensing data, which specifically includes human eye state data, environmental depth data, and posture data. It calculates binocular disparity to solve for human eye accommodation depth, and transmits the sensing data to the central drive and rendering control unit after preprocessing. S2: Acquire a 3D virtual scene and extract the depth distribution data of virtual objects in the scene; Based on the human eye's depth adjustment, the depth distribution data of virtual objects, and the real-world depth map, a layered matching strategy is executed to select the main focal plane and at least one auxiliary focal plane, and output a set of target multi-focal planes. S3: Perform depth slicing and layered rendering of the same 3D virtual scene according to the target multi-focal plane set, apply depth blur to each layered image, perform parallax compensation according to the pose data, and output the layered images to the micro display module. S4: Output the corresponding driving signal to the polarization multiplexing dynamic phase focusing optical module according to the target focal plane depth, switch the phase distribution of each adjustable phase modulation layer, so that the virtual focal plane formed after the superposition of each layer corresponds to the depth of each focal plane in the target multi-focal plane set. S5: The layered image light output by the micro-display module is modulated by the polarization multiplexing dynamic phase focusing optical module to form a multi-layer beam corresponding to different virtual image distances. After being fused with the external ambient light by the optical waveguide perspective imaging module, it is projected to the human eye. S6: Repeat steps S1 to S5 in the next sampling cycle, and update the target multi-focal surface set in real time based on the newly acquired sensing data to achieve continuous closed-loop adaptive control.

7. The adaptive multi-focal plane AR imaging control method according to claim 6, characterized in that: The hierarchical matching strategy in S2 specifically includes: Primary focal plane matching: The virtual object depth with the smallest difference from the human eye's accommodation depth is selected as the primary focal plane. Allocate the highest image brightness and prioritize the use of the phase modulation channel; Auxiliary multi-layer focal plane selection: Select distant and near virtual objects whose depth difference from the main focal plane is greater than a preset threshold ΔD as the auxiliary focal plane set. Limit the number of focal plane layers to avoid multiple layers overlapping and creating ghosting; Virtual-real depth constraint correction: If there is a real object at close range in the area of ​​the gaze, the brightness of the virtual image at the same depth is automatically reduced to enhance the depth-of-field effect of virtual-real occlusion. Output target multifocal plane set The depth of each focal plane target is transmitted to the phase drive chip.

8. The adaptive multi-focal plane AR imaging control method according to claim 7, characterized in that: In S3, the specific steps for depth slicing and layering rendering of a 3D virtual scene include: S31: Based on the target focal plane set The 3D virtual scene is sliced ​​at depth, and each layer retains only the model and texture data of the corresponding depth range; S32: Add optical depth blur to layered images, apply Gaussian defocus blur to pixels that are off-focus, and simulate the defocusing imaging characteristics of the human eye. S33: Perform disparity distortion compensation on each layer of images based on pose data, and apply a disparity distortion compensation matrix to each layer of images; S34: Pack the multi-layer image frames processed by the above steps into time sequence and output them to the micro-display module for time-sharing synchronous refresh.

9. The adaptive multi-focal plane AR imaging control method according to claim 8, characterized in that: In S4, the driving signal is the driving voltage, and the mapping relationship between the target focal plane depth and the driving voltage of each layer is determined by the following formula: The equivalent refractive power of a single layer is calculated using the following formula: ; The depth of a single-layer imaging layer is calculated using the following formula: ; In the formula, For single-layer driving voltage, , , These are the fitting calibration coefficients for a single-layer metasurface; The total refractive power of multi-layer cascaded lenses is calculated using the following formula: ; The total imaging depth of the multi-layer cascade is calculated using the following formula: ; in, This represents the total number of metasurface layers. This is the driving voltage for the nth layer. , , These are the independent calibration fitting coefficients for the nth metasurface layer; Drive voltage during focal plane switching U ( t ) Change according to the linear slope pattern: U ( t )= + · t ,0≤ t ≤Δ T ; Where, Δ T This refers to the transition time for focal plane switching. and These are the switching start voltage and target voltage, respectively. This is for switching time.