Extended Reality Head-Mounted Display System Based on Micro LED Display

Abstract

The application relates to the technical field of display, and particularly discloses an extended reality head-mounted display system based on Micro LED display, which comprises an eye movement tracking device, an image rendering processing unit, a bionic pixel mapping driving unit, a Micro LED display panel and an event-driven control unit; the line-of-sight center position is acquired through eye movement tracking, non-uniform rendering and bionic pixel mapping are realized, full-resolution pixels are driven in the line-of-sight center area, the data amount is reduced by using a pixel merging algorithm in the peripheral area, and the refresh is triggered only in the picture change area in combination with an event-driven mechanism. The application significantly reduces the computing power and bandwidth demand, relieves visual fatigue and dizziness, simultaneously realizes dynamic power consumption optimization, and improves the immersive experience and equipment endurance.

Description

Technical Field

[0001] This invention belongs to the field of display technology, specifically relating to an extended reality head-mounted display system based on Micro LED display. Background Technology

[0002] With the continuous evolution of extended reality technology, head-mounted displays have become the core carrier for realizing virtual-real fusion interaction and immersive perception. As the mainstream direction of next-generation display technology, Micro LED, with its extremely high pixel density, ultra-high brightness, and excellent contrast characteristics, provides key support for improving the visual performance of extended reality systems. In the design of high-performance head-mounted displays, the system needs to present massive amounts of image data in real time on a tiny display panel. This not only requires the display medium to have extremely high physical properties, but also requires the underlying image processing and driving architecture to support high-fidelity visual output in complex scenes.

[0003] The performance of a pixel-driven architecture directly determines the system's power consumption, latency, and final visual presentation quality. To balance display quality and hardware load, non-uniform rendering and pixel mapping techniques based on biological visual characteristics have gradually become a research focus in the field. By combining eye tracking and retinal sampling models, the system attempts to optimize the display resource configuration of the surrounding field of view while ensuring image quality in the visual center area, thereby achieving real-time simulation and efficient response to complex dynamic environments under limited hardware resource constraints.

[0004] However, traditional display systems generally employ uniform pixel distribution and global refresh logic, causing all pixels across the entire screen to operate at a synchronous frequency, resulting in significant waste of computing power and strain on transmission bandwidth. In the application scenarios of Micro LED with ultra-high pixel density, the throughput load of full-resolution rendering on the graphics processor and data interface is approaching physical limits, easily inducing image tearing or system overheating. Simultaneously, existing driving solutions lack deep integration with the retinal topology mapping mechanism, failing to perform pulsed updates for the non-linear dynamic characteristics of the image. This leads to a discrepancy between visual feedback and human eye physiological habits, resulting in significant visual vergence-accommodation conflict and dizziness. Summary of the Invention

[0005] The purpose of this invention is to provide an extended reality head-mounted display system based on Micro LED display, which can effectively solve the problems of high-resolution computing power bottleneck, transmission bandwidth pressure, and dizziness caused by visual convergence-accommodation conflict in the above-mentioned background technology.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows: An extended reality head-mounted display system based on Micro LED display includes an eye-tracking device, an image rendering and processing unit, a bionic pixel mapping driving unit, a Micro LED display panel, and an event-driven control unit, wherein: The eye-tracking device is configured to acquire the gaze direction and gaze center position of the user's eyes in real time, and output the gaze center position information to the image rendering processing unit and the bionic pixel mapping driving unit. The image rendering processing unit is configured to perform non-uniform rendering on the virtual scene based on the received gaze center position information, generate non-uniform image data with high resolution in the gaze center area and low resolution in the surrounding area, and transmit the non-uniform image data to the bionic pixel mapping driving unit. The bionic pixel mapping driving unit is configured to receive the non-uniform image data and, based on the topological structure model of the fovea of ​​the human eye's retina, map the image data into a non-uniform Micro LED pixel driving signal. It drives the full-resolution pixel array in the area corresponding to the center of the field of vision, and merges multiple adjacent Micro LED pixels into a single logical large pixel for unified driving in the area corresponding to the peripheral field of vision through a pixel merging algorithm. The Micro LED display panel is configured to receive the non-uniform driving signal output by the bionic pixel mapping driving unit, and accordingly illuminate the Micro LED pixels in the corresponding area to form a non-uniform high dynamic visual image that conforms to the physiological characteristics of the human eye. The event-driven control unit is configured to monitor the brightness changes and motion states of each region in the non-uniform image data. It triggers the refresh operation of the Micro LED pixels in the corresponding region only when a brightness change or motion feature is detected in a local area of ​​the image. For static regions without change, the current pixel state is maintained, thereby simulating the pulse response mechanism of biological neurons.

[0007] Preferably, the bionic pixel mapping driving unit has a built-in foveal topological mapping model, which is constructed based on human eye physiological parameters and can dynamically adjust the coverage and pixel density gradient of the high-resolution region so that the high-resolution region is always precisely aligned with the user's real-time gaze center.

[0008] Furthermore, the pixel merging algorithm dynamically adjusts the merging ratio based on the centrifugal angle of the surrounding field of view. In areas farther from the center of vision, the number of merged Micro LED pixels increases, resulting in a larger logical pixel size. This significantly reduces data throughput while ensuring the continuity of peripheral visual perception.

[0009] Furthermore, the event-driven control unit determines whether there is a valid visual change by comparing the pixel brightness differences between consecutive frames. Only when the brightness change exceeds a preset threshold will the corresponding area be refreshed, thus avoiding redundant refresh of the static background.

[0010] Preferably, the image rendering processing unit and the bionic pixel mapping driving unit are connected by a low-latency data channel to ensure that the delay of the entire process from the center of vision to the remapping of the pixel driving signal is lower than the time threshold that the human eye can perceive, thereby maintaining the continuity of visual immersion.

[0011] Furthermore, the Micro LED display panel adopts a binocular independent driving architecture, with independent bionic pixel mapping driving units and event-driven control units configured for the left and right eyes respectively, in order to support differentiated non-uniform rendering and driving under stereoscopic vision.

[0012] Furthermore, the eye-tracking device uses a combination of an infrared light source and a high frame rate image sensor, which can stably capture the pupil position in low-light environments. It also uses a built-in calibration algorithm to compensate for the impact of head micro-movements on the accuracy of gaze positioning, ensuring that the accuracy of the gaze center position information is within a specific range.

[0013] Preferably, the event-driven control unit works in conjunction with the bionic pixel mapping drive unit to prioritize high refresh rate and high brightness output in the central viewing area, while further reducing power consumption in the surrounding area by combining the event-driven mechanism, thereby achieving dynamic power consumption optimization.

[0014] Compared with the prior art, the beneficial effects of the present invention are as follows: The extended reality head-mounted display system based on Micro LED display provided by this invention introduces the principle of neuromorphic vision and constructs a collaborative driving architecture that integrates eye tracking, non-uniform rendering and biomimetic pixel mapping, fundamentally changing the inherent mode of uniform pixel distribution and global refresh in traditional display systems.

[0015] This invention maintains full resolution and high brightness output in the central visual area, ensuring clear presentation of key visual information. In the peripheral visual field, it significantly reduces invalid data transmission and redundant refreshes through pixel merging and event-driven mechanisms, significantly alleviating the computational burden on the graphics processor and the bandwidth pressure on high-speed interfaces. At the same time, this non-uniform driving method is highly compatible with the physiological characteristics of the human retina, effectively reducing visual fatigue and dizziness caused by the mismatch between visual convergence and accommodation.

[0016] This invention's event-driven mechanism simulates the sparse response characteristics of biological neurons, triggering refreshes only in areas of screen change, further reducing overall system power consumption and extending device battery life. In summary, this invention achieves highly efficient synergistic optimization of computing resources, transmission bandwidth, and energy consumption while ensuring an immersive visual experience, providing a novel system-level solution for high-performance extended reality head-mounted devices. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of the overall technical solution architecture of an extended reality head-mounted display system based on Micro LED display according to an embodiment of the present invention; Figure 2 This is a schematic diagram of the core principle framework of the bionic pixel mapping drive based on the fovea topology in the extended reality head-mounted display system based on Micro LED display according to an embodiment of the present invention. Figure 3 This is a flowchart illustrating the main stages of non-uniform image rendering and peripheral field pixel merging processing in an extended reality head-mounted display system based on Micro LED display according to an embodiment of the present invention. Figure 4 This is a flowchart illustrating the event-driven refresh control based on brightness changes and motion characteristics in an extended reality head-mounted display system based on Micro LED display according to an embodiment of the present invention. Figure 5 This is a schematic diagram illustrating the multi-level interaction and data flow between eye-tracking, image rendering, and pixel-driving units in an extended reality head-mounted display system based on Micro LED display, according to an embodiment of the present invention. Detailed Implementation

[0018] Example 1: To make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to specific embodiments.

[0019] Please refer to Figures 1 to 5 This embodiment provides an extended reality head-mounted display system based on Micro LED display, including an eye-tracking device, an image rendering processing unit, a bionic pixel mapping driving unit, a Micro LED display panel, and an event-driven control unit.

[0020] The eye-tracking device is equipped with a high-frame-rate infrared photosensitive module and a dedicated vision processor, configured to capture the physical movement of the user's eyes in real time. The device emits infrared structured light of a specific wavelength onto the surface of the user's eyeball and captures the Purkinje dot image formed by the reflection using an infrared camera. Then, it calculates the user's gaze direction in real time using a built-in pupil center corneal reflection algorithm. The output of the eye-tracking device is connected to the input of the image rendering processing unit and the bionic pixel mapping driving unit via a low-latency serial bus, used to synchronize the calculated gaze center position information, i.e., the gaze point coordinates, to the subsequent processing modules in real time at a sampling frequency of no less than 240Hz. The eye-tracking device also includes an ambient light compensation submodule, configured to monitor ambient light intensity and dynamically adjust the emission power of the infrared light source to ensure high-contrast pupil images are acquired under different illumination conditions, thereby ensuring the spatial accuracy of the gaze center position information is better than 0.5 degrees.

[0021] The image rendering processing unit is configured to receive gaze center position information transmitted by the eye-tracking device and execute non-uniform gaze-point rendering logic based on this position information. The image rendering processing unit integrates a high-performance graphics processing chip and a large-capacity video memory. Its core function is to divide the virtual scene to be rendered into hierarchical regions with different rendering precisions based on the visual acuity distribution characteristics of the human retina. Within a preset radius region centered on the gaze center (i.e., the concave rendering region), the image rendering processing unit performs full rendering with the highest geometric precision, lighting model complexity, and texture resolution to generate ultra-high-definition image frames conforming to the physical pixel density of Micro LEDs. In the peripheral region far from the gaze center (i.e., the edge field of view), the image rendering processing unit performs downsampling rendering by reducing the number of polygons, simplifying shader calculations, or using low-order interpolation sampling. Through this non-uniform rendering mechanism, the image rendering processing unit significantly reduces the floating-point operation load and data throughput of the graphics processing chip while ensuring the image quality in the visual center region.

[0022] The bionic pixel mapping driving unit, as the logical core of the system, is configured to receive non-uniform image data generated by the image rendering processing unit and convert this data into specific driving signals required by the Micro LED display panel based on the topological model of the fovea of ​​the human retina. The bionic pixel mapping driving unit integrates a pixel remapping logic circuit, which is configured to dynamically adjust the driving mapping relationship of pixels according to the position information of the viewing center. Specifically, in the high-resolution region corresponding to the viewing center, the bionic pixel mapping driving unit is configured to independently address and drive each physical Micro LED pixel to achieve full-resolution visual presentation. In the region corresponding to the peripheral field of view, the pixel merging submodule within this unit is configured to apply a pixel merging algorithm to merge multiple spatially adjacent Micro LED physical pixels into a single logical driving unit. For example, in the edge field of view region, a 4x4 physical pixel array is mapped into a single large logical pixel by current addition or voltage averaging, thereby significantly reducing the number of effective driving commands that need to be transmitted and processed. The bionic pixel mapping driving unit is also configured to dynamically and smoothly migrate the boundary between the high-resolution driving area and the pixel merging driving area according to the real-time offset of the viewing position, so as to avoid obvious visual jumps.

[0023] The Micro LED display panel is configured to physically present the final visual image. The Micro LED display panel consists of an array of millions of micron-sized light-emitting diode chips integrated on a substrate with a CMOS driving backplane. Each pixel of the Micro LED display panel is directly controlled by the driving current output by the bionic pixel mapping driving unit. Due to the use of Micro LED technology, the panel possesses nanosecond-level response speed and extremely high dynamic range. In operation, the physical pixel distribution of the Micro LED display panel is uniform, but its visual brightness distribution and refresh characteristics exhibit non-uniformity, which completely follows the driving distribution generated by the bionic pixel mapping driving unit.

[0024] The event-driven control unit is configured to implement a change-based pulse refresh strategy by real-time monitoring of content changes between consecutive video frames. The event-driven control unit uses a high-speed differential comparator array to compare the current frame image data output by the image rendering processing unit with the previous frame image data stored in the frame buffer point-by-point or block-by-block. When the pixel brightness change or color motion vector of a local area exceeds a preset sensitivity threshold, the event-driven control unit generates a refresh enable signal, allowing the bionic pixel mapping drive unit to update the pixel state of that area. For areas where the image content remains static or changes minimally, the event-driven control unit outputs a hold signal, ensuring that the corresponding Micro LED pixels maintain their current charge state without redundant physical refresh. This mechanism simulates the impulse response characteristics of biological visual neurons, i.e., responding only to variables in the environment, thereby significantly reducing the dynamic power consumption and thermal power consumption of the display system while maintaining visual coherence.

[0025] The biomimetic pixel mapping driving unit is further detailed as follows: It internally incorporates a highly biomimetic foveal topological mapping model. This model is not merely a simple region division logic, but a nonlinear mapping function constructed based on the spatial distribution function of cone and rod cells in the human retina. The model is configured to calculate a continuously varying pixel density gradient based on human eye physiological parameters, such as the location of the optic disc and the diameter of the macula. At the center of the line of sight, this density gradient is at its maximum value, and it decays exponentially with increasing eccentricity. The biomimetic pixel mapping driving unit calculates the pixel merging ratio at various coordinate points in the peripheral field of vision in real time based on this gradient function. In the region within 10 degrees of the center of the line of sight, the merging ratio is set to 1:1; in the moderate peripheral region between 10 and 30 degrees, the merging ratio is automatically adjusted to 2x2 pixel merging; and in the edge region above 30 degrees, the merging ratio is expanded to 4x4 or even 8x8. This dynamic adjustment mechanism ensures that while satisfying the low spatial resolution characteristics of the human eye's peripheral field of vision, maximum bandwidth compression is achieved.

[0026] The pixel merging algorithm is implemented in the bionic pixel mapping driving unit as follows: The algorithm includes a spatial downsampling step and a signal normalization step. In the spatial downsampling step, the algorithm extracts a subset corresponding to the physical pixel array from the original image data; in the signal normalization step, the algorithm calculates a weighted average of the gray values ​​of all pixels within a predetermined merging region, and the generated average gray value serves as the unified driving command for all Micro LED physical pixels within that region. Furthermore, to prevent jagged edges from appearing at the merged edges, the pixel merging algorithm also introduces a bilinear filtering mechanism, performing transition processing at the boundaries of different merging ratios to ensure visual smoothness.

[0027] The comparison logic of the event-driven control unit is described as follows: This unit is configured to perform macroblocking processing on image frames, with each macroblock containing a predetermined number of pixels. For each macroblock, an accumulator within the unit calculates its total brightness difference in the temporal domain. When the total brightness difference is greater than a preset visual perception threshold, the macroblock is marked as an active region; otherwise, it is marked as a static region. The event-driven control unit only allocates refresh clock signals to active regions. Furthermore, the event-driven control unit also has an adaptive threshold adjustment function, which can dynamically raise or lower the preset visual perception threshold according to the ambient light intensity or the intensity of motion in the virtual scene, in order to balance the relationship between power consumption optimization and display accuracy.

[0028] A dedicated low-latency data channel is used between the image rendering processing unit and the bionic pixel mapping driving unit. This channel is based on the LVDS low-voltage differential signaling standard and employs a simplified transmission protocol designed to eliminate redundant encapsulation overhead in general interface protocols. The low-latency data channel is configured to support peak bandwidths up to 100 Gb per second, ensuring that the total system latency from the eye-tracking device updating the gaze position to the Micro LED display panel completing the corresponding non-uniform drive signal adjustment is less than 8 milliseconds. This value is far lower than the visual persistence time perceptible to the human eye, effectively eliminating visual artifacts caused by lag in the gaze point area.

[0029] The Micro LED display panel employs a binocular independent driving architecture in its physical structure. In this embodiment, the left-eye display module and the right-eye display module are each equipped with an independent bionic pixel mapping driving unit and an event-driven control unit. This architecture is configured to support differentiated processing under stereoscopic vision. For example, when a user's one eye is focused on a nearby object while the other eye is focused on a slightly distant background, the system can generate non-uniformly rendered images and driving signals with different offsets for the left and right eyes respectively. This provides support at the underlying hardware level to alleviate visual convergence-accommodation conflict, reducing the dizziness symptoms commonly seen in extended reality devices from a physical perspective.

[0030] The infrared light source and high frame rate image sensor of the eye-tracking device are configured to operate in asynchronous triggering mode. The high frame rate image sensor performs exposure sampling at a frequency of 500Hz, while the infrared light source flashes at a matching pulse frequency to reduce average power consumption. A built-in calibration algorithm submodule is configured to compensate for geometric deviations caused by minor slippage of the head-mounted device on the face or violent head movements by analyzing the relative displacement of the pupil center relative to the corneal reflection point in real time. This algorithm uses a Kalman filter model to predict the gaze trajectory, ensuring that the delay compensation value for the gaze center position information is within a specific 5 milliseconds during rapid eye saccades.

[0031] The collaborative working logic between the event-driven control unit and the bionic pixel mapping drive unit is as follows: Upon system startup, the event-driven control unit performs a global refresh of all screen pixels. Subsequently, it enters incremental refresh mode. In this mode, upon receiving new image frame data, the bionic pixel mapping drive unit first queries the active region mask generated by the event-driven control unit. Only when the pixel coordinates are within the range defined by the active region mask will the pixel's drive current controller update according to the new data. In the center of the field of view, the event-driven control unit is configured to forcibly enable full-time refresh mode, ensuring that this area operates at the highest refresh rate regardless of changes in the image, thus guaranteeing the highest temporal resolution and brightness response speed at the focal point. In the peripheral field of view, the system relies entirely on the event triggering mechanism, automatically entering an ultra-low power standby state when the image is static.

[0032] This collaborative working mechanism also includes a brightness adaptive adjustment sub-function. Since Micro LED pixels may suffer from uneven brightness under non-uniform driving, the bionic pixel mapping driving unit is configured to perform current compensation for logical large pixels at different merging ratios using a brightness correction lookup table. When multiple pixels are merged and driven, by appropriately reducing the driving current of individual physical pixels, the total luminous intensity of the logical large pixels is made consistent with that of the surrounding non-merged physical pixels at the perception level, thereby eliminating the blocky brightness differences caused by pixel merging.

[0033] Example 2: Building upon Example 1, this example provides an extended reality head-mounted display system based on a Micro LED display and a distributed computing architecture. In this example, the image rendering processing unit is divided into a local preprocessing module and a remote rendering server module, which interact with each other via a high-speed wireless communication link.

[0034] The remote rendering server module is equipped with a large-scale graphics processing array and is configured to perform complex virtual environment construction and ray tracing calculations. The remote rendering server module receives the pre-encoded view center position information uploaded by the local preprocessing module and generates a high-precision raw rendering data stream accordingly.

[0035] The local preprocessing module is integrated into the headband structure of the head-mounted display device and is configured to decode and perform secondary frame interpolation on the data returned by the remote rendering server module. This distributed architecture allows the system to significantly reduce the size and weight of the head-mounted device while maintaining extremely high rendering quality.

[0036] In this embodiment, the bionic pixel mapping driving unit incorporates prediction compensation logic. Due to potential jitter in wireless transmission, the bionic pixel mapping driving unit is configured as a time-warping processing engine. This engine can perform spatial translation and affine transformation on the previously received rendering frame based on the latest gaze angular velocity data provided by the eye-tracking device. This allows for the pre-generation of an approximate image conforming to the current gaze direction before the arrival of a new rendering frame, ensuring the immediacy of the visual display.

[0037] In this embodiment, the Micro LED display panel employs a flexible substrate design to accommodate a wider, more immersive viewing experience. The biomimetic pixel mapping driving unit incorporates a spatial geometry correction module to address the curvature of the flexible panel. This module is configured to automatically introduce a coordinate compensation factor during pixel remapping to counteract image geometric distortion caused by screen curvature, ensuring that the foveal projection area observed by the user from different angles maintains accurate physical proportions.

[0038] In this embodiment, the event-driven control unit is further optimized by adding a visual saliency analysis function. In addition to monitoring brightness changes, this unit is also equipped with a lightweight deep convolutional neural network for identifying salient features in the image, such as text, faces, or fast-moving targets. For identified salient feature regions, even if the brightness change is below a preset threshold, the event-driven control unit is configured to increase its refresh priority and allocate higher bit depth bandwidth to ensure the visual readability of key information.

[0039] In this embodiment, the pixel merging algorithm of the biomimetic pixel mapping driving unit introduces a random sampling dithering mechanism. In the edge field of view, to further eliminate fixed pattern noise caused by pixel merging, the algorithm introduces a small random spatial displacement when selecting physical pixels to participate in the merging. This approach makes the center points of logically large pixels exhibit a weak pseudo-random distribution on the physical array, thereby masking the visual graininess caused by low resolution by utilizing the insensitivity of the human eye's peripheral field of view to high-frequency noise.

[0040] In this embodiment, the eye-tracking device integrates multiple infrared sensors to extract multi-angle, multi-feature point features of the eye. These include not only the pupil center but also the iris edge and the inner canthus. By constructing a complete three-dimensional eye model, the eye-tracking device can output a three-dimensional rotation vector of the eyeball, not just planar coordinates. The image rendering unit uses this three-dimensional rotation vector for perspective correction rendering, making the generated non-uniform image more consistent with physical perspective laws during viewpoint changes, further reducing dizziness.

[0041] In this embodiment, the event-driven control unit is also configured to work in conjunction with the battery management module. When the system battery level is detected to be lower than a preset alarm threshold, the event-driven control unit automatically enters a power-saving mode. In power-saving mode, the system actively expands the coverage area of ​​pixel merging and increases the brightness change threshold triggered by events, thereby maximizing the device's battery life by sacrificing the visual accuracy of some non-critical areas.

[0042] Example 3: This example describes an extended reality head-mounted display system based on Micro LED display and heterogeneous integration of system-on-a-chip, which aims to further improve the system's integration and response efficiency.

[0043] In this embodiment, the image rendering processing unit, the bionic pixel mapping driving unit, and the event-driven control unit are physically integrated on the same VLSI chip, forming a dedicated extended reality visual processing unit. This processing unit achieves electrical connection with the CMOS driving substrate of the Micro LED display panel through direct bonding technology, eliminating electromagnetic interference and transmission delays caused by external cables.

[0044] In this embodiment, the biomimetic pixel mapping driving unit employs a multi-scale pyramid driving structure. This unit is configured to generate driving signal streams at three different resolution levels in parallel: a full-resolution signal stream at the bottom layer, a 1 / 4 resolution signal stream in the middle layer, and a 1 / 16 resolution signal stream at the top layer. The dynamic switching logic within the unit dynamically moves the injection area of ​​the full-resolution signal stream on the Micro LED display panel, like a searchlight, based on real-time coordinates provided by the eye-tracking device. This multi-scale parallel architecture eliminates the logic computation latency during the dynamic remapping process, allowing the movement speed of the high-resolution area to perfectly keep up with the rapid scanning of the eye.

[0045] In this embodiment, the event-driven control unit is deeply integrated into the driving pixel circuit of the Micro LED. Each Micro LED physical pixel unit is equipped with a miniature sample-and-hold circuit and a zero-power comparator circuit. The comparator circuit is configured to monitor the voltage level changes on the current pixel driving line in real time. Only when the voltage difference on the driving line exceeds an internal threshold is the external driving current allowed to pass through the pixel. This pixel-level hardware event-driven mechanism refines power consumption control to an extreme degree, achieving true on-demand illumination and on-demand refresh.

[0046] In this embodiment, the image rendering processing unit specifically performs nonlinear pre-correction of color gamut and gamma value for the physical light emission characteristics of Micro LED. Due to the wavelength shift characteristics of Micro LED under different driving currents, the processing unit is configured to consult the color map table in real time when generating non-uniform image data, and adaptively compensate for pixel values ​​in different brightness areas to ensure a high degree of color consistency between the central bright area and the edge low brightness area rendered at the foveation point.

[0047] In this embodiment, the eye-tracking device incorporates a gaze prediction algorithm based on machine learning. The neural network accelerator integrated within the processing unit is configured to analyze the user's eye movement trajectory habits. Within 10 milliseconds before a rapid eye saccade occurs, the algorithm can predict the next possible gaze point. Based on this prediction, the bionic pixel mapping driving unit preheats the Micro LED pixel driving circuit in the target area and begins loading high-resolution rendering data. This feedforward control mechanism effectively offsets the inherent physical latency of the hardware link, achieving zero-latency gaze response at the user's perception level.

[0048] In this embodiment, the Micro LED display panel employs zoned backlight enhancement technology. The panel is divided into multiple independent power domains, each corresponding to a visual rendering level. The event-driven control unit is configured to dynamically turn off or reduce the voltage of the power domain containing completely static areas based on the activity level of the current screen. This dual-layer energy-saving strategy, combining global power domain control with local pixel-level event driving, reduces the system's average power consumption during dynamic video playback by more than 70% compared to traditional display solutions.

[0049] The biomimetic pixel mapping driving unit also includes a visual comfort assurance submodule. This submodule is configured to monitor the frequency of the user's gaze switching between different depth planes. When it detects that the user's gaze is frequently and rapidly switching between the near and far views of the virtual scene, the submodule instructs the image rendering processing unit to add a certain degree of depth blur effect on top of the non-uniform rendering, simulating the natural depth perception of the human eye, thereby effectively alleviating eye fatigue caused by the conflict between visual convergence and accommodation.

[0050] Furthermore, the bionic pixel mapping driving unit in this embodiment also supports local tone mapping for high dynamic range images. In the high-resolution region at the center of the field of view, the unit is configured to apply a high-bit tone mapping algorithm to preserve the richest highlight and shadow details; in the surrounding regions, a low-bit truncated mapping is used. This approach maintains the core visual impact while further compressing the data transmission bit width of the pixel driving circuit.

[0051] In a further implementation, the system also includes a wearing status monitoring unit. This unit is configured to acquire, in real time, the tightness of the device and its positional offset relative to the face using proximity and pressure sensors integrated within the headset. The bionic pixel mapping driving unit is configured to fine-tune the center origin of the non-uniform mapping in real time based on this offset data, ensuring that even with slight device displacement, the virtual foveal projection area remains precisely aligned with the user's physical pupil center.

[0052] In summary, this embodiment, through deep integration and mechanism innovation at the chip level, constructs a highly biomimetic, extremely low-power extended reality display driving architecture with predictive response capabilities. It solves the computing power and bandwidth bottleneck problem caused by the ultra-high pixel density of Micro LED, providing users with an unprecedented visual experience that balances immersion and comfort.

Claims

1. An extended reality head-mounted display system based on Micro LED display, characterized in that: It includes an eye-tracking device, an image rendering and processing unit, a bionic pixel mapping driving unit, a Micro LED display panel, and an event-driven control unit. The eye-tracking device is configured to acquire the user's gaze direction and gaze center position information in real time, and output the gaze center position information to the image rendering and processing unit and the bionic pixel mapping driving unit. The image rendering processing unit is configured to perform non-uniform rendering on the virtual scene based on the received gaze center position information, generate non-uniform image data with high resolution in the gaze center area and low resolution in the surrounding area, and transmit the non-uniform image data to the bionic pixel mapping driving unit. The bionic pixel mapping driving unit is configured to receive the non-uniform image data and, based on the topological structure model of the fovea of ​​the human eye's retina, map the non-uniform image data into a non-uniform Micro LED pixel driving signal. It drives the full-resolution pixel array in the area corresponding to the center of the field of vision, and merges multiple adjacent Micro LED physical pixels into a single logical large pixel for unified driving in the area corresponding to the peripheral field of vision through a pixel merging algorithm. The Micro LED display panel is configured to receive the non-uniform driving signal output by the bionic pixel mapping driving unit, and accordingly illuminate the Micro LED pixels in the corresponding area. The event-driven control unit is configured to monitor the change status of each region in the non-uniform image data, and trigger the refresh operation of the Micro LED pixels in the corresponding region when a brightness change or motion feature is detected in a local area of ​​the image.

2. The extended reality head-mounted display system based on Micro LED display according to claim 1, characterized in that, The eye-tracking device includes: An infrared light source submodule is configured to emit infrared structured light of a specific wavelength onto the surface of the user's eyeball. The infrared light source submodule adopts a pulse modulation mode and reduces average power consumption by controlling the pulse frequency. A high frame rate image sensor is configured to capture the Purkinje spot image formed by the reflection of the infrared structured light on the surface of the eyeball and to capture the physical motion state of the pupil in real time using an infrared photosensitive module. A dedicated vision processor is connected to the high frame rate image sensor and configured to execute a pupil center corneal reflection algorithm. By analyzing the relative displacement of the pupil center with respect to the corneal reflection point, the coordinates of the user's gaze point are calculated in real time. An ambient light compensation submodule is configured to monitor the intensity of external ambient light and dynamically adjust the emission power of the infrared light source submodule according to the ambient light intensity, so as to ensure that the contrast of the pupil image acquired under different illuminance environments is higher than a preset image processing threshold. The calibration algorithm submodule has a built-in Kalman filter model, which is configured to predict the gaze trajectory and compensate for the geometric deviation caused by the displacement of the head-mounted device on the face by analyzing the impact of the user's head micro-movements on the gaze positioning accuracy, so as to ensure that the spatial accuracy of the gaze center position information is better than 0.5 degrees.

3. The extended reality head-mounted display system based on Micro LED display according to claim 1, characterized in that, The image rendering processing unit includes: The graphics processing chip and large-capacity video memory are configured to execute non-uniform foveated rendering logic, dividing the virtual scene to be rendered into hierarchical regions with different rendering precision. The central concave rendering module is configured to perform full rendering within a preset radius area centered on the center of the line of sight, using preset maximum geometric precision, maximum lighting model complexity, and maximum texture resolution, to generate ultra-high-definition image frames that conform to the physical pixel density of MicroLED. The edge field of view rendering module is configured to perform downsampling rendering in the peripheral area far from the center of the view, by reducing the number of polygons, simplifying the shader calculation logic, or using a low-order interpolation sampling method, so as to reduce the floating-point operation load of the graphics processing chip. The image rendering processing unit is configured to dynamically adjust the range of the hierarchical region according to the visual acuity distribution characteristics of the human eye's retina, and to ensure that the image data generated by the foveal rendering module is aligned with the center of vision in real time.

4. The extended reality head-mounted display system based on Micro LED display according to claim 1, characterized in that, The biomimetic pixel mapping driving unit includes: A pixel remapping logic circuit is configured to receive the gaze center position information and dynamically adjust the driving mapping relationship of pixels according to the real-time offset of the gaze center position information. A foveal topological mapping model is built into the pixel remapping logic circuit. The foveal topological mapping model is based on the spatial distribution function of cone cells and rod cells in the human eye retina to construct a nonlinear mapping function, and is configured to calculate the continuously changing pixel density gradient. The pixel remapping logic circuit is configured to: maximize the pixel density gradient at the center of the line of sight, and decrease the pixel density gradient exponentially as the centrifugal angle increases, thereby calculating the pixel merging ratio at each coordinate point in the surrounding field of view in real time. The bionic pixel mapping driving unit is also equipped with a boundary smoothing migration module, which is used to dynamically migrate the physical boundary between the high-resolution driving area and the pixel merging driving area according to the real-time offset of the viewing position, so as to eliminate the visual jump caused by the switching of driving modes.

5. The extended reality head-mounted display system based on Micro LED display according to claim 1, characterized in that, The execution logic of the pixel merging algorithm integrated in the bionic pixel mapping driving unit includes: Spatial downsampling logic, configured to extract a subset of data corresponding to the Micro LED physical pixel array from the original non-uniform image data; The signal normalization logic is configured to perform a weighted average calculation on the gray values ​​of all physical pixels within a predetermined merging area, and use the generated average gray value as a unified driving instruction for all Micro LED physical pixels within the predetermined merging area. Bilinear filtering logic is configured to perform image filtering at the boundaries of different pixel merging ratios, and generates smooth transition pixel values ​​through interpolation algorithms to eliminate the jaggedness caused by merging edges. The random sampling jitter logic is configured to introduce random spatial displacement within a preset range when selecting physical pixels to participate in merging in the edge field of view. This makes the center point of the logical large pixel present a pseudo-random distribution on the physical array, thereby masking the visual graininess caused by low resolution.

6. The extended reality head-mounted display system based on Micro LED display according to claim 1, characterized in that, The event-driven control unit includes: A high-speed differential comparator array is configured to perform point-by-point or block-by-block comparisons between the current frame image data output by the image rendering processing unit and the previous frame image data stored in the frame buffer. The macroblocking module is configured to divide an image frame into multiple macroblocks containing a predetermined number of pixels, and to use an accumulator to calculate the total brightness difference of each macroblock in the temporal domain; the refresh enable control module is configured to mark the corresponding macroblock as an active region and generate a refresh enable signal when the total brightness difference is greater than a preset visual perception threshold, allowing the bionic pixel mapping driving unit to update the pixel state of the region. The static region holding module is configured to mark the corresponding macroblock as a static region and output a holding signal when the total brightness difference is not greater than a preset visual perception threshold, so that the corresponding Micro LED physical pixel maintains the current charge state. The event-driven control unit also has an adaptive threshold adjustment function, configured to dynamically increase or decrease the preset visual perception threshold according to the ambient light intensity or the intensity of motion in the virtual scene.

7. The extended reality head-mounted display system based on Micro LED display according to claim 1, characterized in that: The image rendering processing unit and the bionic pixel mapping driving unit are connected through a dedicated low-latency data channel. The low-latency data channel is based on the low-voltage differential signal standard and is configured with a simplified transmission protocol to eliminate redundant encapsulation overhead. The Micro LED display panel adopts a binocular independent driving architecture. Its physical structure includes a left-eye display module and a right-eye display module that are independent of each other. The left-eye display module and the right-eye display module are each equipped with an independent set of the bionic pixel mapping driving unit and the event driving control unit. The binocular independent driving architecture is configured to support differentiated non-uniform rendering under stereo vision. Based on the different depth planes viewed by the user's left and right eyes, non-uniform driving signals with different offsets are generated for the left and right eyes respectively.

8. The extended reality head-mounted display system based on Micro LED display according to claim 1, characterized in that, The system also includes a distributed computing architecture, wherein: The image rendering processing unit includes a local preprocessing module and a remote rendering server module, and the local preprocessing module and the remote rendering server module achieve data interaction through a high-speed wireless communication link; The remote rendering server module is configured with a graphics processing array to receive the encoded view center position information uploaded by the local preprocessing module and generate a high-precision raw rendering data stream accordingly. The local preprocessing module is configured to decode and perform secondary frame interpolation on the original rendering data stream. The bionic pixel mapping driving unit is also equipped with a time warp processing engine, which is configured to perform spatial translation and affine transformation on the previously received rendering frame based on the latest gaze angular velocity data provided by the eye tracking device, and generate an approximate image that conforms to the current gaze direction before the new rendering frame arrives.

9. The extended reality head-mounted display system based on Micro LED display according to claim 1, characterized in that: The image rendering processing unit, the bionic pixel mapping driving unit, and the event-driven control unit are physically integrated on the same ultra-large-scale integrated circuit chip to form a visual processing unit. The visual processing unit is electrically connected to the driving substrate of the Micro LED display panel through direct bonding technology. The biomimetic pixel mapping driving unit adopts a multi-scale pyramid driving structure and is configured to generate multiple driving signal streams of different resolution levels in parallel, including full-resolution signal stream, 1 / 4 resolution signal stream and 1 / 16 resolution signal stream. The dynamic switching logic configuration inside the bionic pixel mapping driving unit is used to dynamically move the injection area of ​​the full-resolution signal stream on the Micro LED display panel according to the real-time acquired gaze center position information. The event-driven control unit is integrated into the driving pixel circuit of the Micro LED display panel. Each Micro LED physical pixel unit is equipped with a miniature sample-and-hold circuit and a zero-power comparator circuit. The zero-power comparator circuit is configured to monitor the level changes on the pixel driving line and open the current path when the voltage difference exceeds an internal threshold.

10. The extended reality head-mounted display system based on Micro LED display according to claim 1, characterized in that, The system also includes: The visual comfort assurance submodule is configured to monitor the switching frequency of the user's gaze between different depth planes. When the switching frequency exceeds a preset frequency threshold, the image rendering processing unit is instructed to add a depth blur effect on the basis of non-uniform rendering to simulate the natural depth perception of the human eye and alleviate the visual convergence-accommodation conflict. The brightness adaptive adjustment module is configured to perform current compensation for logical large pixels under different pixel merging ratios through a brightness correction lookup table. When multiple physical pixels are merged and driven, the total luminous intensity of the logical large pixels is kept consistent with that of the physical pixels in the non-merging area at the perception level by reducing the driving current of a single physical pixel. The wearing status monitoring unit includes a proximity sensor and a pressure sensor integrated within the system, configured to acquire information on the tightness of the device and its positional offset relative to the face; the bionic pixel mapping driving unit is also configured to finely adjust the center origin of the non-uniform mapping in real time according to the positional offset information, ensuring that the virtual foveal projection area of ​​the retina remains aligned with the center of the user's physical pupil. The event-driven control unit is also configured to work in conjunction with the battery management module to reduce system power consumption when the system power is below a preset alarm threshold by expanding the coverage area of ​​pixel merging and increasing the brightness change threshold triggered by the event.

Images