Adaptive display buffer system, method and medium for enhancing visual comfort

Abstract

The present invention provides an adaptive display buffering system, method, and medium for enhancing visual comfort. The system obtains raw display data from an off-screen display buffer; generates a coding vector based on the raw display data and performs a content feature score based on the coding vector; modifies the content features of the raw display data based on the content feature score and the user's gaze direction, wherein a content modifier applies different modifications to the display in the foveal area and the peripheral area; and stores the modified display data in the on-screen display buffer. This invention can enhance visual comfort, implement intelligent rendering technology, and alleviate vision-related health issues.

Description

Technical Field

[0001] The present invention relates to the field of display technology, and in particular to an adaptive display buffer system, method, and medium for enhancing visual comfort. Background Art

[0002] Display systems have become an integral part of modern life, with people spending significant amounts of time viewing content on a variety of devices, such as computers, smartphones, tablets, and virtual reality goggles / helmets. While these display devices have greatly improved our access to information and entertainment, prolonged use can lead to visual discomfort and fatigue. These traditional display systems typically present content without consideration for user comfort or potential long-term vision effects. Their static nature cannot meet the dynamic demands of the human visual system, which has evolved to process central and peripheral visual information differently.

[0003] Furthermore, the increasing adoption of high-resolution displays and immersive technologies has created new challenges in managing visual comfort. Users may experience eye strain, headaches, or other discomfort when viewing content for extended periods, particularly in environments with poor lighting conditions or when using devices late at night.

[0004] While significant progress has been made in image processing to improve visual quality, less attention has been paid to optimizing content for visual comfort. While some systems use basic techniques such as blue light filtering or brightness adjustment, more sophisticated approaches are needed to dynamically adapt to individual user needs and viewing conditions. Furthermore, growing concern about the potential impact of prolonged screen time on vision health, particularly among children and young adults, highlights the importance of developing display technologies that not only provide a high-quality visual experience but also prioritize long-term eye health.

[0005] As display technology continues to advance and become more integrated into daily life, there is a growing need for innovative solutions that balance visual quality, user comfort, and potential health concerns. These advances could have profound implications for a wide range of sectors, including education, entertainment, and professional environments where prolonged screen time is required. Summary of the Invention

[0006] In view of the deficiencies in the prior art, the present invention aims to provide an adaptive display buffer system, method and medium for enhancing visual comfort.

[0007] According to the present invention, an adaptive display buffer system for enhancing visual comfort is provided, comprising:

[0008] an off-screen display buffer configured to store raw display data;

[0009] a content analysis module, comprising a feature encoder and a scoring submodule, wherein the feature encoder is configured to generate a coding vector based on the original display data, and the scoring submodule is configured to score content features based on the coding vector;

[0010] a gaze estimation module configured to determine a gaze direction of a user;

[0011] a content adjuster configured to modify content features of the original display data based on the content feature score and the user's gaze direction, wherein the content adjuster applies different modifications to the display in the foveal area and the peripheral area;

[0012] The screen display buffer is configured to store modified display data.

[0013] Furthermore, the method of generating the encoding vector according to the original display data includes: using domain-specific filters or layers to extract relevant content features.

[0014] Furthermore, the method of analyzing the encoding vector to generate content feature scores includes: scoring different content features separately through different scoring submodules;

[0015] Among them, when scoring, the weights of different content features are determined according to their impact on visual comfort. The main content features are analyzed first, and then the secondary content features are analyzed when necessary.

[0016] Furthermore, the content features include brightness distribution, contrast distribution, color distribution and spatial frequency distribution.

[0017] Furthermore, a user feedback loop module is included, which is configured to collect user feedback and input it into the content adjuster, so as to optimize the adjustment process of content features according to the collected user feedback.

[0018] According to the present invention, an adaptive display buffering method for enhancing visual comfort is provided, comprising:

[0019] Get raw display data from the off-screen display buffer;

[0020] generating a coding vector based on the original display data, and performing a content feature score based on the coding vector;

[0021] Modifying content features of the original display data according to the content feature score and the user's gaze direction, wherein the content adjuster applies different modifications to the display in the foveal area and the peripheral area;

[0022] Store the modified display data into the screen display buffer.

[0023] Furthermore, the method of generating the encoding vector according to the original display data includes: using domain-specific filters or layers to extract relevant content features.

[0024] Furthermore, the method of analyzing the encoding vector to generate content feature scores includes: scoring different content features separately through different scoring submodules;

[0025] Among them, when scoring, the weights of different content features are determined according to their impact on visual comfort. The main content features are analyzed first, and then the secondary content features are analyzed when necessary.

[0026] Furthermore, it also includes: collecting user feedback and inputting it into a content adjuster, which is used to optimize the adjustment process of content features based on the collected user feedback.

[0027] According to the present invention, a computer-readable storage medium storing a computer program is provided. When the computer program is executed by a processor, the steps of the adaptive display buffering method for enhancing visual comfort are implemented.

[0028] Compared with the prior art, the present invention has the following beneficial effects:

[0029] The present invention can enhance visual comfort, implement intelligent rendering technology, and alleviate vision-related health problems. Furthermore, the present invention has the following advantages:

[0030] 1. Personalized visual experience: By integrating user feedback and gaze information, the system provides a customized visual experience adapted to individual needs and preferences.

[0031] 2. Improve XR device performance: Foveated rendering can significantly reduce the computational requirements in VR, AR, and MR applications without sacrificing perceived quality.

[0032] 3. Potential health benefits: The ability of the system to adjust to visual stimuli, particularly in the peripheral visual field, may promote eye health by potentially slowing the progression of myopia or reducing eye strain.

[0033] 4. Adaptability to different content types: Comprehensive content analysis allows the system to apply appropriate modifications to various types of content, from text documents to dynamic videos.

[0034] 5. Real-time processing: By compressing information using feature encoders, the system can perform complex analysis and adjustments in real time, ensuring a responsive user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Other features, objects and advantages of the present invention will become more apparent upon reading the detailed description of non-limiting embodiments with reference to the following drawings:

[0036] Figure 1 Schematic diagram of the system structure of the present invention;

[0037] Figure 2 A diagram showing the coordination relationship between the modules of the present invention;

[0038] Figure 3 It is the workflow diagram of the present invention. DETAILED DESCRIPTION

[0039] The present invention will be described in detail below with reference to specific embodiments. The following examples will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, for those skilled in the art, several changes and improvements can be made without departing from the scope of the present invention. These all fall within the scope of protection of the present invention.

[0040] like Figure 1 and Figure 2 As shown, the present invention provides an adaptive display buffer system for enhancing visual comfort, including an off-screen display buffer, a content analysis module, a gaze estimation module, a content adjuster and an on-screen display buffer.

[0041] The off-screen display buffer is configured to store raw display data, which includes raw image or video data before any comfort enhancement modifications are applied, wherein the video data is composed of image data for each frame. The raw display data may include, but is not limited to, pixel values, color information, and other relevant data related to the content to be displayed.

[0042] The content analysis module includes a feature encoder and a scoring submodule, which work together to analyze the raw display data and generate content feature scores.

[0043] The feature encoder is configured to take as input the raw display data in the off-screen display buffer and output an encoded vector. This process may compress the information in the frame, enabling faster analysis and processing. The feature encoder may utilize a convolutional neural network or other machine learning method to efficiently extract relevant features from the input data.

[0044] Feature encoders are specifically designed to extract task-related features such as brightness, contrast, color, and spatial frequency using domain-specific filters or layers. For example, for brightness distribution feature extraction, a brightness convolution kernel is designed that can highlight the differences between different brightness regions. K l (m, n), whose element values ​​are set according to the mathematical properties of brightness features, such as giving different weights to the central pixel and its surrounding pixels to enhance the sensitivity to brightness changes. For contrast distribution feature extraction, first calculate the contrast mean of the image μ I , then the designed contrast convolution kernel K c (m, n) , focuses more on highlighting the areas in the image that are significantly different from the mean, thereby extracting contrast features. When extracting color distribution features, the color space conversion principle is used to design the color convolution kernel. K s (m, n) , to extract the relationship features between different color channels. For spatial frequency distribution feature extraction, the Fourier transform correlation principle is used to design the frequency convolution kernel. K f (m, n) , so that it can capture the information of different frequency components in the image. m, n are offsets.

[0045] The original display data I(x,y) As input, the convolution operation is performed on these domain-specific convolution kernels to obtain the brightness distribution feature map L(x,y) , contrast distribution feature map C(x,y) , color distribution feature map S(x,y) , spatial frequency distribution characteristic diagram F(x,y) .

[0046]

[0047]

[0048]

[0049]

[0050] Then, the feature maps of each frame are combined into an encoding vector.

[0051] This ensures that the encoded latent vector is optimized for use directly in the scoring module, minimizing additional preprocessing and improving efficiency.

[0052] The scoring submodule generates a coding vector based on the raw display data and is configured to score content features based on the coding vector. This module may include multiple submodules, each responsible for analyzing and scoring a specific aspect of the content. These submodules may be implemented using arithmetic functions, machine learning models, or neural networks. They may take coding or latent vectors as input and output scoring values ​​for various content features. These content features may include, but are not limited to, brightness (brightness distribution), contrast (contrast distribution), color (color distribution), and spatial frequency (spatial frequency distribution). The scores provided by these submodules may serve as input to content moderators to make content modification decisions.

[0053] To ensure effective and rapid adjustments of content moderators, a scoring process was designed.

[0054] Parameter prioritization and weighting: A weighted scoring system is used to prioritize key parameters (e.g., brightness, contrast) based on their impact on visual comfort. Less critical parameters are assigned lower weights, reducing the computational burden. Content scoring is implemented in a hierarchical process, where primary parameters (e.g., brightness and contrast) are analyzed first for immediate adjustment, and secondary parameters (e.g., spatial frequency) are analyzed only when needed.

[0055] The scoring submodule uses principal component analysis (PCA) or similar techniques to reduce the complexity of the latent vector, ensuring faster computation without significant information loss. Adaptive scoring thresholds allow the content moderator to focus only on parameters exceeding a certain predefined threshold, ignoring negligible changes. A parallel processing framework allows multiple parameters to be scored simultaneously, significantly reducing latency and ensuring real-time content adjustments.

[0056] The gaze estimation module is configured to determine the user's gaze direction. This module may be configured to determine the user's current gaze direction. In some cases, the gaze estimation module may utilize eye-tracking hardware or software-based gaze estimation technology. Gaze information may be used to differentiate between the user's foveal (central) and peripheral vision, enabling targeted content adjustment. In some cases, determining the user's current gaze direction is crucial for identifying areas of interest (AOIs) within a display.

[0057] The content conditioner is configured to modify content features of the original display data based on the content feature score and the user's gaze direction, wherein the content conditioner applies different modifications to the display in the foveal area and the peripheral areas. These modifications may be based on the score provided by the content analysis module and the gaze information provided by the gaze estimation module. The content conditioner may modify various content parameters, including brightness (brightness distribution), contrast (contrast distribution), color (color distribution), and spatial frequency (spatial frequency distribution). In some cases, the modifications applied to the foveal area may be different in intensity or type than the modifications applied to the peripheral areas. This ability to apply different modifications to different areas of the displayed content based on the user's gaze direction may enable advanced techniques such as foveated rendering. These modifications can enhance visual comfort, enable advanced rendering techniques, and potentially alleviate vision-related health issues.

[0058] The content conditioner applies parameter modifications directly to the raw image data within the foveal (ROI) and peripheral (non-ROI) regions based on real-time integration. Dynamic adjustments to brightness, contrast, spatial frequency, and color in both regions ensure seamless transitions while maintaining content fidelity. Modifications to the foveal region are precisely focused for high detail (e.g., sharpness, color enhancement). Peripheral region adjustments are designed for energy efficiency and reduced processing load, such as subtle blurring or dimming. The process uses content scoring outputs (e.g., brightness levels or gaze heatmaps) to dynamically prioritize parameters for adjustment, making it personalized and resource-efficient.

[0059] The modifications applied by the content modifier to the foveal region may differ in strength or type from those applied to peripheral regions. This ability to apply different modifications to different regions of displayed content based on the user's gaze direction may enable advanced techniques such as foveated rendering, in which image quality or characteristics can be selectively adjusted to match the capabilities of the human visual system. This approach may improve the user's overall visual comfort during extended viewing, reducing eye strain and fatigue, and may promote eye health by potentially slowing the progression of myopia or reducing eye fatigue.

[0060] For example, for the fovea, the histogram equalization algorithm is used to enhance image details. Assuming that the original image pixel value is p, the pixel value q after histogram equalization is calculated as follows:

[0061]

[0062] Where L is the total number of gray levels in the image, N is the total number of pixels in the image, is the number of pixels with gray level i.

[0063] For the peripheral area, Gaussian blur algorithm is used for blurring. Original display data I'(x,y) After Gaussian blurring, I'(x,y) .

[0064]

[0065] in, is the standard deviation of the Gaussian kernel, which is adjusted according to the visual characteristics of the peripheral area, and e is a natural constant. The closer to the current pixel, the smaller the values ​​of m and n. The closer it is to 1, the greater the weight of the surrounding pixels in the blur operation, and the farther the distance, the smaller the weight. This reflects the characteristic of Gaussian blur that the weighted sum of surrounding pixels is assigned different weights according to the distance, which is consistent with the human visual perception of the degree of blur in the peripheral area.

[0066] In some cases, the content conditioner may detect that the overall brightness of the content is too high for comfortable viewing, particularly in low-light environments. The content conditioner may then reduce the brightness of the peripheral areas more significantly than the foveal area, preserving detail where the user is focused while reducing any discomfort caused by brighter peripheral areas. In other cases, the content conditioner may identify low-contrast regions in the foveal area that are important for content comprehension. The content conditioner may selectively boost the contrast in these areas while leaving the peripheral areas unchanged, thereby improving readability without affecting the overall perceived brightness of the display. In some aspects, the system may modulate spatial frequency content, particularly in the peripheral areas, to potentially address myopia progression. This may involve enhancing certain spatial frequencies hypothesized to be protective against myopia progression while ensuring that the foveal area maintains the high visual quality required for the task at hand. This ability to apply different modifications to different areas of displayed content based on the user's gaze direction may enable advanced techniques such as foveated rendering, in which image quality or features can be selectively adjusted to match the capabilities of the human visual system.

[0067] The screen display buffer is configured to store modified display data for presentation on a display device. The screen display buffer may receive modified content from a content regulator and present it on the display device to provide an enhanced visual experience for the user.

[0068] In some embodiments, the system may also include a user feedback loop module for collecting user feedback and optimizing the content moderation process based on the collected feedback. This loop may be configured to collect user feedback and optimize the content moderation process based on the collected feedback. In some cases, user feedback may be collected through user interface elements or inferred from physiological responses. The feedback may be used to adjust the behavior of the content moderator over time, allowing the system to learn user preferences and visual comfort requirements. This feedback loop may be particularly active during the initial calibration phase, allowing the system to adapt to the user's needs and preferences.

[0069] In some aspects, a user feedback loop module may be connected to a display device to allow user input to influence the content moderation process. This feedback may then be passed back to the content moderator, creating a closed-loop system that can adapt to user preferences or needs.

[0070] This adaptive display buffering system is designed to provide a personalized visual experience that adapts to individual needs and preferences, potentially improving the performance of extended reality devices and supporting eye health. The system's ability to analyze and modify displayed content in real time ensures a responsive user experience that adapts to different content types and viewing conditions.

[0071] This system has several potential applications. For example, a primary goal could be to improve a user's overall visual comfort during extended viewing. By dynamically adjusting content based on analysis and user gaze, the system could reduce eye strain and fatigue. In virtual reality (VR), augmented reality (AR), and mixed reality (MR) devices, the system could enable efficient foveated rendering. This technique allocates more computing resources to the user's fovea to render high-quality content, while reducing detail in peripheral areas, optimizing performance without sacrificing perceived quality. Furthermore, the system could potentially help slow, reverse, prevent, delay, inhibit, or control eye growth and / or the refractive condition of the eye. By carefully modulating the visual stimuli presented to the user, particularly in peripheral vision, the system could potentially influence eye growth patterns.

[0072] like Figure 3 As shown, the present invention also provides an adaptive display buffer method for enhancing visual comfort. The adaptive display buffer system for enhancing visual comfort can be implemented by executing the process steps of the adaptive display buffer method for enhancing visual comfort. In other words, those skilled in the art can understand the adaptive display buffer method for enhancing visual comfort as a preferred embodiment of the adaptive display buffer system for enhancing visual comfort. The method includes:

[0073] Gets raw display data from the off-screen display buffer.

[0074] An encoding vector is generated according to the original display data, and a content feature score is performed according to the encoding vector.

[0075] The content features of the original display data are modified according to the content feature score and the user's gaze direction, wherein the content adjuster applies different modification modes to the display in the fovea area and the peripheral area.

[0076] Store the modified display data into the screen display buffer.

[0077] In certain embodiments, a system or method for enhancing the visual comfort of displayed content may be implemented as a non-transitory computer-readable storage medium. The medium may store instructions that, when executed by a processor, cause the processor to perform operations for enhancing the visual comfort of displayed content. These operations may include receiving raw data from an off-screen buffer, encoding the raw data to generate an encoding vector, analyzing the encoding vector to generate a content feature score, determining a user's gaze direction, modifying the raw image data based on the content feature score and the user's gaze direction, and outputting the modified image data to a screen buffer for display.

[0078] Application examples:

[0079] Brightness adjustment

[0080] In one implementation, the system may detect that the overall brightness of the content is too high for comfortable viewing, particularly in low-light environments. The content adjuster may reduce the brightness of the peripheral area more aggressively than the foveal area, preserving detail where the user is focusing while reducing potential discomfort from bright areas in the peripheral area.

[0081] Contrast enhancement

[0082] The system may identify low-contrast areas within the foveal region that are important for content comprehension. The content moderator can selectively enhance the contrast in these areas while leaving surrounding areas unchanged, thereby improving readability without affecting the overall perceived brightness of the display.

[0083] Color temperature adjustment

[0084] Depending on the time of day or user preference, the system can gradually adjust the color temperature of the display. Content moderators may apply stronger blue light reduction to peripheral areas compared to the foveal area, which may help reduce eye strain during nighttime use without significantly affecting color perception in the focal area.

[0085] spatial frequency modulation

[0086] To potentially address myopia progression, the system could modulate spatial frequency content, particularly in the periphery. This could involve enhancing certain spatial frequencies hypothesized to offer protection against myopia progression, while ensuring that the foveal region maintains the high visual quality required for the task at hand.

[0087] Those skilled in the art will appreciate that, in addition to implementing the system and its various devices, modules, and units provided by the present invention in purely computer-readable program code, it is entirely possible to implement the same functions of the system and its various devices, modules, and units provided by the present invention in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, the system and its various devices, modules, and units provided by the present invention can be considered a hardware component, and the devices, modules, and units included therein for implementing various functions can also be considered as structures within the hardware component; the devices, modules, and units for implementing various functions can also be considered as both software modules implementing the method and structures within the hardware component.

[0088] The above describes specific embodiments of the present invention. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art may make various changes or modifications within the scope of the claims, which do not affect the essence of the present invention. The embodiments of this application and the features in the embodiments may be combined with each other in any manner unless there is a conflict.

Claims

1. An adaptive display buffer system for enhancing visual comfort, characterized in that: include: an off-screen display buffer configured to store raw display data; a content analysis module, comprising a feature encoder and a scoring submodule, wherein the feature encoder is configured to generate a coding vector based on the original display data, and the scoring submodule is configured to score content features based on the coding vector; a gaze estimation module configured to determine a gaze direction of a user; a content adjuster configured to modify content features of the original display data based on the content feature score and the user's gaze direction, wherein the content adjuster applies different modifications to the display in the foveal area and the peripheral area; A screen display buffer configured to store modified display data; The feature encoder converts the raw display data I(x,y) As input, the convolution operation is performed on the domain-specific convolution kernel to obtain the brightness distribution feature map L(x,y) , contrast distribution feature map C(x,y) , color distribution feature map S(x,y) , spatial frequency distribution characteristic diagram F(x,y) ; in, is the brightness convolution kernel, is the color convolution kernel, is the frequency convolution kernel, is the contrast convolution kernel, m and n are offsets; Then, the feature maps of each frame are combined into an encoding vector; The content regulator includes: For the fovea area, the histogram equalization algorithm is used to enhance image details. The original image pixel value is p, and the pixel value q after histogram equalization is calculated as follows: Where L is the total number of gray levels in the image, N is the total number of pixels in the image, is the number of pixels with gray level i; For the peripheral area, Gaussian blur algorithm is used for blurring, and the original display data I'(x,y) After Gaussian blurring, I'(x,y) ; in, is the standard deviation of the Gaussian kernel, which is adjusted according to the visual characteristics of the peripheral area. e is a natural constant. The closer to the current pixel, the smaller the value of m and n. The closer it is to 1, the greater the weight of the surrounding pixels in the blur operation, and the farther the distance, the smaller the weight.

2. The adaptive display buffer system for enhancing visual comfort according to claim 1, characterized in that: The encoding vector is generated according to the raw display data in a manner that includes: using domain-specific filters or layers to extract relevant content features.

3. The adaptive display buffer system for enhancing visual comfort according to claim 1, characterized in that: The method of analyzing the coding vector to generate content feature scores includes: scoring different content features separately through different scoring submodules; Among them, when scoring, the weights of different content features are determined according to their impact on visual comfort. The main content features are analyzed first, and then the secondary content features are analyzed when necessary.

4. The adaptive display buffer system for enhancing visual comfort according to claim 1, characterized in that: The content characteristics include brightness distribution, contrast distribution, color distribution and spatial frequency distribution.

5. The adaptive display buffer system for enhancing visual comfort according to claim 1, characterized in that: The system further includes a user feedback loop module configured to collect user feedback and input the feedback into a content adjuster for optimizing the adjustment process of content features according to the collected user feedback.

6. An adaptive display buffering method for enhancing visual comfort, characterized in that: include: Get raw display data from the off-screen display buffer; generating a coding vector based on the original display data, and performing a content feature score based on the coding vector; Modifying content features of the original display data according to the content feature score and the user's gaze direction, wherein the content adjuster applies different modifications to the display in the foveal area and the peripheral area; Store the modified display data into the screen display buffer; Ways to generate encoding vectors include: The original display data I(x,y) As input, the convolution operation is performed on the domain-specific convolution kernel to obtain the brightness distribution feature map L(x,y) , contrast distribution feature map C(x,y) , color distribution feature map S(x,y) , spatial frequency distribution characteristic diagram F(x,y) ; in, is the brightness convolution kernel, is the color convolution kernel, is the frequency convolution kernel, is the contrast convolution kernel, m and n are offsets; Then, the feature maps of each frame are combined into an encoding vector; The content regulator includes: For the fovea area, the histogram equalization algorithm is used to enhance image details. The original image pixel value is p, and the pixel value q after histogram equalization is calculated as follows: Where L is the total number of gray levels in the image, N is the total number of pixels in the image, is the number of pixels with gray level i; For the peripheral area, Gaussian blur algorithm is used for blurring, and the original display data I'(x,y) After Gaussian blurring, I'(x,y) ; in, is the standard deviation of the Gaussian kernel, which is adjusted according to the visual characteristics of the peripheral area. e is a natural constant. The closer to the current pixel, the smaller the value of m and n. The closer it is to 1, the greater the weight of the surrounding pixels in the blur operation, and the farther the distance, the smaller the weight.

7. The adaptive display buffering method for enhancing visual comfort according to claim 6, characterized in that: The encoding vector is generated according to the original display data in a manner that includes: using domain-specific filters or layers to extract relevant content features.

8. The adaptive display buffering method for enhancing visual comfort according to claim 6, characterized in that: The method of analyzing the coding vector to generate content feature scores includes: scoring different content features separately through different scoring submodules; Among them, when scoring, the weights of different content features are determined according to their impact on visual comfort. The main content features are analyzed first, and then the secondary content features are analyzed when necessary.

9. The adaptive display buffering method for enhancing visual comfort according to claim 6, characterized in that: Also includes: User feedback is collected and input into a content moderator for optimizing the content feature moderation process based on the collected user feedback.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the adaptive display buffering method for enhancing visual comfort according to any one of claims 6 to 9 are implemented.

Images