A scene-based sub-pixel rendering method and system

Abstract

The application belongs to the field of display devices, and discloses a scene-based sub-pixel rendering method and system, which comprises the following steps: in a manual SPR adjustment mode, collecting mobile phone power, and user's preference data of SPR parameters in different display scenes; creating a neural network, and training a machine learning algorithm according to the collected data; in an automatic SPR adjustment mode, acquiring battery power data and the actual display application scene of the user respectively; inputting the above data into the trained machine learning algorithm, and adaptively adjusting the SPR parameters. The application determines the optimal parameters of sub-pixel rendering effect based on the visual effect preference data of the user in different scenes, can automatically adjust the display parameters according to the content and scene of the picture to achieve the best visual effect, reduces power consumption, and greatly improves the user experience.

Description

Technical Field

[0001] This invention belongs to, but is not limited to, the field of display device technology, and particularly relates to a scene-based subpixel rendering method and system. Background Technology

[0002] Organic light-emitting diode (OLED) displays have many advantages, such as being thin and light, actively emitting light, having a fast response time, a wide viewing angle, a wide color gamut, high brightness, and low power consumption. They are gradually becoming the third generation of display technology after liquid crystal displays (LCDs).

[0003] Driven by consumer demand, OLED displays are trending towards higher resolution and smaller sizes to provide more detailed and realistic images and a more convenient and comfortable user experience. However, due to technological limitations, manufacturing extremely small and highly dense pixel units is very difficult, affecting yield rates, increasing manufacturing costs, and adding to the design complexity of the driving circuit system. Subpixel rendering (SPR) technology improves perceived resolution by having adjacent pixels share some subpixels. This allows for higher perceived resolution with the same subpixel density, or reduces the required subpixel density while maintaining the same perceived resolution. Therefore, subpixel rendering technology provides an effective solution to these challenges, and the subpixel rendering algorithm directly impacts the image quality of OLED displays.

[0004] Subpixel rendering quality depends on subjective evaluation and is directly related to the displayed image and user preferences; there are no uniform parameters that satisfy all scenarios. For text images, which contain a lot of high-frequency information, smoothing is necessary to avoid color aliasing and resulting edge-like artifacts. Natural images primarily contain low- to mid-frequency information, allowing for a sharper parameter strategy to preserve image details. However, current subpixel rendering methods focus on eliminating color aliasing, using fixed smoothing parameter settings across different scenarios. While this approach improves display quality in text-centric applications, it's detrimental to the realistic rendering of details in other scenarios such as images and videos. Furthermore, because smoothing settings require a large data window for hardware processing, indiscriminate use increases power consumption and degrades the user experience.

[0005] Based on the above analysis, the urgent technical problems that need to be solved in the existing technology are:

[0006] Existing subpixel rendering methods use fixed parameter settings, which are not conducive to presenting personalized display effects in rich scenarios. Furthermore, using them indiscriminately will increase power consumption and reduce user experience. Summary of the Invention

[0007] To address the problems existing in the prior art, this invention provides a scene-based subpixel rendering method and system.

[0008] This invention is implemented as follows: a scene-based subpixel rendering method, characterized in that the scene-based subpixel rendering method specifically includes:

[0009] S1: In manual SPR adjustment mode, data on phone battery level and user preferences for SPR parameters in different display scenarios are collected.

[0010] S2: Create a machine learning algorithm model and train the machine learning algorithm based on the data collected in S1;

[0011] S3: In automatic SPR adjustment mode, battery power data and the user's actual display application scenario are obtained respectively;

[0012] S4: Input the data collected in S3 into the trained machine learning algorithm and adaptively adjust the SPR parameters.

[0013] Furthermore, S1 collects current battery power data, the user's actual display application scenario, such as reading mode, movie mode, game mode, etc., and the corresponding SPR parameters manually set by the user.

[0014] Furthermore, the training of the machine learning algorithm can be carried out offline or on a cloud server. The trained machine learning algorithm is built locally on the terminal and completed with the help of the Neural Processing Unit (NPU) in the terminal application processor (AP).

[0015] Another objective of this invention is to provide a scene-based subpixel rendering system, which specifically includes:

[0016] Data collection module 1 is used to collect data in manual SPR adjustment mode;

[0017] The algorithm module is used to build machine learning algorithm models;

[0018] Data collection module 2 is used to collect data in automatic SPR adjustment mode;

[0019] The adaptive adjustment module is used to adaptively adjust the SPR parameters based on the output of the algorithm module.

[0020] Based on the above technical solutions and the technical problems solved, the advantages and positive effects of the technical solution to be protected by this invention are as follows:

[0021] This invention proposes a scene-based subpixel rendering method and system. By collecting and analyzing user visual effect preference data in different display scenarios, the optimal parameter settings for subpixel rendering are determined. This method is based on the user's actual display application scenario, such as reading mode, movie mode, and game mode, combined with the current battery status of the device. Using a trained machine learning algorithm model, it adaptively adjusts the subpixel rendering parameters, thereby achieving a dynamic balance between display effect and energy consumption. This technical solution can significantly improve the overall efficiency of device use while meeting the user's visual experience requirements.

[0022] This invention utilizes intelligent algorithm design to collect user preference data in manual adjustment mode and adjust display parameters in real time in automatic adjustment mode, enabling subpixel rendering optimization to adapt to different scenarios and content. For example, in reading mode, which primarily features text information and contains many spatial high-frequency components, a smooth subpixel rendering strategy is needed to reduce the color fringing effect at text edges and improve the reading experience. In game mode, the overall screen transition is natural, with fewer spatial high-frequency components, and the color fringing effect is not obvious; therefore, a sharper subpixel rendering strategy is used to preserve the image details. Furthermore, game scenarios generally have higher refresh rates and higher power consumption; using a sharper subpixel rendering strategy can save power and extend the usage time of the terminal device. Compared with traditional fixed parameter settings, this invention can intelligently adjust the display effect for different scenarios, truly achieving a personalized user experience while avoiding the inconvenience of frequent manual operations.

[0023] The technical solution of this invention has significant expected benefits, mainly reflected in improving the display effect and power consumption optimization capabilities of terminal devices. By adaptively adjusting sub-pixel rendering parameters, the device can extend battery life and provide a higher quality visual experience, meeting users' personalized needs for display effects in different scenarios. This technical solution not only improves user satisfaction but also gives terminal devices a differentiated advantage in market competition, especially in the fields of smartphones, tablets, and televisions, where it has significant commercial value and market potential.

[0024] By applying this invention to actual terminal devices, it effectively enhances the market competitiveness of products. Its intelligent and scenario-based display optimization functions provide a strong selling point for terminal devices, attracting consumers' demand for high-performance and intelligent devices. Simultaneously, this technical solution can reduce product energy consumption, aligning with green and environmentally friendly design trends, further expanding its application scope and market acceptance. In summary, the implementation of this invention not only brings a better user experience but also creates significant added value for terminal device manufacturers. Attached Figure Description

[0025] Figure 1 This is a flowchart of a scene-based subpixel rendering method provided in an embodiment of the present invention;

[0026] Figure 2 This is a schematic diagram illustrating the process of training a machine learning algorithm using user data collected in manual mode according to an embodiment of the present invention;

[0027] Figure 3 This is a schematic diagram of the adaptive adjustment of SPR mode in automatic mode provided in an embodiment of the present invention;

[0028] Figure 4 This is a schematic diagram of a scenario-based SPR implementation scheme provided in an embodiment of the present invention;

[0029] Figure 5 This is a schematic diagram of the second scenario-based SPR implementation scheme provided in this embodiment of the invention;

[0030] Figure 6 This is a block diagram of a scene-based subpixel rendering system provided in an embodiment of the present invention;

[0031] Figure 7 This is a comparison of SPR effects in different scenarios provided by the embodiments of the present invention. Detailed Implementation

[0032] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0033] like Figure 1 As shown, this embodiment of the invention provides a scene-based subpixel rendering method, which specifically includes:

[0034] S1: In manual SPR adjustment mode, data on phone battery level and user preferences for SPR parameters in different display scenarios are collected.

[0035] S2: Create a machine learning algorithm model and train the machine learning algorithm based on the data collected in S1;

[0036] S3: In automatic SPR adjustment mode, battery power data and the user's actual display application scenario are obtained respectively;

[0037] S4: Input the data collected in S3 into the trained machine learning algorithm and adaptively adjust the SPR parameters.

[0038] In manual SPR (Sub-Pixel Rendering) adjustment mode, the system uses data collection module 1 to acquire user parameter setting behaviors in different usage scenarios, such as user preferences for SPR parameter adjustments in reading mode, movie mode, and game mode. Simultaneously, the system records factors affecting user experience, such as current battery level and ambient brightness. The collected data is stored uniformly for subsequent training of machine learning models. This process establishes a correlation between user display habits and device operating status, providing a foundation for automatically adjusting SPR parameters.

[0039] Based on user-defined SPR parameters and corresponding display scenarios, the algorithm module creates a machine learning algorithm model and trains it using machine learning techniques. Training can be completed on an offline server or in the cloud, and the model is deployed in a lightweight form on the terminal device, relying on the terminal's neural network processing unit (NPU) for efficient operation. During training, the algorithm learns the non-linear relationship between user preferences and display scenarios, establishing a mapping from input (battery level and display scenario) to output (optimal SPR parameters).

[0040] When a user switches to automatic SPR adjustment mode, data collection module 2 acquires the current device status in real time, including battery level, ambient brightness, and the display application scenario the user is using. This data is input into a pre-trained machine learning algorithm model, which calculates the most suitable SPR parameter values ​​for the current scenario based on the input feature parameters. This calculation process combines the user's historical usage preferences and device status to ensure optimized display effects and a consistent user experience.

[0041] The adaptive adjustment module adjusts the SPR parameters in real time based on the output of the machine learning algorithm model to optimize the display effect. For example, when the battery is low, the system prioritizes the energy-saving SPR configuration; in reading mode with a full battery, the system prioritizes a smooth SPR setting to achieve the best display effect; and when watching movies or playing games, the system chooses a sharper SPR setting to balance display effect and power consumption. This adaptive adjustment mechanism not only meets the user's display needs but also maximizes the device's battery life. Through real-time data collection and continuous feedback, the system can also gradually update and optimize the user preference model, further improving display quality and user satisfaction.

[0042] S1 collects current battery power data, the user's actual display application scenario, such as reading mode, movie mode, game mode, etc., and the corresponding SPR parameters manually set by the user.

[0043] Figure 2 This is a diagram illustrating the process of training machine learning algorithms using user data collected in manual mode.

[0044] Figure 3 This is a schematic diagram of the adaptive adjustment process of SPR mode in automatic mode.

[0045] In manual SPR adjustment mode, user-related data is collected through the mobile terminal's sensors and operation log system. This includes battery status, the user's current actual display application scenario (such as reading mode, movie mode, game mode, etc.), and the SPR parameters manually adjusted by the user. The collected data is recorded with timestamps to ensure time-series consistency in dynamic environments. In addition, outliers, such as user misoperations or erroneous data collected by sensors, are removed to improve data accuracy.

[0046] The preprocessed data is input into the feature extraction module to analyze the potential relationship between battery level, application scenario, and SPR parameters. For example, in reading mode, users prefer realistic, detailed, and color-accurate text display, while in gaming mode, they prioritize device battery life. Extracted features include scene category, remaining battery percentage, and user adjustment frequency. All features are processed using normalization methods to ensure data consistency and training efficiency for the neural network.

[0047] Based on collected user data, machine learning algorithms (such as multilayer perceptrons or convolutional neural networks) are used for training. The model primarily considers the diversity of user display needs and the complexity of battery level changes, establishing prediction models for SPR parameters under different scenarios. Supervised learning methods are employed during training, using historical user preference data as labels, and cross-validation and regularization methods are used to prevent overfitting while optimizing the model's generalization ability.

[0048] In automatic SPR adjustment mode, the phone's real-time sensor system acquires current battery level and application scenario data. This data is preprocessed and input into a trained machine learning model to predict SPR parameters suitable for the current scenario in real time. Finally, the device's display driver module adjusts the sub-pixel rendering parameters of the display screen to achieve adaptive optimization, satisfying both the user's visual experience and extending the device's battery life.

[0049] In practical applications, machine learning algorithms can be trained offline or on cloud servers. The trained algorithm needs to be built locally on the terminal to achieve real-time scene recognition and parameter adjustment. This solution utilizes the Neural Processing Unit (NPU) within the terminal's application processor (AP) to accomplish this.

[0050] Option 1: The NPU acquires images from the Graphics Processing Unit (GPU), performs scene recognition, and obtains other parameters from the AP. Based on a pre-established machine learning algorithm, it determines the final selected SPR parameters. The transmitting end transmits the SPR parameter information and the image signal output by the GPU to the receiving end of the Display Driver IC (DIC). The receiving end then transmits the two signals to the SPR module in the DIC. The SPR module selects an appropriate rendering method based on the SPR parameters. The rendered data is then output to the display panel directly or after passing through other image processing modules.

[0051] Option 2: The NPU acquires images from the Graphics Processing Unit (GPU), performs scene recognition, and obtains other parameters from the AP. Based on a pre-established machine learning algorithm, it determines the final selected SPR parameters. The SPR parameter information and the image signal output by the GPU are passed to the SPR module in the AP. The SPR module selects an appropriate rendering method based on the SPR parameters. The rendered data is transmitted to the DIC receiver through the AP's transmitter. The receiver outputs the signal directly or after passing through other image processing modules to the display panel.

[0052] The two schemes use different data formats from the AP transmitter to the DIC receiver, resulting in different data transmission protocols. Since the AP's manufacturing process is generally more advanced than the DIC's, in scheme two, integrating the SPR module into the AP saves power.

[0053] Figure 4 , Figure 5 These are schematic diagrams of SPR implementation scheme one and SPR item scheme two based on scenarios.

[0054] like Figure 6 As shown in the figure, an embodiment of the present invention provides a scene-based subpixel rendering system, which specifically includes:

[0055] Data collection module 1 is used to collect data in manual SPR adjustment mode;

[0056] The algorithm module is used to build machine learning algorithms;

[0057] Data collection module 2 is used to collect data in automatic SPR adjustment mode;

[0058] The adaptive adjustment module is used to adaptively adjust the SPR parameters based on the output of the algorithm module.

[0059] like Figure 6 As shown, this embodiment of the invention provides a scene-based subpixel rendering system designed to achieve an intelligent balance between display quality and power consumption. The system includes multiple functional modules that work together to dynamically adjust subpixel rendering parameters according to scene requirements, improving display quality while reducing power consumption.

[0060] Data collection module 1 is used to collect relevant data in manual SPR adjustment mode. Specifically, this module collects battery power data, displays application scenario data, and user-set SPR parameters, while recording the timestamp information of the data to provide training data for the subsequent establishment of machine learning algorithm models.

[0061] The algorithm module is used to build a machine learning algorithm neural network model based on the sample data provided by the data collection module 1. By analyzing the collected battery power and display scene data, this module trains a set of SPR parameters applicable to different scenarios, enabling the system to optimize display performance and power consumption for different usage scenarios.

[0062] Data collection module 2 operates in automatic SPR adjustment mode, collecting current battery power data and display application scenario data in real time. The adaptive adjustment module dynamically adjusts the sub-pixel rendering settings of the display screen based on the SPR parameters output by the algorithm module, specifically adjusting contrast, brightness, refresh rate, sharpness, and color rendering to meet the user's display needs in different scenarios.

[0063] The technical solution of this invention can be applied to terminal devices with display systems, such as smartphones, tablets, and televisions.

[0064] In manual SPR adjustment mode, data collection module 1 is responsible for collecting user operation-related data, including battery level, actual display application scenario (such as reading mode, movie mode, game mode, etc.), and the SPR parameters manually adjusted by the user in each scenario. During data collection, the system automatically records timestamps and scene classification labels, and preprocesses the sensor data, such as denoising and filtering outliers, to ensure the accuracy and consistency of the collected data.

[0065] The algorithm module utilizes data collected in manual mode to generate feature vectors for training through feature extraction. Features include scene type, battery status, and user preference adjustment parameters. Neural network algorithms from machine learning, such as Convolutional Neural Networks (CNNs) or Multilayer Perceptrons (MLPs), are used to model the data and uncover the potential relationship between user display needs and battery status. Supervised learning methods are used during training, incorporating historical data as labels, and regularization and cross-validation are employed to prevent overfitting and optimize the algorithm's generalization ability.

[0066] In automatic SPR adjustment mode, data collection module 2 collects real-time data on the device's battery status and displayed application scenarios. The collected data is rapidly analyzed by the module's signal processor, including classifying the current display scenario and assessing battery change trends. The processed data is then input into the algorithm module in the form of feature vectors as the basis for real-time parameter adjustments.

[0067] The adaptive adjustment module dynamically adjusts the subpixel rendering (SPR) parameters of the display screen based on the output of the algorithm module. For example, in low-power mode, the system prioritizes energy-efficient SPR parameter combinations; in reading mode, it prioritizes SPR parameter configurations that offer the best text display effect; and in movie and game modes, it focuses on rendering image details and improving battery life. The adjusted parameters are applied to the device screen through the display driver interface to achieve the best visual experience in different scenarios, while balancing battery life performance and user needs.

[0068] In typical reading scenarios, there is a lot of text, resulting in a high frequency of information displayed on the screen. If sharpening parameters are used, the difference in pixel brightness at the text edges can be significant, easily causing a visual fringing effect at the text edges. Figure 7 (a) SPR parameter 1 setting is shown. Therefore, in this scenario, a smoother parameter setting is generally used to reduce the difference in brightness at the text edges and alleviate the colored edge phenomenon, such as... Figure 7 (a) SPR parameter 2 settings are shown. It should be noted that... Figure 7 The effect shown is in the captured image; in actual applications, it will be more noticeable to the naked eye. Furthermore, most users prefer to reduce color fringing; in this scenario, they will select the smoothing parameter setting in manual mode, so that the automatic mode will intelligently select the smoothing parameter setting when the scene is detected. However, a few users are not sensitive to color fringing and prefer a sharper display; therefore, they will select the sharpening parameter setting in manual mode, so that the automatic mode will intelligently select the sharpening parameter setting when the scene is detected. This technical solution will select the parameter settings according to the user's actual preferences for the application scenario, meeting personalized needs.

[0069] In green landscape scenes with a predominantly green background, the color fringing is relatively weak due to the regular arrangement of subpixels. Therefore, sharpening parameters can be used to enhance detail, such as... Figure 7 (b) SPR parameter setting 1 is shown. Similarly, in this scenario, some users may prefer a softer display effect, such as... Figure 7 (b) SPR parameter settings are shown in Figure 2. This technical solution will select the parameter settings for the application scenario according to the user's actual preferences to meet personalized needs.

[0070] In ordinary natural scenes, because the displayed image contains relatively little high-frequency information, the difference between sharpening and smoothing settings is small and imperceptible to the naked eye. Figure 7 As shown in (c). However, with the smoothing parameter set, the SPR algorithm needs to store data from other rows during implementation, and the amount of data to be calculated is large, resulting in higher power consumption. In this display scenario, the SPR parameter setting can be selected according to the actual power consumption of the device to improve power utilization and extend the device's usage time.

[0071] Example 1: Subpixel rendering optimization based on reading mode

[0072] In manual SPR adjustment mode, the system collects display parameter settings data from the user in reading mode, including battery level, scene category labels (such as e-book reader or document browser), and manually adjusted SPR parameters (such as different filtering ranges and corresponding weights). Based on the collected data, a neural network model is trained to extract the user's display preference features at different battery levels. In automatic SPR adjustment mode, when the user opens the e-book application, the system obtains the current battery level (e.g., 90% high battery) and scene labels in real time. After calculation by the neural network model, it outputs SPR parameters suitable for the reading scenario, such as smoothing parameters to improve the text display effect on the screen and achieve an efficient reading experience.

[0073] Example 2: Subpixel rendering optimization based on game mode

[0074] In manual SPR adjustment mode, the system records the user's SPR parameter adjustments and corresponding battery status during game scenarios. The system builds a neural network model by collecting data multiple times, learning the relationship between user preferences and battery levels. In automatic SPR adjustment mode, when the user enters the game application, the system detects the current battery level (e.g., low 30%) and the application scenario in real time, and uses the trained neural network to select the most suitable SPR parameters, such as sharper parameters, to enhance image details and optimize battery power consumption, providing a more realistic gaming experience.

[0075] It should be noted that embodiments of the present invention can be implemented in hardware, software, or a combination of both. The hardware portion can be implemented using dedicated logic; the software portion can be stored in memory and executed by a suitable instruction execution system, such as a microprocessor or dedicated-design hardware. Those skilled in the art will understand that the above-described devices and methods can be implemented using computer-executable instructions and / or included in processor control code, for example, such code provided on a carrier medium such as a disk, CD, or DVD-ROM, a programmable memory such as read-only memory (firmware), or a data carrier such as an optical or electronic signal carrier. The devices and modules of the present invention can be implemented by hardware circuitry such as very large-scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, or programmable hardware devices such as field-programmable gate arrays, programmable logic devices, etc., or by software executed by various types of processors, or by a combination of the above-described hardware circuitry and software, such as firmware.

[0076] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications, equivalent substitutions, and improvements made by those skilled in the art within the scope of the technology disclosed in the present invention, and within the spirit and principles of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A scene-based subpixel rendering method, characterized in that, The method includes the following steps: S1: In manual SPR adjustment mode, data on phone battery level and user preferences for SPR parameters in different display scenarios are collected. The SPR parameters are different filtering ranges and corresponding weights. S2: Create a machine learning algorithm model and train the machine learning algorithm based on the data collected in S1; S3: In automatic SPR adjustment mode, battery power data and the user's actual display application scenario are obtained respectively; S4: Input the data collected in S3 into the trained machine learning algorithm and adaptively adjust the SPR parameters; S5: Adjust subpixel rendering based on SPR parameters; The SPR parameter preference data includes timestamp information and user-set SPR parameters; The display scenarios include at least one of reading mode, movie mode, and game mode.

2. The method according to claim 1, characterized in that, The acquired battery power data is preprocessed and normalized to a percentage format.

3. The method according to claim 1, characterized in that, The SPR parameter-based subpixel rendering adjustment includes dynamically adjusting at least one of the following: contrast, brightness, refresh rate, sharpness and smoothness, color rendering, and power consumption of the display screen.

4. The method according to claim 1, characterized in that, The machine learning algorithm model is trained using supervised learning based on historical display data and the SPR parameter dataset.

5. The method according to claim 1, characterized in that, The subpixel rendering adjustment process further incorporates temperature data to optimize rendering parameters.

6. A scene-based subpixel rendering system that implements the scene-based subpixel rendering method as described in claim 1, characterized in that, The system includes: Data collection module 1 is used to collect user display scene, SPR parameter preference data and battery power data in manual SPR adjustment mode; The algorithm module is used to build machine learning algorithm models and train machine learning algorithms; Data collection module 2 is used to collect equipment status data in real time during automatic SPR adjustment mode; The adaptive adjustment module is used to adjust the SPR parameters in real time based on the results output by the algorithm module.

7. The scene-based subpixel rendering system as described in claim 6, characterized in that, The data collection module 2 is integrated with the terminal's sensors, which can acquire information on device power, usage environment, and display scene changes in real time, and use it for SPR parameter optimization and adjustment.

Images