A method to accelerate image rendering under lvgl

Abstract

This invention provides a method for accelerating image rendering in an LVGL environment. This method improves image rendering performance through adaptive chunking, cache priority adjustment, intelligent predictive scheduling, and GPU parallel acceleration. Specific steps include: first, adaptively chunking the image and dynamically adjusting the cache priority based on image complexity, access frequency, and user interaction behavior; second, performing intelligent predictive scheduling based on historical user operation data, and preloading and rendering upcoming chunks; and third, accelerating high-priority chunks through GPU parallel processing, and adjusting the image compression rate and format based on device performance and current system load. This method effectively reduces image rendering latency, improves system response speed, and is suitable for embedded devices and resource-constrained system environments.

Description

Technical Field

[0001] This invention relates to the field of information processing technology, and in particular to a method for accelerating image rendering under LVGL. Background Technology

[0002] The proliferation of embedded and IoT devices has driven the growing demand for graphical user interface (GUI) systems, but these devices are often limited by processing power and memory resources. Traditional rendering methods easily become performance bottlenecks when using lightweight graphics libraries such as LVGL (Light and Versatile Graphics Library). Existing technologies primarily reduce computational burden through simple caching, fixed tile division, or reducing image quality; however, these methods lack the ability to adjust in real-time to user interaction behavior, image complexity, and system load, resulting in insufficient rendering optimization and an inability to flexibly respond to changes in system load and user needs.

[0003] In high-resolution images or complex interactive scenarios, this deficiency leads to system lag, especially when users frequently scroll, zoom, or click on images, where rendering delays are significant, thus affecting the overall user experience. Summary of the Invention

[0004] This invention provides a method for accelerating image rendering under LVGL to solve the above-mentioned technical problems.

[0005] This invention provides a method for accelerating image rendering under LVGL, comprising the following steps:

[0006] Step 1: Based on the complexity, texture, and color variations of each region of the image, adaptively divide the image into blocks;

[0007] Step 2: Analyze the complexity, access frequency, and user interaction behavior of image chunks, and adjust the chunk caching priority accordingly;

[0008] Step 3: Perform intelligent prediction and scheduling based on user behavior, and preload the image blocks that the user is about to access;

[0009] Step 4: Using GPU parallel acceleration, the image chunking tasks predicted and scheduled in Step 3 are allocated to GPU cores;

[0010] Step 5: Adjust the image compression rate and format based on the priority of image segmentation, device performance, and current system load.

[0011] Specifically, step 1 is carried out using the following steps:

[0012] Step 101: Load the image to be rendered into the LVGL rendering system and process the image using pixel-by-pixel rendering and double buffering mechanisms;

[0013] Step 102: After completing the LVGL rendering compatibility initialization, perform complexity analysis on each pixel region of the image and generate a complexity matrix;

[0014] Step 103: Perform adaptive block partitioning based on the complexity matrix generated in Step 102;

[0015] Step 104: After the adaptive block processing is completed, perform hierarchical rendering of each region according to the complexity of each block.

[0016] Specifically, step 2 is carried out using the following steps:

[0017] Step 201: Determine the caching priority of each block by analyzing the complexity, access frequency, and user interaction data of each block of the image;

[0018] Step 202: Adjust the system's processing order of blocks based on the block cache priority;

[0019] Step 203: Based on block priority, divide the blocks into different levels of the cache system for management;

[0020] Step 204: Calculate and score the blocks based on their priority and dwell time, and eliminate low-priority blocks.

[0021] Specifically, step 3 is carried out using the following steps:

[0022] Step 301: Collect user action data such as clicks, scrolling, and zooming, record action trajectory information, and generate a user interaction behavior dataset;

[0023] Step 302: Using the behavioral data provided by dataset D, construct the user's path from the current block to the next block A. j The state transition matrix;

[0024] Step 303: Combine the state transition matrix with the block cache priority to schedule the image blocks;

[0025] Step 304: Based on the scheduling results in step 303, the chunks that the user is about to access are preloaded and rendered in advance;

[0026] Step 305: Update the scheduling model and adjust the state transition matrix based on the user's operation trajectory.

[0027] Specifically, step 4 is carried out using the following steps:

[0028] Step 401: Allocate the block tasks to the various computing cores of the GPU according to the size of each block and the cache priority;

[0029] Step 402: Adjust the workload of each core through a load balancing mechanism;

[0030] Step 403: After task allocation and load balancing are completed, the GPU is used to render each block in parallel;

[0031] Step 404: Synchronize the rendering results using LVGL's double buffering mechanism;

[0032] Step 405: Adjust the block resolution and release unused computing resources.

[0033] Specifically, step 5 is carried out using the following steps:

[0034] Step 501: Adjust the compression rate of each image block according to the device's performance status and the block's cache priority;

[0035] Step 502: Select a suitable image format based on the compression quality and image type of each block;

[0036] Step 503: Adjust the compression parameters of each block by monitoring the device load and image rendering requirements;

[0037] Step 504: Reclaim the image resources used in the rendering process and optimize storage through format conversion;

[0038] Step 505: Submit the image chunk data, which has undergone dynamic compression and format conversion, to the LVGL rendering engine.

[0039] Specifically, the complexity matrix is ​​calculated using the following formula:

[0040]

[0041] Where M(Ai) represents the summation and average of the complexity of all pixels in the block;

[0042] |Ai| represents the number of pixels in block Ai;

[0043] (x,y)∈Ai(x,y) represents each pixel in block Ai, with coordinates (x,y);

[0044] G(x,y) represents the edge gradient of the image at pixel (x,y);

[0045] C(x,y) represents the rate of color change of the image at pixel (x,y);

[0046] T(x,y) represents the texture complexity of the image at pixel (x,y);

[0047] λ1,λ 2, λ3: These are the weighting coefficients, corresponding to edge gradient, color change, and texture complexity, respectively.

[0048] The beneficial effects of this invention are as follows:

[0049] 1. Improve rendering speed: By adaptively segmenting images and combining the complexity of image segments, historical access frequency, and user interaction behavior, the system can effectively adjust the rendering priority of each segment, ensuring that high-complexity and high-interaction areas are rendered first, thereby reducing image rendering latency and improving overall response speed.

[0050] 2. Optimize resource utilization: Effectively allocate system resources through cache priority management and GPU parallel acceleration. When cache resources are limited, low-priority blocks are dynamically evicted to free up space for high-priority areas, ensuring reasonable allocation and utilization of resources.

[0051] 3. Enhance user experience: The intelligent predictive scheduling mechanism preloads the chunks that the user is about to access based on the user's historical operation behavior, avoiding the delay between user operation and image rendering, and improving the smoothness of interface operation.

[0052] 4. Reduce system load: Through dynamic compression and format conversion, the system can dynamically adjust the image compression rate according to the current device performance and system load, reduce the rendering requirements of low-priority blocks, thereby reducing system load and maintaining system stability. Attached Figure Description

[0053] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0054] Figure 1 This is a schematic diagram of the method for accelerating image rendering under LVGL provided by the present invention.

[0055] Figure 2 This is a schematic diagram of the structure of Embodiment 1 of the present invention;

[0056] Figure 3 This is a schematic diagram of the structure of Embodiment 2 of the present invention;

[0057] Figure 4 This is a schematic diagram of the structure of Embodiment 3 of the present invention;

[0058] Figure 5This is a schematic diagram of the structure of Embodiment 4 of the present invention;

[0059] Figure 6 This is a schematic diagram of the structure of Embodiment 5 of the present invention; Detailed Implementation

[0060] To facilitate understanding of the present invention, a more complete description will be given below with reference to the accompanying drawings. Preferred embodiments of the invention are shown in the drawings. However, the invention can be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided to provide a thorough and complete understanding of the disclosure of the invention.

[0061] Example 1

[0062] This embodiment provides an image adaptive block processing method in an LVGL environment. It aims to analyze the complexity of each region of the image and achieve dynamic block processing for different regions to optimize the rendering efficiency and reduce the amount of rendering computation.

[0063] This embodiment provides a method for accelerating image rendering under LVGL, including the following steps:

[0064] Step 1: Based on the complexity, texture, and color variations of each region of the image, adaptively divide the image into blocks;

[0065] Step 2: Analyze the complexity, access frequency, and user interaction behavior of image chunks, and adjust the chunk caching priority accordingly;

[0066] Step 3: Perform intelligent prediction and scheduling based on user behavior, and preload the image blocks that the user is about to access;

[0067] Step 4: Using GPU parallel acceleration, the image chunking tasks predicted and scheduled in Step 3 are allocated to GPU cores;

[0068] Step 5: Adjust the image compression rate and format based on the priority of image segmentation, device performance, and current system load.

[0069] Step 1 is carried out using the following steps:

[0070] Step 101: LVGL Rendering Compatibility and Initialization

[0071] This step aims to ensure smooth image rendering in the LVGL environment, optimize the rendering process, and improve overall rendering efficiency. It consists of the following two parts:

[0072] (1) Initialize the LVGL renderer:

[0073] Before rendering the image, the LVGL renderer is first initialized to ensure the system rendering framework is ready. At this point, the image is loaded into the LVGL cache so that the system can begin rendering. Initially, the image is loaded into the system in a full-image rendering mode.

[0074] (2) Pixel-by-pixel rendering framework:

[0075] LVGL's rendering mechanism is based on pixel-by-pixel processing, meaning the system processes each pixel in the image one by one to render the complete image. In this embodiment, this mechanism is optimized by dividing the image into multiple regions or blocks to accommodate the complexity of each block. This allows more computational resources to be allocated to complex regions, reducing the computational burden on simpler regions, thereby accelerating the overall image rendering speed.

[0076] Step 102: Calculation of the complexity matrix

[0077] After completing the LVGL rendering compatibility initialization, a complexity analysis is performed on each pixel region of the image, generating a complexity matrix M(A). i This matrix is ​​used to represent the complexity of image segmentation and to provide a basis for adaptive segmentation.

[0078] The complexity matrix is ​​calculated using the following formula:

[0079]

[0080] Among them, M(A) i ): Represents image block A i The overall complexity is the sum of the complexities of all pixels in that block and the average of those complexities.

[0081] |A i |: represents block A i The number of pixels, i.e., the total number of pixels in that block. The formula sums the complexity over all pixels and divides by |A|. i | Calculate the average complexity.

[0082] (x,y)∈A i (x,y): Represents block A i Each pixel in the array has coordinates (x, y).

[0083] G(x,y): Represents the edge gradient at pixel (x,y) in an image. The edge gradient reflects the edge information of the image by calculating the changes in the horizontal and vertical directions at that point. It is generally used to detect details and contours in an image.

[0084] C(x,y): Represents the rate of color change at pixel (x,y) in the image, reflecting the color difference between that point and its surrounding pixels.

[0085] T(x,y): Represents the texture complexity of the image at pixel (x,y), based on Local Binary Pattern (LBP) or other texture feature extraction methods, used to describe the texture structure around that point. The higher the texture complexity, the richer the details in that region.

[0086] λ1, λ2, λ3: These are weighting coefficients, corresponding to edge gradient, color variation, and texture complexity, respectively. These weights are used to adjust the relative influence of different features in complexity calculation, and their values ​​depend on the specific application scenario.

[0087] The weighting coefficient λ1 ranges from 0.2 to 0.5 and is used to detect image edges and contour information to ensure edge clarity and improve visual effects, especially in high-contrast areas, while avoiding its excessive proportion in complexity calculations.

[0088] The value range of λ2 color change rate is 0.3-0.6. In order to accurately reflect color information and avoid excessive weighting that would lead to overly complex calculations, the value is set to a medium range to ensure that the details of color changes can be properly captured.

[0089] The texture complexity λ3 ranges from 0.1 to 0.4. Texture complexity is calculated using methods such as local binary patterning to capture the detailed features of the image. While texture information is crucial for some highly detailed areas, its effect is relatively weak in ordinary image areas. Therefore, the weight of texture complexity should be low to avoid impacting the overall computational efficiency of rendering.

[0090] G(x,y) represents the edge gradient at pixel (x,y), and its calculation formula is as follows:

[0091]

[0092] Where I(x,y) represents the brightness value (grayscale value or color intensity) of the image at pixel (x,y).

[0093] and These represent the gradients (brightness changes) of the image in the x and y directions, respectively.

[0094] C(x,y): Represents the color change rate, calculated using the following formula:

[0095]

[0096] Where C(x,y) represents the color change rate at pixel (x,y) in the image. It reflects the color difference between this pixel and its surrounding pixels in the RGB color space. The greater the color change rate, the more complex the color transition in that area.

[0097] This represents the rate of change of the red channel (R channel) in the x-direction. It is calculated by comparing the red intensity changes of a pixel (x, y) with those of its neighboring pixels. This parameter reflects the color change of that pixel in the R channel.

[0098] This represents the rate of change of the green channel (G channel) in the x-direction. It describes the change in green intensity at the point (x,y) in the image, indicating the variation of the green component at that pixel in the x-direction.

[0099] This represents the rate of change of the blue channel (B channel) in the x-direction. It is derived by comparing the intensity changes of a pixel (x, y) with those of its neighboring pixels in the blue channel, indicating the change of the blue component of that pixel in the x-direction.

[0100] T(x,y): Represents the local texture complexity, calculated using Local Binary Pattern (LBP):

[0101]

[0102] Where T(x,y) represents the texture complexity of the image at pixel (x,y). It reflects the texture characteristics of the region by comparing the gray values ​​of the pixel's neighbors with the gray value of the pixel itself. The larger the value, the more complex the texture.

[0103] In this embodiment, a 3x3 pixel window is used to sum the values ​​of the eight neighboring pixels surrounding the center pixel (x, y). This choice balances computational efficiency and detail capture in image processing. Compared to a larger 5x5 window, a 3x3 window significantly reduces computation while providing sufficient local information to ensure no loss of detail; reducing the number of neighboring pixels to four may result in the loss of edge and texture details.

[0104] I p (x,y): Represents the grayscale value of the p-th neighboring pixel at position (x,y). Grayscale values ​​are typically used to represent the brightness of a pixel in an image, and usually range from 0 (black) to 255 (white).

[0105] I(x,y): Represents the grayscale value of the center pixel at position (x,y).

[0106] I(·): is an indicator function, defined as follows: In this formula, the indicator function is used to compare the center pixel I(x,y) and the neighboring pixels I. pThe grayscale difference between (x, y) pixels. If the grayscale value of a neighboring pixel is greater than that of the center pixel, the indicator function is set to 1; otherwise, it is 0. This method can capture changes in local texture.

[0107] Step 103: Adaptive Blocking Strategy

[0108] Based on the complexity matrix M(A) generated in step 102 i Adaptive block processing is performed. This strategy uses the mean M(A) of the complexity matrix. i) Determine the level of detail in the chunks. Complex areas will be subdivided into smaller chunks, while less complex areas will use larger chunks to reduce the rendering burden.

[0109] The specific rules are as follows:

[0110]

[0111] Indicates image block A i The average complexity. It is the average complexity of all pixels within the block, used to reflect the overall complexity of the entire block.

[0112] Step 104: Patch rendering and optimization

[0113] After adaptive chunking is completed, each region is rendered hierarchically based on the complexity of each chunk. Chunks with higher complexity are rendered at higher resolution first, while chunks with lower complexity are rendered at lower resolution. This process is executed in LVGL's per-pixel rendering engine, leveraging its double-buffering mechanism to achieve efficient rendering.

[0114] Example 2

[0115] This embodiment provides a method for dynamically adjusting the cache priority of image blocks in an LVGL environment by analyzing the complexity, access frequency, and user interaction behavior of image blocks. This embodiment aims to improve the utilization efficiency of cached resources during image rendering, ensuring that high-priority image blocks are rendered first, thereby optimizing overall rendering performance.

[0116] Step 201: Calculate the priority of the block cache

[0117] First, by analyzing each block A of the image i The relevant characteristics, including its complexity C. i Access frequency F i User interaction behavior data U i The cache priority P of the blocks is determined comprehensively. i The priority calculation formula is as follows:

[0118]

[0119] in:

[0120] C i The complexity of block partitioning is given by the complexity matrix M(A) in Example 1. i The calculations show that the image details and color changes in the image area are reflected.

[0121] F i The system can accumulate access counts each time a user views the records of a particular segment. For example, suppose that within a certain time window, segment A... i If a site is accessed N times, then the access frequency Fi can be represented as the cumulative value, Fi = cumulative access count.

[0122] U i User interaction data reflects the frequency of user actions within this segment, such as clicking, scrolling, or zooming. More frequent interactions indicate higher user engagement with that segment. i =Number of clicks + Scrolling frequency + Zooming times

[0123] In this formula, the adjustment coefficient α 1、 α 2、 α3 is used to adjust the cache priority P of the blocks. i The weights of different factors, including complexity C. i Historical access frequency F i and user interaction behavior data U i The specific value range is as follows:

[0124] α 1： 0.3-0.6, α1 adjusts the complexity C of image segmentation. i This refers to the influence of features such as the edges, textures, and colors of the image blocks. Since image complexity directly impacts rendering quality, appropriate weighting ensures the system prioritizes rendering highly complex areas, but complexity shouldn't be the sole determining factor. Therefore, α1 is set in a moderate range (0.3-0.6) to balance computational burden and visual quality.

[0125] The value of α2 ranges from 0.2 to 0.5, and α2 is used to adjust the historical access frequency F. i Impact on cache priority. Historical access frequency reflects a user's long-term attention to a particular area; however, the frequency of access in each operation is not the only determining factor. Therefore, setting α2 in a moderately low range of 0.2-0.5 ensures that frequently accessed chunks are prioritized without significantly impacting the overall allocation of rendering resources.

[0126] The value of α3 ranges from 0.4 to 0.7. α3 adjusts the user interaction behavior (Ui), that is, the user's actions within a specific area (such as clicking, scrolling, zooming, etc.). Since user interaction behavior directly reflects the user's immediate attention level, this weight is particularly important in the system. Therefore, α3 is set to a relatively high value, between 0.4 and 0.7, to ensure that the area where the user is currently highly interacting can respond quickly and be prioritized, thus improving the user experience.

[0127] Step 202: Dynamic Weight Adjustment

[0128] To adapt to the real-time load of the system, this embodiment dynamically adjusts the weights to ensure that more complex blocks are cached first when cache resources are scarce. The weight coefficient α is adjusted using the Sigmoid function. 1：

[0129]

[0130] Where: R cache θ represents the current cache utilization rate; θ is the preset warning threshold for cache resources; k is the adjustment rate constant, which determines the sensitivity of cache utilization changes to priority.

[0131] With cache utilization R cache Approaching the warning line θ, the complexity is C. i The weight α1 will gradually increase to ensure that high-complexity regions can be stored in the cache first, avoiding frequent reading of high-complexity regions from external storage, which would affect rendering performance.

[0132] Step 203: Cache Tiered Management

[0133] Based on priority P i Determine the caching hierarchy for each block. In an LVGL environment, blocks are managed through a three-tier caching system:

[0134] Level 1 cache (L1): Stores high-priority blocks to ensure they can be accessed quickly. It is used for frequently accessed critical areas or areas with high complexity.

[0135] Second-level cache (L2): Stores secondary priority blocks, suitable for areas of medium complexity or low access frequency.

[0136] Level 3 cache (L3): Stores low-priority blocks, typically image regions that are rarely accessed or parts with low complexity.

[0137] Priority P iHigher-priority blocks are preferentially stored in the L1 cache, while lower-priority blocks are stored in the L2 or L3 cache. When cache space is limited, lower-priority blocks will be removed from the cache to make room for higher-priority blocks.

[0138] Step 204: Dynamic Cache Eviction Mechanism

[0139] The cached blocks are sorted according to their priority and usage frequency. When cache space is insufficient, the system will gradually evict low-priority blocks, prioritizing the retention of high-priority regions. The cache eviction strategy is as follows:

[0140] Calculate the cache score S for each block. i :

[0141]

[0142] Among them, P i It is the cache priority of the block, t i This is the dwell time of the block in the cache. Blocks with longer dwell times and lower priority will be evicted first.

[0143] When the cache utilization exceeds the set maximum limit (i.e., R) cache >>θ max The system will remove cache blocks with lower scores one by one to make room for high-priority data.

[0144] Example 3

[0145] This embodiment provides intelligent predictive scheduling based on user behavior in an LVGL environment.

[0146] Step 301: User Behavior Data Collection

[0147] First, during the LVGL rendering process, the system records user interaction data in real time. These interactions include, but are not limited to: click operations: the coordinate information of different image areas clicked by the user; scroll operations: the direction, speed, and frequency of scrolling on the image; zoom operations: the user's zooming in or out of the image.

[0148] Step 302: Based on the historical interaction behavior recorded in dataset D, a Markov chain model is used to predict the user's next operation path. Specifically, the user starts from an image block A... i Move to another block A j The frequency and direction are calculated using the following formula:

[0149]

[0150] This formula constructs a user's behavior from the current segment A using historical behavior data provided by dataset D.i To the next block A j State transition matrix T (i,j) This allows us to predict the areas a user is about to visit.

[0151] Step 303: Schedule based on cache priority

[0152] Based on the generated state transition matrix, the system further incorporates the block cache priority P j This ensures that not only the regions the user is about to access are prioritized, but also the rendering priority of that chunk is taken into account. Chunk priority P j The result from Example 3 is a comprehensive evaluation based on image complexity, access frequency, and user interaction behavior. The intelligent predictive scheduling formula is as follows:

[0153]

[0154] This formula is used to determine the image segment A that the user is most likely to access in the next step. next Based on the current state of the current block (`current`), the probability of future access, and the cache priority, this formula combines the state transition probability T(`current,j`) and the block priority P. j Select the optimal next block A. next The specific explanation is as follows:

[0155] A next This indicates the image tile that the user is likely to visit next. In the system, this tile will be preloaded or rendered in advance to improve the user experience.

[0156] argmax j : This indicates selecting the block j that maximizes the subsequent values. In other words, the system will choose the next optimal block j to maximize the result of subsequent calculations.

[0157] T (current,j) This represents the probability of a user transitioning from the current segment (current) to segment (j). This value is typically obtained through analysis of the user's historical behavior patterns, reflecting the likelihood of the user moving from the currently viewed area (current) to the target area (j). A higher probability value indicates a greater likelihood of the user moving from the current area to that segment.

[0158] P j : Indicates the cache priority of chunk j. Cache priority is determined by factors such as the historical access frequency of the chunk, user operation patterns, and chunk complexity. A higher priority chunk indicates its greater importance in the system and requires priority rendering or caching.

[0159] Step 304: Preloading and Early Rendering

[0160] Once the system determines the segment A that the user is about to access using the intelligent predictive scheduling formula... next The system will preload and render the segment before the user interacts with it. Specific steps include:

[0161] (1) Preloading: The predicted block Anext is preloaded into the LVGL double buffer to ensure a fast response when the user operates on the area.

[0162] (2) Pre-rendering: Combined with the pixel-by-pixel rendering mechanism of LVGL, high-priority blocks are rendered with high precision to ensure that the image has been rendered when the user views the block.

[0163] Step 305: Dynamically adjust the scheduling model

[0164] As user interaction continues, the system updates the state transition matrix (T(i,j)) in real time based on the latest operation trajectory, dynamically optimizing the prediction results. Specifically:

[0165] (1) State transition matrix update: When the user moves from block A i Transfer to block A j At that time, the system will update the state transition matrix based on the user's latest operation frequency:

[0166]

[0167] Historical access frequency (F) i ) and interactive behavior (U i The weight values ​​(0.6-0.8 and 0.2-0.4 respectively) are kept in harmony, highlighting both long-term operational trends and flexible responses to immediate actions, especially ensuring that the handling of complex areas is not obscured by short-term operations.

[0168] (2) Real-time scheduling adjustment: As the state transition matrix and cache priority (Pj) are updated, the system automatically adjusts the preloading and rendering plan to ensure that the scheduling strategy always matches the latest operating habits of users.

[0169] Example 4

[0170] This embodiment provides a method for accelerating image rendering through GPU parallel processing in an LVGL environment. This is achieved by distributing image chunking tasks in parallel to multiple GPU cores.

[0171] Step 401: Task Assignment

[0172] After completing the adaptive image segmentation and determining the cache priority, the system determines the size S of each segment. iThe size and priority of the chunks, Pi, are used to distribute the chunked tasks to the various computing cores of the GPU. The size and priority of the chunks directly affect the order in which they are distributed across the GPU.

[0173] The GPU task allocation formula is as follows:

[0174]

[0175] in:

[0176] W k This represents the total task load allocated to the k-th GPU core;

[0177] S i It is block A i The size of the block indicates the number of pixels it occupies;

[0178] P i It's a block-based cache priority, taking into account image complexity and access frequency.

[0179] And user interaction behavior (see the priority calculation formula in Example 1).

[0180] Step 402: Load Balancing

[0181] To ensure reasonable task allocation across the GPU cores, this embodiment employs a load balancing mechanism to dynamically adjust the workload of each core, preventing some cores from being overloaded while others are underloaded. The load balancing formula is as follows:

[0182]

[0183] When the calculated load balance exceeds the set threshold φ (φ is set in the range of 1.2-1.5), the system will automatically redistribute some tasks to ensure that the load of each GPU core is nearly balanced.

[0184] In this invention, the threshold φ is set between 1.2 and 1.5 to balance efficiency and system stability. The specific considerations are as follows: (1) Reduce task redistribution: When the load balance is between 1.2 and 1.5, each GPU core can still work efficiently, avoiding frequent task adjustments. (2) Adapt to differences in core performance: Different GPU cores have different processing capabilities for different task types. Allowing 20%-50% load imbalance helps to leverage the core's advantages. (3) Tolerate short-term fluctuations: Short-term load fluctuations within this range will not affect system stability. (4) Reduce scheduling overhead: Too low a threshold will lead to frequent redistribution, increasing overhead; the range of 1.2-1.5 ensures that scheduling only occurs when necessary, thereby improving overall efficiency.

[0185] Step 403: Parallel Rendering

[0186] After task allocation and load balancing are completed, the GPU begins to render each block in parallel. Combined with LVGL's per-pixel rendering mechanism, each GPU core processes different blocks, significantly reducing rendering time through parallel computing.

[0187] (1) High-complexity block processing: GPUs prioritize processing high-complexity, high-priority image blocks. These blocks typically involve more edges, color variations, and texture complexity, and therefore receive more computational resources.

[0188] (2) Low-complexity block processing: Low-complexity image blocks are simplified by other cores of the GPU to reduce GPU computing time and resource consumption.

[0189] Step 404: Synchronize GPU and LVGL rendering

[0190] After the GPU completes the tile rendering, the system uses LVGL's double buffering mechanism to synchronize the results to the display, avoiding latency and preventing screen tearing. The double buffering mechanism allows rendering to be completed in the background and switched to the foreground display all at once. High-priority tiles can be presented immediately during user interaction, greatly reducing waiting time and ensuring a smooth experience.

[0191] Step 405: Performance Optimization

[0192] To further improve rendering performance, this embodiment introduces a dynamic performance optimization mechanism, which mainly includes the following two aspects:

[0193] (1) Dynamic resolution adjustment: When the GPU load is high, the system will dynamically adjust the resolution of low-priority blocks according to their priority to reduce the computational burden. For example, low-priority blocks will be rendered at a lower resolution to ensure the balance of the overall rendering progress.

[0194] (2) Resource reclamation: After the block rendering is completed, the GPU will actively release unused computing resources, reduce memory usage, and make room for new tasks.

[0195] Example 5

[0196] This embodiment provides a method for dynamic image compression and format conversion in an LVGL environment.

[0197] Step 501: Compression Ratio Adjustment

[0198] When rendering images, the system first determines the current device performance status (such as CPU and GPU load) and the cache priority of the blocks (P). i Dynamically adjust the compression ratio of each image block.

[0199] The formula for calculating compression ratio is as follows:

[0200]

[0201] Among them: Q i It is a block (A) i The compression mass of C; max It is the device's maximum computing power; L current This indicates the current system load (such as CPU or GPU utilization); P i It is the cache priority of the block.

[0202] Step 502: Format Conversion

[0203] The formula for format conversion is as follows:

[0204]

[0205] This formula is used to determine image chunking (A). i The best format for Format(A) i This formula selects the image format that achieves the best balance between file size and quality. It combines the file size to image quality ratio of different image formats (such as WebP, JPEG, and PNG) to choose the optimal format for compression and storage. Specifically:

[0206] Format(A i ): Represented as a block (A) i The system will compare the selected image format (such as WebP, JPEG, PNG) and select the format that best suits the current image segment.

[0207] argmin: Indicates which format will minimize the value of the subsequent expression. In other words, the system will choose the image format f that minimizes the file size to image quality ratio.

[0208] f∈{WebP, JPEG, PNG}: Represents the set of image formats available, from which the system will select.

[0209] This represents the ratio of file size to image quality for image format f.

[0210] Step 503: Dynamically adjust compression parameters

[0211] During actual operation, the system continuously monitors the device load and image rendering requirements, dynamically adjusting the compression parameters for each block. Under high load, the system reduces the rendering precision of low-priority blocks and automatically increases the compression ratio to reduce rendering pressure.

[0212] Step 504: Resource recycling and format conversion optimization

[0213] After image chunk rendering is complete, the system will automatically reclaim the image resources used during rendering and further optimize storage through format conversion. Low-priority chunks will be compressed into efficient formats (such as WebP) after rendering, reducing storage space usage.

[0214] Step 505: Submit and render the compression results

[0215] Finally, the image chunks, after dynamic compression and format conversion, are submitted to the LVGL rendering engine. LVGL is responsible for rendering these chunks to the display buffer and adjusting the display resolution and rendering details according to the device's load.

[0216] (1) High-priority block display: For high-priority blocks, LVGL will display them at full resolution to ensure that details and image quality are perfectly displayed.

[0217] (2) Simplified display of low priority blocks: For low priority blocks, LVGL will simplify the display details based on the compression quality and format conversion results to reduce rendering pressure.

[0218] Example 6

[0219] Smart home control panels typically need to render complex images such as room layout diagrams and device status icons, and handle frequent user clicks, scrolling, and zooming operations. On resource-constrained embedded devices, such as those with limited processors and memory, this invention improves image rendering efficiency through a series of optimization steps and parameter adjustments, ensuring smooth user operation.

[0220] Step 1: Determine the priority of block cache

[0221] The system first calculates the cache priority for each image block. The cache priority is determined based on a combination of the following parameters:

[0222] (C i (Image complexity): Reflects the edge, texture, and color variations of image segments. For example, the device icon area in a room layout diagram may have higher complexity, while the background area may have lower complexity.

[0223] (F i (Historical Access Frequency): Sections that users frequently access in the past, such as light control icons or air conditioner icons that users have clicked multiple times. These areas have a high access frequency.

[0224] (U i)(Current User Interaction): Reflects the frequency of the user's real-time interaction with a specific segment during the current operation. For example, if the user is clicking or dragging a device icon, that icon's priority will be increased accordingly.

[0225] Weighting coefficients: By adjusting α1, α2 and α3, the impact of image complexity, historical access frequency and current interaction behavior on cache priority is controlled.

[0226] Step 2: Adaptive Block Subdivision

[0227] For highly complex image regions, the system further subdivides the blocks to ensure that complex areas (such as device icons and text areas) can be rendered with precision. The following parameters determine this process:

[0228] Threshold τ: The system sets a complexity threshold. When the average complexity of a block exceeds this threshold, the system automatically subdivides that block into smaller blocks. Higher thresholds are suitable for simple image scenarios, while lower thresholds are used to ensure that areas with rich detail can be processed more finely.

[0229] By adjusting τ, the system can find a balance between high-detail and low-complexity areas, saving rendering resources and ensuring accurate rendering of the details that users care about.

[0230] Step 3: Intelligent Predictive Scheduling

[0231] Based on the user's historical actions, the system predicts the user's next action and pre-schedules the rendering of potentially accessed chunks. Key parameters include:

[0232] T(current,j) (state transition probability): Represents the probability that a user will transition from the current block to another block. This probability is calculated based on the user's past operation records.

[0233] Cache priority (P) j ): Combining user operation prediction and cache priority, the system selects the most likely accessed chunks and renders them in advance.

[0234] On smart home control panels, users may frequently interact with icons for lights, air conditioning, and other functions. By analyzing users' common operating patterns, the system predicts the areas the user is about to click and pre-loads them, thereby improving response speed and reducing operation latency.

[0235] Step 4: GPU Parallel Acceleration Processing

[0236] To improve rendering performance, the system distributes a large number of image chunking tasks to multiple GPU cores for parallel processing. Parameters affecting parallel task allocation include:

[0237] (S i (Track size): The number of pixels in a block. Complex image regions often contain more pixels, requiring more GPU computing resources.

[0238] Cache priority (P) i High-priority chunks will be assigned to the GPU for priority rendering, ensuring faster rendering speeds for areas with frequent user interaction or complex terrain.

[0239] The GPU cores are assigned tasks based on the size and priority of each block. The system dynamically adjusts the allocation to maintain a balanced GPU load and prevent one core from being overloaded while another is idle.

[0240] Step 5: Dynamic Compression and Format Conversion

[0241] Based on the current system load and device status, the system dynamically adjusts the image compression rate and format to optimize memory usage and processing speed. Key parameters include:

[0242] (Lcurrent) (Current System Load): When the system load is high, the image will use a higher compression rate to reduce memory usage and computational burden. Image Complexity: High-complexity image areas will maintain a lower compression rate to ensure rendering quality, while lower-complexity areas can use a higher compression rate.

[0243] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of this invention patent should be determined by the appended claims.

Claims

1. A method for accelerating image rendering under LVGL, characterized in that, Includes the following steps: Step 1: Based on the complexity, texture, and color variations of each region of the image, adaptively divide the image into blocks; Step 2: Analyze the complexity, access frequency, and user interaction behavior of image chunks, and adjust the chunk caching priority accordingly; Step 3: Perform intelligent prediction and scheduling based on user behavior, and preload the image blocks that the user is about to access; Step 4: Using GPU parallel acceleration, the image chunking tasks predicted and scheduled in Step 3 are allocated to GPU cores; Step 5: Adjust the image compression ratio and format based on the priority of image segmentation, device performance, and current system load; Step 1 specifically involves the following steps: Step 101: Load the image to be rendered into the LVGL rendering system and process the image using pixel-by-pixel rendering and double buffering mechanisms; Step 102: After completing the LVGL rendering compatibility initialization, perform complexity analysis on each pixel region of the image and generate a complexity matrix; Step 103: Perform adaptive block partitioning based on the complexity matrix generated in Step 102; Step 104: After the adaptive block processing is completed, perform hierarchical rendering of each region according to the complexity of each block; The complexity matrix is ​​calculated using the following formula: Where M(Ai) represents the summation and average of the complexity of all pixels in the block; |Ai| represents the number of pixels in block Ai; (x,y)∈Ai(x,y) represents each pixel in block Ai, with coordinates (x,y); G(x,y) represents the edge gradient of the image at pixel (x,y); C(x,y) represents the rate of color change of the image at pixel (x,y); T(x,y) represents the texture complexity of the image at pixel (x,y); λ1,λ 2, λ3: These are the weighting coefficients, corresponding to edge gradient, color change, and texture complexity, respectively. The weighting coefficient λ1 ranges from 0.2 to 0.5 and is used to detect image edges and contour information to ensure edge clarity and improve visual effects. The value of the λ2 color change rate ranges from 0.3 to 0.6 to ensure that the details of color changes can be captured properly; The texture complexity λ3 ranges from 0.1 to 0.

4. Texture complexity is calculated using methods such as local binary pattern recognition to capture the detailed features of the image. Step 2 specifically employs the following steps: Step 201: Determine the caching priority of each block by analyzing the complexity, access frequency, and user interaction data of each block of the image; Step 202: Adjust the system's processing order of blocks based on the block cache priority; Step 203: Based on block priority, divide the blocks into different levels of the cache system for management; Step 204: Calculate and score the blocks based on their priority and dwell time, and eliminate low-priority blocks; Step 201: Calculation of block cache priority First, by analyzing each block A of the image i The relevant characteristics, including its complexity C. i Access frequency F i User interaction behavior data U i The cache priority P of the blocks is determined comprehensively. i The priority calculation formula is as follows: in: C i The complexity of the blocks reflects the image details and color changes in the image region; F i The system can accumulate access counts each time a user views the records for that segment; U i User interaction behavior data reflects the frequency of user operations in this segmented area; In this formula, the adjustment coefficients α1, α2, and α3 are used to adjust the cache priority P of the blocks, respectively. i The weights of different factors, including complexity C. i Historical access frequency F i and user interaction behavior data U i The specific value range is as follows: α 1： 0.3-0.6, α1 adjusts the complexity C of image segmentation. i That is, the influence of features such as the edges, texture, and color of the blocks; The value of α2 ranges from 0.2 to 0.5, and α2 is used to adjust the historical access frequency F. i Impact on cache priority; The value range of α3 is (0.4-0.7), and α3 adjusts the user interaction behavior Ui; in step 3, the following steps are specifically adopted: Step 301: Collect user action data such as clicks, scrolling, and zooming, record action trajectory information, and generate a user interaction behavior dataset; Step 302: Using the behavioral data provided by dataset D, construct the user's path from the current block to the next block A. j The state transition matrix; Step 303: Combine the state transition matrix with the block cache priority to schedule the image blocks; Step 304: Based on the scheduling results in step 303, the chunks that the user is about to access are preloaded and rendered in advance; Step 305: Update the scheduling model and adjust the state transition matrix based on the user's operation trajectory; Step 303: Schedule based on cache priority Based on the generated state transition matrix, the system further incorporates the block cache priority P j This ensures that not only the areas users are about to access are prioritized for scheduling, but the intelligent predictive scheduling formula is as follows: A next : Indicates the image blocks that the user is likely to visit next; argmax j : indicates taking the block j that makes the subsequent values ​​reach the maximum value; T (current,j) : Indicates the probability of a user transitioning from the current block to block j; P j : Indicates the cache priority of block j; Step 4 is specifically performed using the following steps: Step 401: Allocate the block tasks to the various computing cores of the GPU according to the size of each block and the cache priority; Step 402: Adjust the workload of each core through a load balancing mechanism; Step 403: After task allocation and load balancing are completed, the GPU is used to render each block in parallel; Step 404: Synchronize the rendering results using LVGL's double buffering mechanism; Step 405: Adjust the block resolution and release unused computing resources; Step 401 includes task allocation. The GPU task allocation formula is as follows: in: W k This represents the total task load allocated to the k-th GPU core; S i It is block A i The size of the block indicates the number of pixels it occupies. P i It is a block-based cache priority that combines image complexity, access frequency, and user interaction behavior.

2. The method for accelerating image rendering under LVGL according to claim 1, characterized in that, Step 5 is carried out using the following steps: Step 501: Adjust the compression rate of each image block according to the device's performance status and the block's cache priority; Step 502: Select a suitable image format based on the compression quality and image type of each block; Step 503: Adjust the compression parameters of each block by monitoring the device load and image rendering requirements; Step 504: Reclaim the image resources used in the rendering process and optimize storage through format conversion; Step 505: Submit the image chunk data, which has undergone dynamic compression and format conversion, to the LVGL rendering engine.

Images