Method for accelerating picture rendering under lvgl

Abstract

The invention provides a method for accelerating picture rendering in an LVGL environment. According to the method, the performance of picture rendering is improved through adaptive blocking processing, cache priority adjustment, intelligent prediction scheduling and GPU parallel acceleration. The method comprises the following specific steps: firstly, adaptively partitioning a picture, and dynamically adjusting a cache priority according to the complexity, access frequency and user interaction behavior of the picture; intelligent prediction scheduling is carried out based on historical operation data of a user, and blocks to be accessed are pre-loaded and rendered in advance; high-priority blocks are processed in parallel and accelerated through a GPU, and the compression ratio and format of the picture are adjusted in combination with the equipment performance and the current system load. Through the method, the picture rendering delay can be effectively reduced, the system response speed is improved, and the method is suitable for embedded equipment and a resource-limited system environment.

Description

Technical Field

[0001] The present invention relates to the field of information processing technology, and in particular to a method for accelerating image rendering under LVGL. Background Art

[0002] The popularity of embedded and IoT devices has driven the growth of demand for graphical user interface (GUI) systems, but these devices are often limited by processing power and memory resources. When using lightweight graphics libraries such as LVGL (Light and Versatile Graphics Library), traditional rendering methods are prone to performance bottlenecks. Existing technologies mainly reduce the computational burden through simple caching, fixed blocking or reducing image quality. However, these methods lack the ability to adjust user interaction behavior, image complexity and system load in real time, resulting in insufficient rendering optimization and inability to flexibly respond to changes in system load and user needs.

[0003] In scenarios with high-resolution images or complex interactive operations, this deficiency causes system response lag, especially when users frequently scroll, zoom, or click on images, rendering delays are obvious, thus affecting the overall user experience. Summary of the invention

[0004] The embodiment of the present invention provides a method for accelerating image rendering under LVGL to solve the above technical problems.

[0005] The present invention provides a method for accelerating image rendering under LVGL, comprising the following steps:

[0006] Step 1: Adaptively divide the image into blocks according to the complexity, texture and color changes of each area of ​​the image;

[0007] Step 2: Analyze the complexity, access frequency, and user interaction behavior of the image blocks, and adjust the block cache priority;

[0008] Step 3: Perform intelligent prediction and scheduling based on user behavior to load the image blocks that the user is about to access in advance;

[0009] Step 4: Through GPU parallel acceleration processing, the image segmentation tasks predicted and scheduled in step 3 are assigned to the GPU core;

[0010] Step 5: Adjust the image compression rate and format based on the image segmentation priority, device performance, and current system load.

[0011] Specifically, in step 1, the following steps are specifically adopted:

[0012] Step 101: Load the image to be rendered into the LVGL rendering system, and process the image using pixel-by-pixel rendering and double buffering mechanism;

[0013] Step 102: After completing the LVGL rendering compatibility initialization, perform complexity analysis on each pixel area of ​​the image to generate a complexity matrix;

[0014] Step 103: performing adaptive block processing according to the complexity matrix generated in step 102;

[0015] Step 104: After the adaptive block processing is completed, each area is rendered in a graded manner according to the complexity of each block.

[0016] Specifically, in step 2, the following steps are specifically adopted:

[0017] Step 201: determining the cache priority of each block by analyzing the complexity, access frequency and user interaction behavior data of each block of the image;

[0018] Step 202: Based on the cache priority of the blocks, adjust the processing order of the blocks by the system;

[0019] Step 203: Based on the block priorities, the blocks are divided into different levels of cache systems for management;

[0020] Step 204: Calculate and score the blocks according to the block priority and residence time, and eliminate low-priority blocks.

[0021] Specifically, in step 3, the following steps are specifically adopted:

[0022] Step 301: Collect user's click, scroll, zoom and other operation behavior data, record operation track information, and generate user interaction behavior data set;

[0023] Step 302: Use the behavior data provided by dataset D to construct a user's progress from the current block to the next block A. j The state transition matrix of

[0024] Step 303: Scheduling the image blocks based on the state transfer matrix and the block cache priority;

[0025] Step 304: According to the scheduling result in step 303, the block that the user is about to access is preloaded and rendered in advance;

[0026] Step 305: Update the scheduling model and adjust the state transfer matrix according to the user's operation trajectory.

[0027] Specifically, in step 4, the following steps are specifically adopted:

[0028] Step 401: Allocate the block tasks to the various computing cores of the GPU according to the size and cache priority of each block;

[0029] Step 402: adjusting the workload of each core through a load balancing mechanism;

[0030] Step 403: After task allocation and load balancing are completed, each block is rendered in parallel through the GPU;

[0031] Step 404: display the rendering result synchronously through the double buffer mechanism of LVGL;

[0032] Step 405: Adjust the block resolution and release unused computing resources.

[0033] Specifically, in step 5, the following steps are specifically adopted:

[0034] Step 501: adjusting the compression rate of each image block according to the performance status of the device and the cache priority of the block;

[0035] Step 502: Select a suitable picture format according to the compression quality and picture type of each block;

[0036] Step 503: adjusting the compression parameters of each block by monitoring the load of the device and the rendering requirements of the image;

[0037] Step 504: Recycle the image resources used in the rendering process and optimize storage through format conversion;

[0038] Step 505: Submit the image block data after dynamic compression and format conversion processing to the LVGL rendering engine.

[0039] Specifically, the complexity matrix is ​​calculated by the following formula:

[0040]

[0041] Among them, M(Ai) represents the sum of the complexity of all pixels in the block and the average value;

[0042] |Ai| represents the number of pixels of block Ai;

[0043] (x,y)∈Ai(x,y) represents each pixel in block Ai, whose coordinates are (x,y);

[0044] G(x,y) represents the edge gradient of the image at the pixel point (x,y);

[0045] C(x,y) represents the color change rate of the image at the pixel point (x,y);

[0046] T(x,y) represents the texture complexity of the image at the pixel point (x,y);

[0047] λ1,λ 2, λ3: weight coefficient, corresponding to edge gradient, color change and texture complexity respectively.

[0048] The beneficial effects of the present invention are:

[0049] 1. Improve rendering speed: By adaptively dividing the image into blocks and combining the complexity of the image blocks, historical access frequency and user interaction behavior, the system can effectively adjust the rendering priority of each block to ensure that high-complexity and high-interaction areas are rendered first, thereby reducing the delay in image rendering and improving the overall response speed.

[0050] 2. Optimize resource utilization: Through cache priority management and GPU parallel acceleration processing, system resources are effectively allocated. When cache resources are limited, low-priority blocks will be dynamically eliminated to make room for high-priority areas, ensuring the rational allocation and utilization of resources.

[0051] 3. Improve user experience: The intelligent prediction scheduling mechanism preloads the blocks that the user is about to access in advance based on the user's historical operation behavior, avoiding the delay between user operation and image rendering, and improving the smoothness of interface operation.

[0052] 4. Reduce system load: Through dynamic compression and format conversion, the system can dynamically adjust the compression rate of the image according to the current device performance and system load, reduce the rendering requirements of low-priority blocks, thereby reducing the system load and maintaining the stability of system operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0054] Figure 1 A schematic diagram of the structure of the method for accelerating image rendering under LVGL provided by the present invention;

[0055] Figure 2 This is a schematic diagram of the structure of Embodiment 1 of the present invention;

[0056] Figure 3 This is a schematic diagram of the structure of Embodiment 2 of the present invention;

[0057] Figure 4 This is a schematic diagram of the structure of Embodiment 3 of the present invention;

[0058] Figure 5This is a schematic diagram of the structure of Embodiment 4 of the present invention;

[0059] Figure 6 This is a schematic diagram of the structure of Embodiment 5 of the present invention; DETAILED DESCRIPTION

[0060] In order to facilitate the understanding of the present invention, the present invention will be described more fully below with reference to the relevant drawings. Preferred embodiments of the present invention are provided in the drawings. However, the present invention can be implemented in many different forms and is not limited to the embodiments described herein. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present invention more thorough and comprehensive.

[0061] Embodiment 1

[0062] This embodiment provides a method for adaptively partitioning an image in an LVGL environment, which aims to realize dynamic partitioning for different regions by analyzing the complexity of each region of the image, so as to optimize the rendering efficiency of the image and reduce the amount of rendering calculation.

[0063] This embodiment provides a method for accelerating image rendering under LVGL, including the following steps:

[0064] Step 1: Adaptively divide the image into blocks according to the complexity, texture and color changes of each area of ​​the image;

[0065] Step 2: Analyze the complexity, access frequency, and user interaction behavior of the image blocks, and adjust the block cache priority;

[0066] Step 3: Perform intelligent prediction and scheduling based on user behavior to load the image blocks that the user is about to access in advance;

[0067] Step 4: Through GPU parallel acceleration processing, the image segmentation tasks predicted and scheduled in step 3 are assigned to the GPU core;

[0068] Step 5: Adjust the image compression rate and format based on the image segmentation priority, device performance, and current system load.

[0069] In step 1, the following steps are specifically performed:

[0070] Step 101: LVGL rendering compatibility and initialization

[0071] This step aims to ensure that the image rendering is carried out smoothly in the LVGL environment, optimize the rendering process, and improve the overall rendering efficiency. It is divided into the following two parts:

[0072] (1) Initialize the LVGL renderer:

[0073] Before rendering the image, the LVGL renderer is initialized first to ensure that the system rendering framework is ready. At this point, the image will be loaded into the LVGL cache so that the system can start rendering. In the initial state, the image is loaded into the system in full image rendering mode.

[0074] (2) Pixel-by-pixel rendering framework:

[0075] The rendering mechanism of LVGL is based on pixel-by-pixel processing, that is, the system processes each pixel in the image one by one to render a complete image. In this embodiment, this mechanism is optimized to divide the image into multiple regions or blocks to adapt to the complexity of each block. This allows more computing resources to be obtained for complex areas, reducing the computing burden of simple areas, thereby speeding up the overall image rendering speed.

[0076] Step 102: Calculation of complexity matrix

[0077] After completing the LVGL rendering compatibility initialization, perform complexity analysis on each pixel area of ​​the image to generate a complexity matrix M(A i ), which is used to represent the complexity of image partitioning and provide a basis for adaptive partitioning.

[0078] The complexity matrix is ​​calculated by the following formula:

[0079]

[0080] Among them, M(A i ): indicates image block A i The comprehensive complexity of the block is the sum of the complexity of all pixels in the block and the average value.

[0081] ∣A i ∣: indicates block A i The number of pixels is the total number of pixels in the block. The formula is obtained by summing the complexity of all pixels and dividing by |A i ∣Calculate the average complexity.

[0082] (x,y)∈A i (x,y): represents block A i Each pixel in has coordinates (x, y).

[0083] G(x,y): represents the edge gradient of the image at the pixel point (x,y). The edge gradient reflects the edge information of the image by calculating the horizontal and vertical changes of the point. It is generally used to detect details and contours in the image.

[0084] C(x,y): represents the color change rate of the image at the pixel point (x,y), reflecting the color difference between this point and the surrounding pixels.

[0085] T(x,y): represents the texture complexity of the image at the pixel point (x,y), which is used to describe the texture structure around the point based on the local binary pattern (LBP) or other texture feature extraction methods. The higher the texture complexity, the richer the details of the area.

[0086] λ1,λ2,λ3: weight coefficients corresponding to edge gradient, color change and texture complexity respectively. These weights are used to adjust the relative influence of different features in complexity calculation, and the value depends on the requirements of specific application scenarios.

[0087] The weight coefficient λ1 ranges from 0.2 to 0.5 and is used to detect image edge and contour information to ensure edge clarity and improve visual effects, especially in high-contrast areas, while avoiding excessive proportion in complexity calculations.

[0088] The value range of the λ2 color change rate is 0.3-0.6. In order to accurately reflect the color information and avoid excessive weight that leads to overly complex calculations, the value is set to a medium range to ensure that the details of the color changes can be properly captured.

[0089] The value range of λ3 texture complexity is 0.1-0.4. Texture complexity is calculated by methods such as local binary patterns to capture the detailed features of the image. Although texture information is crucial for some high-detail areas, its role is relatively weak in ordinary image areas. Therefore, the weight of texture complexity should be low to avoid affecting the computational efficiency of the entire rendering.

[0090] G(x,y) represents the edge gradient at the pixel point (x,y), and its calculation formula is as follows:

[0091]

[0092] Among them, I(x,y) represents the brightness value (grayscale value or color intensity) of the image at the pixel point (x,y).

[0093] and Represents the gradient (brightness change) of the image in the x direction and y direction respectively.

[0094] C(x,y): represents the color change rate, and the calculation formula is as follows:

[0095]

[0096] Among them, C(x,y): represents the color change rate of the image at the pixel point (x,y). It reflects the color difference between the pixel point and the surrounding pixels in the RGB color space. The larger the color change rate, the more complex the color transition in the area.

[0097] Indicates the rate of change of the red channel (R channel) in the x direction. It is calculated by comparing the red intensity change of the pixel (x, y) and its neighboring pixels. This parameter reflects the color change of the pixel on the R channel.

[0098] Indicates the rate of change of the green channel (G channel) in the x direction. It describes the change of the green intensity of the image at the (x,y) point, indicating the change of the green component of the pixel in the x direction.

[0099] Indicates the rate of change of the blue channel (B channel) in the x direction. It is obtained by comparing the intensity changes of the pixel (x, y) and its neighboring pixels on the blue channel, indicating the change of the blue component of the pixel in the x direction.

[0100] T(x,y): represents the local texture complexity, calculated by the local binary pattern (LBP):

[0101]

[0102] Among them, T(x,y): represents the texture complexity of the image at the pixel point (x,y. By comparing the grayscale values ​​of the neighbors around the pixel point with the grayscale value of the pixel point, the texture characteristics of the area are reflected. The larger the value, the more complex the texture.

[0103] In this embodiment, a 3x3 pixel window is used to sum the 8 neighboring pixels around the central pixel (x, y). This choice balances computational efficiency and detail capture in the field of image processing. Compared with a larger 5x5 window, a 3x3 window can significantly reduce the amount of computation while providing enough local information to ensure that details are not lost; if it is reduced to 4 neighboring pixels, edge and texture details are easily missed.

[0104] I p (x,y): represents the gray value of the pth neighbor pixel at position (x,y). Gray values ​​are usually used to represent the brightness of a pixel in an image, and usually range from 0 (black) to 255 (white).

[0105] I(x,y): represents the gray value of the center pixel at the (x,y) position.

[0106] I(·): is the indicator function, which is defined as: In this formula, the indicator function is used to compare the center pixel I(x, y) and the neighboring pixel I pThe grayscale difference between (x, y). If the grayscale value of the neighboring pixel is larger than that of the central pixel, the indicator function takes the value of 1; otherwise, it takes the value of 0. This method can capture the changes in local texture.

[0107] Step 103: Adaptive Blocking Strategy

[0108] According to the complexity matrix M(A) generated in step 102 i ), and adaptive block processing is performed. This strategy uses the complexity matrix mean M(A i) Determines the level of refinement of the tiles. Areas with high complexity will be subdivided into smaller tiles, while areas with low complexity will use larger tiles to reduce the rendering burden.

[0109] The specific rules are as follows:

[0110]

[0111] Indicates image block A i It is the average complexity of all pixels in the block, which is used to reflect the overall complexity of the entire block.

[0112] Step 104: Block rendering and optimization

[0113] After the adaptive tile processing is completed, each area is rendered in a graded manner according to the complexity of each tile. Tiles with higher complexity are rendered at a higher resolution first, while tiles with lower complexity are rendered at a lower resolution. This process is performed in the LVGL pixel-by-pixel rendering engine, which uses its double buffering mechanism to achieve efficient rendering.

[0114] Embodiment 2

[0115] This embodiment provides a method for dynamically adjusting the cache priority of image blocks by analyzing the complexity, access frequency and user interaction behavior of image blocks in an LVGL environment. This embodiment aims to improve the utilization efficiency of cache resources during image rendering, ensure that high-priority image blocks can be rendered first, and thus optimize the overall rendering performance.

[0116] Step 201: Block cache priority calculation

[0117] First, by analyzing each block A of the image i The relevant properties of , including its complexity C i , access frequency F i and user interaction behavior data U i , comprehensively determine the cache priority P of the block i The priority calculation formula is as follows:

[0118]

[0119] in:

[0120] C i : The complexity of the block is obtained by the complexity matrix M(A) in the first embodiment. i ) is calculated to reflect the image details and color changes in the picture area.

[0121] F i :The system can accumulate the number of visits by each time a user views the record of the block. For example, suppose that within a certain time window, block A i If it has been visited N times, the visit frequency Fi can be expressed as the cumulative value, Fi = the cumulative value of the number of visits.

[0122] U i :User interaction behavior data, reflecting the frequency of user operations in the block area, such as click, scroll or zoom behavior. The more frequent the interaction, the higher the user attention to the block. i = Number of clicks + Scrolling frequency + Zoom times

[0123] In this formula, the adjustment coefficient α 1、 α 2、 α3 is used to adjust the cache priority P of the blocks. i The weights of different factors in i , historical access frequency F i , and user interaction behavior data U i The specific value range is as follows:

[0124] α 1： 0.3-0.6, α1 adjusts the complexity C of the image block i , that is, the influence of the edge, texture, color and other features of the block. Since the complexity of the image has a direct impact on the rendering quality, appropriate weights can ensure that the system gives priority to rendering when processing high-complexity areas, but at the same time complexity cannot be the only decisive factor. Therefore, the value of α1 is set in a medium range (0.3-0.6) to balance the computational burden and visual quality.

[0125] α2 value range (0.2-0.5), α2 is used to adjust the historical access frequency F i Impact on cache priority. The historical access frequency reflects the user's long-term attention to a certain area. However, the frequency of user access in each operation is not the only determining factor. Therefore, setting α2 in the medium-low range of 0.2-0.5 can ensure that frequently accessed blocks are given priority, but will not have a significant impact on the allocation of the overall rendering resources of the system.

[0126] α3 value range: (0.4-0.7), α3 adjusts the user interaction behavior Ui, that is, the user's operation behavior in a certain block area (such as clicking, scrolling, zooming, etc.). Since user interaction behavior can directly reflect the user's immediate attention, this weight is particularly important in the system. Therefore, the value of α3 is relatively high, set to 0.4-0.7, to ensure that the area where the user is currently highly interactive can respond quickly and be processed first, thereby improving the user experience.

[0127] Step 202: Dynamic weight adjustment

[0128] In order to adapt to the real-time load of the system, this embodiment dynamically adjusts the weight to ensure that when cache resources are tight, blocks with high complexity are cached first. Use Sigmoid function to adjust the weight coefficient α 1：

[0129]

[0130] Where: R cache represents the current cache occupancy rate; θ is the preset warning line of the cache resource; k is the adjustment rate constant, which determines the sensitivity of cache occupancy changes to priority.

[0131] As the cache usage rate R cache Approaching the warning line θ, complexity C i The weight α1 will gradually increase to ensure that high-complexity areas can be stored in the cache first, avoiding high-complexity areas from being frequently read from external storage, affecting rendering performance.

[0132] Step 203: Cache hierarchical management

[0133] Based on the priority P i Determine the cache level for each tile. In the LVGL environment, tiles are managed through a three-layer cache system:

[0134] Level 1 cache (L1): stores high-priority blocks to ensure that they can be accessed quickly. It is used for frequently accessed critical areas or areas with high complexity.

[0135] Level 2 cache (L2): stores lower priority blocks and is suitable for areas of medium complexity or less frequent access.

[0136] Level 3 cache (L3): stores low-priority tiles, usually rarely accessed areas of the image or parts of lower complexity.

[0137] Priority P iHigher priority blocks are stored in L1 cache first, while lower priority blocks are stored in L2 or L3 cache. When cache space is limited, lower priority blocks will be removed from the cache to make room for higher priority blocks.

[0138] Step 204: Dynamic cache elimination mechanism

[0139] The blocks in the cache are sorted according to their priority and frequency of use. When the cache space is insufficient, the system will gradually eliminate low-priority blocks and give priority to retaining high-priority areas. The cache elimination strategy is as follows:

[0140] Calculate the cache score S for each block i :

[0141]

[0142] Among them, P i is the cache priority of the block, t i is the time that the block resides in the cache. Blocks with longer residency times and lower priorities will be eliminated first.

[0143] When the cache usage exceeds the maximum limit (R cache >>θ max ), the system will remove chunks with lower cache scores one by one to make room for high-priority data.

[0144] Embodiment 3

[0145] This embodiment provides intelligent predictive scheduling based on user behavior in an LVGL environment.

[0146] Step 301: User behavior data collection

[0147] First, during the LVGL rendering process, the system records the user's operation behavior data in real time. These behaviors include but are not limited to: click operation: the coordinate information of the user clicking on different image areas; scroll operation: the user's scrolling direction, speed and frequency on the image; zoom operation: the user's behavior of zooming in or out on the image.

[0148] Step 302: Based on the historical interaction behaviors recorded in the dataset D, a Markov chain model is used to predict the user's next operation path. Specifically, the user starts from a picture block A. i Move to another block A j The frequency and direction are calculated using the following formula:

[0149]

[0150] This formula constructs the user's current block A based on the historical behavior data provided by the dataset D.i To the next block A j The state transition matrix T (i,j) , thereby predicting the area that the user is about to visit.

[0151] Step 303: Scheduling based on cache priority

[0152] Based on the generated state transition matrix, the system further combines the block cache priority P j , to ensure that not only the area that the user is about to visit is scheduled first, but also the rendering priority of the tile is considered. j From Example 3, it is a comprehensive evaluation result based on image complexity, access frequency and user interaction behavior. Among them, the intelligent prediction scheduling formula is as follows:

[0153]

[0154] This formula is used to determine the image block A that the user is most likely to access in the next operation. next , based on the state of the current block current and the probability of future access and cache priority. This formula combines the state transition probability T(current,j) and the block priority P j , select the best next block A next , the specific explanation is as follows:

[0155] A next : Indicates the image block that is predicted to be accessed by the user next. In the system, this block will be preloaded or rendered in advance to improve the user experience.

[0156] argmax j : Indicates taking the block j that makes the subsequent value reach the maximum value. In other words, the system will select the next optimal block j so that the result of subsequent calculation is the largest.

[0157] T (current,j) : Indicates the state transition probability of the user from the current block current to the block j. This value is usually obtained by analyzing the user's historical behavior pattern, reflecting the possibility of the user transferring from the currently viewed area current\text{current}current to the target area j. The higher the probability value, the greater the possibility of the user transferring from the current area to the block.

[0158] P j : represents the cache priority of block j. The cache priority is determined by factors such as the historical access frequency of the block, user operation mode, and block complexity. The higher the priority of the block, the more important it is in the system, and it needs to be rendered or cached first.

[0159] Step 304: Preloading and early rendering

[0160] Once the system determines the block A that the user is about to access through the intelligent prediction scheduling formula next , the system will preload the block before the user operates it and render it in advance. The specific steps include:

[0161] (1) Preloading: The predicted block Anext is loaded into the LVGL double buffer in advance to ensure a quick response when the user operates on the area.

[0162] (2) Pre-rendering: Combined with the pixel-by-pixel rendering mechanism of LVGL, high-priority blocks Anext are rendered with high precision to ensure that the image has been rendered when the user views the block.

[0163] Step 305: Dynamically adjust the scheduling model

[0164] As user interaction continues, the system updates the state transfer matrix (T(i,j) in real time based on the latest operation trajectory and dynamically optimizes the prediction results. Specifically:

[0165] (1) State transfer matrix update: When the user updates from block A i Transfer to Block A j , the system will update the state transfer matrix according to the user's latest operation frequency:

[0166]

[0167] Historical access frequency (F i ) and interactive behavior (U i )’s weight values ​​(0.6-0.8 and 0.2-0.4 respectively) are coordinated to highlight long-term operation trends while flexibly responding to immediate behaviors, especially the processing of complex areas will not be obscured by short-term operations.

[0168] (2) Real-time scheduling adjustment: As the state transfer matrix and cache priority (Pj) are updated, the system automatically adjusts the preloading and rendering plans to ensure that the scheduling strategy always fits the user's latest operating habits.

[0169] Embodiment 4

[0170] This embodiment provides a method for accelerating the processing of image rendering by GPU parallel processing in LVGL environment. By assigning image block tasks to multiple GPU cores in parallel.

[0171] Step 401: Task Assignment

[0172] After completing the adaptive segmentation of the image and determining the cache priority, the system allocates the image to the buffer based on the size S of each segment. iThe size and priority of the blocks directly affect the order of task allocation on the GPU.

[0173] The task allocation formula of GPU is as follows:

[0174]

[0175] in:

[0176] W k represents the total task load assigned to the kth GPU core;

[0177] S i It is block A i The size of indicates the number of pixels occupied by the block;

[0178] P i It is the cache priority of the block, combining the complexity and access frequency of the image

[0179] and user interaction behavior (see the priority calculation formula in Example 1).

[0180] Step 402: Load Balancing

[0181] In order to ensure that the tasks of each core of the GPU are reasonably allocated, this embodiment dynamically adjusts the workload of each core through a load balancing mechanism to avoid the situation where some cores are overloaded while other cores are underloaded. The load balancing formula is as follows:

[0182]

[0183] When the calculated load balance Balance exceeds the set threshold φ (the value range of φ is set to 1.2-1.5), the system will automatically reallocate some tasks to ensure that the load of each GPU core is close to balanced.

[0184] In the present invention, the threshold φ is set at 1.2-1.5 to balance efficiency and system stability. The specific considerations are as follows: (1) Reduce task redistribution: When the load balance is between 1.2 and 1.5, each GPU core can still work efficiently and avoid frequent task adjustments. (2) Adapt to core performance differences: Different GPU cores have different processing capabilities for task types. Allowing 20%-50% load imbalance helps to give full play to core advantages. (3) Tolerate short-term fluctuations: Short-term load fluctuations within this range will not affect system stability. (4) Reduce scheduling overhead: Too low a threshold will lead to frequent reallocation and increase overhead; the range of 1.2-1.5 ensures that scheduling only occurs when necessary, thereby improving overall efficiency.

[0185] Step 403: Parallel Rendering

[0186] After task allocation and load balancing are completed, the GPU starts to render each block in parallel. Combined with LVGL's pixel-by-pixel rendering mechanism, each GPU core processes different blocks separately, greatly shortening the rendering time through parallel computing.

[0187] (1) High-complexity block processing: The GPU prioritizes processing high-complexity, high-priority image blocks. These blocks usually involve more edges, color changes, and texture complexity, so more computing resources are allocated to them.

[0188] (2) Low-complexity block processing: Low-complexity image blocks are simplified and processed by other cores of the GPU to reduce GPU computing time and resource consumption.

[0189] Step 404: GPU and LVGL rendering synchronization

[0190] After the GPU completes the block rendering, the system uses LVGL's double buffering mechanism to synchronize the results to the display screen to avoid delays and prevent screen tearing. The double buffering mechanism enables rendering to be completed in the background and switched to the foreground display at one time. High-priority blocks can be presented immediately when the user interacts, greatly reducing waiting time and ensuring a smooth experience.

[0191] Step 405: Performance Optimization

[0192] In order to further improve the rendering performance, this embodiment introduces a dynamic performance optimization mechanism, which mainly includes the following two aspects:

[0193] (1) Dynamic resolution adjustment: When the GPU load is high, the system will dynamically adjust the resolution of low-priority blocks according to the priority of the blocks to reduce the computing burden. For example, low-priority blocks are rendered at a lower resolution to ensure the balance of the overall rendering progress.

[0194] (2) Resource recycling: After block rendering is completed, the GPU will actively release computing resources that are no longer in use, reduce memory usage, and make room for new tasks.

[0195] Embodiment 5

[0196] This embodiment provides a method for dynamic compression and format conversion of images in an LVGL environment.

[0197] Step 501: Compression rate adjustment

[0198] When rendering an image, the system first calculates the cache priority (P i ), dynamically adjust the compression rate of each image block

[0199] The compression ratio is calculated as follows:

[0200]

[0201] Where: Q i It is a block (A i ) compression quality; C max is the maximum computing capability of the device; L current Indicates the current system load (such as CPU or GPU occupancy); P i is the cache priority of the chunk.

[0202] Step 502: Format conversion

[0203] The selection formula for format conversion is as follows:

[0204]

[0205] This formula is used to determine the image block (A i Format(A i ), by selecting the image format that strikes the best balance between file size and quality. The formula combines the file size and image quality ratio of different image formats (such as WebP, JPEG, PNG) to select the optimal format for compression and storage. Among them:

[0206] Format(A i ): expressed as blocks (A i ) The system will compare different formats (such as WebP, JPEG, PNG) and choose the format that best suits the current image segment.

[0207] argmin: Indicates that the format that minimizes the value of the subsequent expression is selected. In other words, the system will select the image format f that minimizes the ratio of file size to image quality.

[0208] f∈{WebP,JPEG,PNG}:represents a set of available image formats. The system will select from these common image formats.

[0209] Indicates the ratio of file size to image quality of image format f.

[0210] Step 503: Dynamically adjust compression parameters

[0211] In actual operation, the system will continuously monitor the device load and image rendering requirements, and dynamically adjust the compression parameters of each block. When the load is high, the system will reduce the rendering accuracy of low-priority blocks and automatically increase the compression rate to reduce rendering pressure.

[0212] Step 504: Resource recovery and format conversion optimization

[0213] After the image blocks are rendered, the system will automatically recycle the image resources used in the rendering process and further optimize storage through format conversion. After rendering, low-priority blocks will be compressed into an efficient format (such as WebP) to reduce storage space.

[0214] Step 505: Compression result submission and rendering

[0215] Finally, the image block data after dynamic compression and format conversion will be submitted to the LVGL rendering engine. LVGL is responsible for rendering these block data into the display buffer and adjusting the display resolution and rendering details according to the load status of the device.

[0216] (1) High-priority tile display: For high-priority tiles, LVGL will display them at full resolution to ensure that details and image quality are perfectly displayed.

[0217] (2) Simplified display of low-priority blocks: For low-priority blocks, LVGL will simplify their display details based on the compression quality and format conversion results to reduce rendering pressure.

[0218] Embodiment 6

[0219] Smart home control panels usually need to render complex images such as room layouts and device status icons, and handle frequent user clicks, scrolling, and zooming operations. On resource-constrained embedded devices, such as those with limited processors and memory, the present invention improves image rendering efficiency through a series of optimization steps and parameter adjustments to ensure smooth user operations.

[0220] Step 1: Determine the block cache priority

[0221] The system first calculates the cache priority for each image block. The cache priority is determined based on the following parameters:

[0222] (C i ) (Image Complexity): reflects the edge, texture, and color changes of image blocks. For example, the device icon area in a room layout may have high complexity, while the background area has low complexity.

[0223] (F i )(Historical access frequency): The blocks that users frequently visited in the past. For example, if a user clicks on the light control icon or air conditioning icon multiple times, the access frequency of these areas is high.

[0224] (U i)(User's current interaction behavior): reflects the user's immediate interaction frequency with a certain block during the current operation. For example, if the user is clicking or dragging a device icon, the priority of the icon will be increased.

[0225] Weight coefficient: By adjusting α1, α2, and α3, we can control the impact of image complexity, historical access frequency, and current interaction behavior on cache priority.

[0226] Step 2: Adaptive Block Segmentation

[0227] For highly complex image areas, the system will further subdivide the blocks to ensure that complex areas (such as device icons, text areas, etc.) can be rendered finely. The following parameters determine this process:

[0228] Threshold τ: The system sets a complexity threshold. When the average complexity of a block exceeds the threshold, the system automatically subdivides the block into smaller blocks. A higher threshold is suitable for simple image scenes, while a lower threshold is used to ensure that areas with rich details can be processed more finely.

[0229] By adjusting τ, the system is able to find a balance between high-detail areas and low-complexity areas, saving rendering resources and ensuring that the details that the user cares about are rendered accurately.

[0230] Step 3: Intelligent predictive scheduling

[0231] Predict the user's next operation based on the user's historical operation behavior, and schedule the rendering of the blocks that may be accessed in advance. Key parameters include:

[0232] T(current,j) (state transition probability): represents the probability of a user transferring from the current block to another block. This probability is calculated based on the user's past operation records.

[0233] Cache priority (P j ): Combining user operation prediction and cache priority, the system selects the tiles that are most likely to be accessed and renders them in advance.

[0234] On the smart home panel, users may frequently operate control icons such as lights and air conditioners. The system analyzes the user's common operation patterns, predicts the area the user is about to click, and preloads it in advance, thereby improving response speed and reducing operation delays.

[0235] Step 4: GPU parallel acceleration processing

[0236] In order to improve rendering performance, the system assigns a large number of image block tasks to multiple cores of the GPU for parallel processing. The parameters that affect the parallel task allocation include:

[0237] (S i )(Block size): The number of pixels in a block. Complex image regions often contain more pixels and require more GPU computing resources.

[0238] Cache priority (P i ): Tiles with higher priority will be assigned to the GPU for priority rendering, ensuring faster rendering of areas with frequent user interactions or complex areas.

[0239] The task allocation of the GPU core is based on the size and priority of each block. The system dynamically adjusts the allocation to maintain the balance of the GPU load and avoid overloading one core while another core is idle.

[0240] Step 5: Dynamic compression and format conversion

[0241] According to the current system load and device status, the system dynamically adjusts the compression rate and format of the image to optimize memory usage and processing speed. The main parameters include:

[0242] (Lcurrent) (Current system load): When the system load is high, the image will use a higher compression rate to reduce memory usage and computing burden. Image complexity: High-complexity image areas will maintain a lower compression rate to ensure rendering quality, while lower-complexity areas can use a higher compression rate.

[0243] The above-mentioned embodiments only express several implementation methods of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. A method for accelerating image rendering under lvgl, characterized in that: The following steps are involved: Step 1: Adaptively divide the image into blocks according to the complexity, texture and color changes of each area of ​​the image; Step 2: Analyze the complexity, access frequency, and user interaction behavior of the image blocks, and adjust the block cache priority; Step 3: Perform intelligent prediction and scheduling based on user behavior to load the image blocks that the user is about to access in advance; Step 4: Through GPU parallel acceleration processing, the image segmentation tasks predicted and scheduled in step 3 are assigned to the GPU core; Step 5: Adjust the image compression rate and format based on the image segmentation priority, device performance, and current system load.

2. A method for accelerating image rendering under lvgl according to claim 1, characterized in that: In step 1, the following steps are specifically performed: Step 101: Load the image to be rendered into the LVGL rendering system, and process the image using pixel-by-pixel rendering and double buffering mechanism; Step 102: After completing the LVGL rendering compatibility initialization, perform complexity analysis on each pixel area of ​​the image to generate a complexity matrix; Step 103: performing adaptive block processing according to the complexity matrix generated in step 102; Step 104: After the adaptive block processing is completed, each area is rendered in a graded manner according to the complexity of each block.

3. A method for accelerating image rendering under lvgl according to claim 1, characterized in that: In step 2, the following steps are specifically performed: Step 201: determining the cache priority of each block by analyzing the complexity, access frequency and user interaction behavior data of each block of the image; Step 202: Based on the cache priority of the blocks, adjust the processing order of the blocks by the system; Step 203: Based on the block priorities, the blocks are divided into different levels of cache systems for management; Step 204: Calculate and score the blocks according to the block priority and residence time, and eliminate low-priority blocks.

4. A method for accelerating image rendering under lvgl according to claim 1, characterized in that: In step 3, the following steps are specifically performed: Step 301: Collect user's click, scroll, zoom and other operation behavior data, record operation track information, and generate user interaction behavior data set; Step 302: Use the behavior data provided by dataset D to construct a user's progress from the current block to the next block A. j The state transition matrix of Step 303: Scheduling the image blocks based on the state transfer matrix and the block cache priority; Step 304: According to the scheduling result in step 303, the block that the user is about to access is preloaded and rendered in advance; Step 305: Update the scheduling model and adjust the state transfer matrix according to the user's operation trajectory.

5. The method for accelerating image rendering under lvgl according to claim 1, characterized in that: In step 4, the following steps are specifically performed: Step 401: Allocate the block tasks to each computing core of the GPU according to the size and cache priority of each block; Step 402: adjusting the workload of each core through a load balancing mechanism; Step 403: After task allocation and load balancing are completed, each block is rendered in parallel through the GPU; Step 404: display the rendering result synchronously through the double buffer mechanism of LVGL; Step 405: Adjust the block resolution and release unused computing resources.

6. The method for accelerating image rendering under lvgl according to claim 1, characterized in that: In step 5, the following steps are specifically performed: Step 501: adjusting the compression rate of each image block according to the performance status of the device and the cache priority of the block; Step 502: Select a suitable picture format according to the compression quality and picture type of each block; Step 503: adjusting the compression parameters of each block by monitoring the load of the device and the rendering requirements of the image; Step 504: Recycle the image resources used in the rendering process and optimize storage through format conversion; Step 505: Submit the image block data after dynamic compression and format conversion processing to the LVGL rendering engine.

7. A method for accelerating image rendering under lvgl according to claim 2, characterized in that: The complexity matrix is ​​calculated by the following formula: Among them, M(Ai) represents the sum of the complexity of all pixels in the block and the average value; |Ai| represents the number of pixels of block Ai; (x,y)∈Ai(x,y) represents each pixel in block Ai, whose coordinates are (x,y); G(x,y) represents the edge gradient of the image at the pixel point (x,y); C(x,y) represents the color change rate of the image at the pixel point (x,y); T(x,y) represents the texture complexity of the image at the pixel point (x,y); λ1,λ 2, λ3: weight coefficient, corresponding to edge gradient, color change and texture complexity respectively.

Images