Method for optimizing display of embedded UI pictures
By converting images from embedded devices into binary format and preloading them into off-chip memory, the problem of image display lag on embedded devices is solved, achieving efficient image rendering and smooth interface switching, thus improving user experience and system stability.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- ZHEJIANG MOVENPICK INTELLIGENT EQUIP MFG CO LTD
- Filing Date
- 2025-12-19
- Publication Date
- 2026-06-05
AI Technical Summary
Existing technologies suffer from stuttering issues when displaying images in embedded devices, especially when loading multiple or large images, resulting in significant display delays and interface stuttering, as well as consuming CPU resources and impacting user experience.
Multiple original images are converted into binary format files to generate a unified resource package file, which is then loaded into the off-chip programmable static memory at power-on. Contiguous memory space is pre-allocated, and the image data is directly read from the memory for display during graphical interface rendering, avoiding repeated memory allocation and decoding operations.
Significantly reduces read latency, improves rendering efficiency, reduces CPU usage, eliminates interface lag, supports high frame rate image switching and carousel, and improves long-term system stability.
Smart Images
Figure CN122152406A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of graphics display technology, and in particular relates to an embedded UI image display optimization method and its storage medium. Background Technology
[0002] In embedded device UI development, image display directly impacts user experience. Users' demands for visual interaction in embedded devices are constantly increasing. Traditionally, two technical solutions are often employed. First, images are converted into C language arrays and directly compiled into the firmware. While this method offers fast read speeds, a large number of images quickly consume valuable Flash space, severely limiting functional expansion. Second, images are stored as files on external media and read and decoded at runtime through the file system. While the second method saves on main control Flash, for example, Chinese patent document [CN111314699A] discloses an image decoding method, device, electronic device, and storage medium, which uses a first processor and a second processor in the electronic device to handle image data parsing and decoding operations respectively. The above technical solution utilizes OpenCL for rapid development and efficient decoding, and combines it with FPGA for hardware decoding, which can improve decoding speed and reduce power consumption. However, repeated I / O operations and decoding can lead to significant display latency and interface stuttering, especially when reading large images or loading multiple images simultaneously. Formats such as PNG / JPG need to be decompressed and converted using software decoders (such as lodepng, jpeg_decoder). The decoding process consumes CPU resources, and the decoding speed is greatly affected by image size and compression ratio, resulting in a very poor user experience. Summary of the Invention
[0003] The purpose of this invention is to address the above-mentioned problems by providing an embedded UI image display optimization method that solves the lag issue in dynamically rendered pages with images and avoids repeated memory allocation that leads to prolonged lag.
[0004] To achieve the above objectives, the present invention adopts the following technical solutions: This embedded UI image display optimization method includes the following steps: S1. Convert the multiple original image resources required by the application into binary format files compatible with the target graphics library, and package all binary format files into a unified resource package file; S2. The resource package file is burned into the non-volatile memory of the embedded device in one go, and the partition is configured as a read-only data area in the linker script of the device firmware; S3. When the embedded device is powered on and initialized, the entire resource package file is read from the non-volatile memory, the image metadata in it is parsed, and a continuous memory space is pre-allocated for each image in the off-chip programmable static memory according to the image metadata. All image data in the resource package file is loaded into the corresponding pre-allocated memory address in the off-chip programmable static memory at one time. S4. During the graphical interface rendering process, when an image needs to be displayed, the image data is directly read from the corresponding memory address that has been loaded in the off-chip programmable static memory and passed to the graphics library for drawing.
[0005] In the above-mentioned embedded UI image display optimization method, Resource preprocessing steps include: S11. Use the image conversion tool of the target graphics library to convert the original image into a binary file compatible with the target graphics library, and select the corresponding color format according to the display characteristics of the target device; S12. Use a custom packaging tool to package all the converted single files. Figure 2 The binary files are packaged to generate a unified resource package file, which includes three parts: a file header, an index table, and image data body.
[0006] In the above-mentioned embedded UI image display optimization method, the file header of the resource package file includes a resource package identifier and the total number of images; the index table includes multiple records, each corresponding to one image, and includes an image identifier, metadata offset address, data length, and format identifier; the image data body is a continuous concatenation of all binary files.
[0007] In the above-mentioned embedded UI image display optimization method, in S2, the resource package file is burned to an independent partition of the off-chip programmable static memory using a burning tool. In the linker script of the firmware compilation, the starting address, size and access attributes of the partition where the resource package file is located in the non-volatile memory are defined, and the memory protection unit or memory management unit is configured to control the access permissions of the read-only data area.
[0008] In the above-mentioned embedded UI image display optimization method, the preloading initialization step specifically includes: S31. Read the file header of the resource package file and verify the validity of the resource package identifier and version number; S32. Read the index table of the resource package file and cache the index table data in the on-chip random access memory; S33. Memory pre-allocation: Based on the data length of each image in the index table, allocate a contiguous memory space of matching size for each image in the off-chip programmable static memory, and establish a mapping table between image identifiers and memory addresses. S34. Batch loading of data: Based on the data offset address and data length in the index table, all image data in the resource package file are loaded in batches into the corresponding pre-allocated memory space.
[0009] In the aforementioned embedded UI image display optimization method, S33 specifically includes: S331. Initialize the off-chip programmable static memory and obtain its total available address range; S332. Allocate memory space according to the display priority of the image and the size of the data in the metadata, following the principle of contiguous addresses; S333. Generate a mapping table between image identifiers and off-chip programmable static memory addresses, and store the mapping table in on-chip random access memory; S334. Reserve a predetermined proportion of redundant space during allocation to avoid allocation failure caused by memory fragmentation.
[0010] In the above-mentioned embedded UI image display optimization method, memory pre-allocation also includes memory alignment processing, in which the allocated memory space is aligned according to the CPU cache line size, and the alignment size is selected as 32 bytes, 64 bytes or 128 bytes.
[0011] In the above-mentioned embedded UI image display optimization method, the high-speed access steps specifically include: S41. Based on the image identifier requested to be displayed, query the mapping table between image identifiers and memory addresses to obtain the corresponding memory address; S42. Data is read directly without going through the file system search and decoding process; image pixel data is read directly from the memory address of the off-chip programmable static memory. S43. Graphics library interface call: Display the read image pixel data through the image settings interface of the target graphics library.
[0012] This embedded UI image display optimization system includes: The resource preprocessing module is used to convert multiple original image resources into binary format files compatible with graphics libraries, and package all binary format files into a unified resource package file; The resource deployment module is used to burn the resource package file into an independent partition of the non-volatile memory of the embedded device, and configure the device firmware linker script to set the partition as a read-only data area. The preloading module is used to read the resource package file from the read-only data partition of the non-volatile memory when the embedded device is powered on, parse the image metadata therein, pre-allocate independent contiguous memory space for each image in the off-chip programmable static memory according to the image metadata, and load all image data into the corresponding pre-allocated memory space. The high-speed access module is used to read image data directly from the corresponding memory address of the off-chip programmable static memory and pass it to the graphics library for rendering and display when an image is needed during the graphical interface rendering process.
[0013] This computer can read a storage medium on which a computer program is stored. When the computer program is executed by a processor, it implements the above-described embedded graphical interface display optimization method.
[0014] Compared with existing technologies, the advantages of this embedded UI image display optimization method are: Significantly reduced read latency: Traditional methods rely on on-demand Flash reading, which is limited by Flash addressing speed (usually 1-5MB / s), resulting in a read latency of 50-100ms for a single 100KB image; This invention loads the image into high-speed PSRAM (read speed up to 20-50MB / s) at startup and accesses it directly in memory during runtime, reducing read latency to 1-5ms, a reduction of over 90%, and completely eliminating UI lag.
[0015] Significantly optimized rendering efficiency: If the traditional method uses compressed images, real-time decoding is required during display, which consumes CPU resources and adds a delay of 10-30ms; this invention completes RLE decompression and stores the original pixel data synchronously during the startup loading stage, and can directly read the pixel data of the adapted format during LVGL rendering without additional decoding or format conversion operations, improving the rendering efficiency of a single image by 3-5 times.
[0016] Improved smoothness of multi-image switching: Traditional multi-image switching requires repeated execution of the "Flash read-decode" process, with switching intervals often exceeding 100ms, which can easily cause screen tearing; In this invention, all images reside in PSRAM, and only the address of the LVGL image descriptor needs to be modified during switching, so the switching latency can be controlled within 5ms, supporting high frame rate (above 60FPS) image carousel scenarios.
[0017] Reduced CPU utilization: Traditional methods of reading single bytes from Flash and real-time decoding require continuous CPU resources (peak utilization of 30%-50%). This invention uses "DMA batch loading + PSRAM direct reading", with CPU utilization below 10% during the loading phase. During the runtime display phase, the CPU only needs to process LVGL rendering logic, reducing utilization to below 5%, freeing up more computing power for other core services (such as sensor data processing and communication protocol parsing).
[0018] Mitigation of memory fragmentation: Traditional runtime dynamically allocates memory to store image data, which easily leads to memory fragmentation and may cause allocation failures over long-term operation; This invention pre-allocates contiguous PSRAM space based on an index table at startup, and all image data storage addresses are fixed and contiguous, with no dynamic memory allocation / release operations throughout the process, resulting in a memory fragmentation rate close to 0 and improved long-term system stability. Attached Figure Description
[0019] Figure 1 This is a flowchart of the image display method provided by the present invention.
[0020] Figure 2 This is a preloading initialization flowchart provided by the present invention. Detailed Implementation
[0021] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.
[0022] In this embodiment, an external SPI Flash is used as a non-volatile memory, and an SPI PSRAM is used as an off-chip programmable static memory.
[0023] First, the lv_img_conv image conversion tool provided by the target graphics library LVGL is called. Based on the display characteristics of the target device, the corresponding color format is selected to convert PNG and JPG resources such as icons and background images in the UI design into binary format files. This ensures that the image data format is directly compatible with the image library and the display screen, avoiding runtime format conversion and achieving the goal of high-speed rendering later.
[0024] Furthermore, using a custom packaging tool, all converted single... Figure 2 The binary files are packaged to generate a unified resource package file, which includes three parts: a file header, an index table, and image data body.
[0025] In this embodiment, the custom packaging tool reads and parses the configuration file to determine the numeric identifiers, binary file paths, color format encodings, and loading priority information of all image resources to be packaged. Then, it performs the first round of traversal according to the configuration list, opens each specified single image .bin file to obtain its size, and dynamically calculates the layout of the entire resource package in memory, determines the starting positions and space requirements of the three parts of the file header, index table, and data body, and temporarily stores the metadata of each image.
[0026] Furthermore, a predefined four-byte magic word and a two-byte total number of images are written to the beginning of the output buffer to form the file header. An index table space matching the number of images is reserved after the file header, and the second traversal begins. For each configuration list, the absolute offset address of its image data within the final resource package file is calculated. The image ID, offset address, data length, and format identifier are then packaged into a 16-byte record in little-endian byte order and sequentially filled into the index table.
[0027] After the index table is filled, a third traversal is performed. Following the order in the configuration file, the raw byte content of all individual .bin files is read sequentially and seamlessly, contiguously appended to the index table to form a compact and complete image data body. All image data is integrated into a linear, contiguous memory block, optimizing data transfer efficiency in subsequent steps. Whether written to Flash storage via a programming tool or loaded into PSRAM via an embedded system, this achieves fast block transfer performance. A custom packaging tool packages the complete memory buffer containing the file header, index table, and contiguous data body into a resource file package.
[0028] In this embodiment, the file header contains a 4-byte resource package identifier and a 2-byte total number of images, providing a fast entry point for authentication and scale confirmation. Each record in the index table is 16 bytes, accurately recording the ID, metadata offset address, data length, and format identifier of each image, establishing an efficient addressing directory. This allows the system to load and access images without traversing the entire resource package, thus reducing time complexity. All .bin files are concatenated sequentially according to the index table within the image data body, resulting in a compact storage structure that facilitates subsequent one-time, batch data loading operations.
[0029] Furthermore, the resource package file is burned into the non-volatile memory of the embedded device in one go, and the partition is configured as a read-only data area in the linker script of the device firmware.
[0030] The resource package file is burned to an independent partition of the off-chip programmable static memory using a burning tool. In the linker script of the firmware compilation, the starting address, size and access attributes of the partition where the resource package file is located in the non-volatile memory are defined, and the memory protection unit is configured to control access permissions to the read-only data area.
[0031] By embedding the resource package file as read-only data, accidental tampering during program execution is prevented, thus improving system robustness. On the other hand, hardware protection via the MPU enhances system security and allows resource data to be accessed directly by logical addressing, simplifying the loading logic.
[0032] When the embedded device is powered on and initialized, the entire resource package file is read from the non-volatile memory. First, the file header of the resource package file is read to verify the validity of the resource package identifier and version number; then, the index table of the resource package file is read, and the index table data is cached in the on-chip RAM. The most frequently used addressing information is placed in high-speed memory, which greatly speeds up subsequent query operations.
[0033] Furthermore, the off-chip programmable static memory is initialized to obtain its total available address range. Based on the index table cached in RAM, and according to the data length of each image and its display priority in the UI, a contiguous, matching memory space is allocated for each image in the off-chip programmable static memory. The allocation follows the principle of contiguous addresses and reserves approximately 10% of redundant space. The pre-allocation of contiguous space avoids memory fragmentation caused by dynamic allocation at runtime, improving memory utilization efficiency and long-term system stability. The reserved redundant space provides a buffer for possible future resource updates or dynamic content, enhancing the system's scalability.
[0034] And establish a mapping table between image identifiers and memory addresses; The image metadata is parsed, and a contiguous memory space is pre-allocated for each image in the off-chip programmable static memory based on the image metadata. All image data in the resource package file is loaded into the corresponding pre-allocated memory address in the off-chip programmable static memory at once.
[0035] During allocation, memory alignment is performed, aligning to the CPU cache line size of 64 bytes. This ensures that subsequent memory accesses can fully utilize the CPU cache, significantly reducing the performance loss caused by unnamed cache entries and further improving data read speed.
[0036] A mapping table between image identifiers and off-chip programmable static memory addresses is generated, and the mapping table is stored in on-chip random access memory to establish a final and efficient image addressing path.
[0037] Furthermore, based on the data offset address and data length in the index table, all image data in the resource package file are loaded in batches into the corresponding pre-allocated memory addresses. DMA is used to free up the CPU, reducing CPU usage during the loading phase to below 10%. One-time loading avoids repeated access to slow Flash memory during runtime, reducing image read latency from the traditional 50-100ms to 1-5ms, and eliminating UI stuttering caused by I / O, etc.
[0038] During the graphical interface rendering process, when an image needs to be displayed, the image data is directly read from the corresponding memory address that has been loaded in the off-chip programmable static memory and passed to the graphics library for drawing.
[0039] First, based on the image identifier requested to be displayed, the mapping table between image identifiers and memory addresses is queried to obtain the corresponding memory address.
[0040] Further reading of direct data does not require file system lookup and decoding. Image pixel data is read directly from the memory address of the off-chip programmable static memory. The access path is short, and the reading speed of off-chip programmable static memory is much higher than that of Flash. Since the data has been adapted to the original pixel format of LVGL during preloading, the CPU consumption and latency of real-time decoding are completely eliminated.
[0041] Finally, the graphics library interface is called, and the read image pixel data is displayed through the target graphics library's image settings interface. The entire "addressing, reading, and rendering" process is highly efficient and concise, greatly improving single-image rendering efficiency and allowing multi-image switching latency to be easily controlled within 5ms. It supports smooth animations and image carousels at 60FPS or even higher, eliminating screen tearing. Simultaneously, during runtime, the CPU only needs to process the rendering logic during the display phase, reducing CPU usage and freeing up significant computing power for other business operations.
[0042] This embedded UI image display optimization system includes: The resource preprocessing module is used to convert multiple original image resources into binary format files compatible with graphics libraries, and package all binary format files into a unified resource package file; The resource deployment module is used to burn the resource package file into an independent partition of the non-volatile memory of the embedded device, and configure the device firmware linker script to set the partition as a read-only data area. The preloading module is used to read the resource package file from the read-only data partition of the non-volatile memory when the embedded device is powered on, parse the image metadata therein, pre-allocate independent contiguous memory space for each image in the off-chip programmable static memory according to the image metadata, and load all image data into the corresponding pre-allocated memory space. The high-speed access module is used to directly read image data from the corresponding memory address of the off-chip programmable static memory and pass it to the graphics library for rendering and display when an image is needed during the graphical interface rendering process. The modules work together to optimize the entire chain from resource preparation, deployment, and initialization to efficient rendering.
[0043] This computer can read a storage medium on which a computer program is stored. When the computer program is executed by a processor, it implements the display optimization method for the embedded graphical interface as described in any one of claims 1-8.
[0044] The specific embodiments described herein are merely illustrative of the spirit of the invention. Those skilled in the art to which this invention pertains may make various modifications or additions to the described specific embodiments or use similar methods to substitute them, without departing from the spirit of the invention or exceeding the scope defined by the appended claims.
[0045] Although this document uses terms such as non-volatile memory, off-chip programmable static memory, display interface, and resource package file frequently, the possibility of using other terms is not excluded. These terms are used merely for the convenience of describing and explaining the essence of this invention; interpreting them as any additional limitation would contradict the spirit of this invention.
Claims
1. An embedded UI image display optimization method, characterized in that, Includes the following steps: S1. Convert the multiple original image resources required by the application into binary format files compatible with the target graphics library, and package all binary format files into a unified resource package file; S2. The resource package file is burned into the non-volatile memory of the embedded device in one go, and the partition is configured as a read-only data area in the linker script of the device firmware; S3. When the embedded device is powered on and initialized, the entire resource package file is read from the non-volatile memory, the image metadata in it is parsed, and a continuous memory space is pre-allocated for each image in the off-chip programmable static memory according to the image metadata. All image data in the resource package file is loaded into the corresponding pre-allocated memory address in the off-chip programmable static memory at one time. S4. During the graphical interface rendering process, when an image needs to be displayed, the image data is directly read from the corresponding memory address that has been loaded in the off-chip programmable static memory and passed to the graphics library for drawing.
2. The embedded UI image display optimization method according to claim 1, characterized in that, In S1, the resource preprocessing steps include: S11. Use the image conversion tool of the target graphics library to convert the original image into a binary file compatible with the target graphics library, and select the corresponding color format according to the display characteristics of the target device; S12. Using a custom packaging tool, package all the converted single-image binary files into a unified resource package file. The resource package file includes three parts: a file header, an index table, and an image data body.
3. The embedded UI image display optimization method according to claim 2, characterized in that, The header of the resource package file includes a resource package identifier and the total number of images; the index table includes multiple records, each corresponding to one image, and includes an image identifier, metadata offset address, data length, and format identifier; the image data body is a contiguous concatenation of all binary files.
4. The embedded UI image display optimization method according to claim 1, characterized in that, In S2, the resource package file is burned to an independent partition of the off-chip programmable static memory using a burning tool. In the linker script of the firmware compilation, the starting address, size and access attributes of the partition where the resource package file is located in the non-volatile memory are defined, and the memory protection unit or memory management unit is configured to control the access permissions of the read-only data area.
5. The embedded UI image display optimization method according to claim 4, characterized in that, In S3, the preload initialization steps specifically include: S31. Read the file header of the resource package file and verify the validity of the resource package identifier and version number; S32. Read the index table of the resource package file and cache the index table data in the on-chip random access memory; S33. Memory pre-allocation: Based on the data length of each image in the index table, allocate a contiguous memory space of matching size for each image in the off-chip programmable static memory, and establish a mapping table between image identifiers and memory addresses. S34. Batch loading of data: Based on the data offset address and data length in the index table, all image data in the resource package file are loaded in batches into the corresponding pre-allocated memory space.
6. The embedded UI image display optimization method according to claim 5, characterized in that, Specifically, S33 includes: S331. Initialize the off-chip programmable static memory and obtain its total available address range; S332. Allocate memory space according to the display priority of the image and the size of the data in the metadata, following the principle of contiguous addresses; S333. Generate a mapping table between image identifiers and off-chip programmable static memory addresses, and store the mapping table in on-chip random access memory; S334. Reserve a predetermined proportion of redundant space during allocation to avoid allocation failure caused by memory fragmentation.
7. The embedded UI image display optimization method according to claim 6, characterized in that, Memory pre-allocation also includes memory alignment, where the allocated memory space is aligned according to the CPU cache line size, with the alignment size only selectable as 32 bytes, 64 bytes, or 128 bytes.
8. The embedded UI image display optimization method according to claim 1, characterized in that, In S4, the high-speed access steps specifically include: S41. Based on the image identifier requested to be displayed, query the mapping table between image identifiers and memory addresses to obtain the corresponding memory address; S42. Data is read directly without going through the file system search and decoding process; image pixel data is read directly from the memory address of the off-chip programmable static memory. S43. Graphics library interface call: Display the read image pixel data through the image settings interface of the target graphics library.
9. An embedded UI image display optimization system, characterized in that, include: The resource preprocessing module is used to convert multiple original image resources into binary format files compatible with graphics libraries, and package all binary format files into a unified resource package file; The resource deployment module is used to burn the resource package file into an independent partition of the non-volatile memory of the embedded device, and configure the device firmware linker script to set the partition as a read-only data area. The preloading module is used to read the resource package file from the read-only data partition of the non-volatile memory when the embedded device is powered on, parse the image metadata therein, pre-allocate independent contiguous memory space for each image in the off-chip programmable static memory according to the image metadata, and load all image data into the corresponding pre-allocated memory space. The high-speed access module is used to read image data directly from the corresponding memory address of the off-chip programmable static memory and pass it to the graphics library for rendering and display when an image is needed during the graphical interface rendering process.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the display optimization method for the embedded graphical interface as described in any one of claims 1-8.