Instrument picture compression and context sensing coding method

Abstract

The invention discloses an instrument picture compression and context sensing coding method, and the method comprises the steps: carrying out the hierarchical sparse voxelization representation of an instrument picture, and obtaining a first picture representation; and performing context-aware coding on the first picture representation to obtain a first picture code. In this way, the compression efficiency can be improved.

Description

Technical Field

[0001] The present application relates to the field of image compression technology, and in particular to an instrument image compression and context-aware coding method. Background Art

[0002] Modern automotive instrument clusters are moving towards intelligence and information technology, with central control displays and LCD instrument panels becoming standard features. Instrument UI design is also becoming increasingly complex, requiring the display of rich graphical information such as various indicator icons, dynamic effects, and high-resolution background images.

[0003] However, current instrument clusters, especially in cost-conscious automotive electronics solutions, generally use domestically produced chips with relatively limited performance. These chips have the following inherent bottlenecks in processing graphics and images:

[0004] Slow Flash reading speed: The Flash memory supporting some domestic chips has a slow reading speed, which causes the time for loading images from Flash to memory to be too long, seriously affecting the startup speed and smoothness of the UI.

[0005] Libpng decoding efficiency is low: The commonly used PNG image decoding library Libpng has low decoding efficiency on some domestic chips, especially for high-resolution PNG images. The CPU decoding takes a huge amount of time, exacerbating the UI lag.

[0006] Lack of hardware acceleration: Many low-end chips lack a GPU or dedicated hardware acceleration unit and rely entirely on the CPU for graphics processing, which makes it extremely difficult to handle complex dynamic effects and high-resolution images.

[0007] High adaptation difficulty: Different chip platforms have performance differences and shortcomings, which requires instrument software UI developers to perform tedious adaptation work for different platforms, significantly increasing the development difficulty and cycle.

[0008] Existing technical solutions mainly rely on the following two methods:

[0009] GPU acceleration: This approach uses a chip with GPU acceleration to offload image processing tasks to the GPU. While this approach can provide a smooth UI experience, it significantly increases hardware costs, which goes against the current trend of cost reduction and efficiency improvement in automotive electronics.

[0010] Standard CPU solution + Libpng decoding: This solution uses a standard CPU without hardware acceleration and a software library like Libpng for image decoding. This solution is relatively low-cost, but due to the aforementioned shortcomings, it is difficult to achieve a smooth instrument UI. Especially in scenarios requiring complex dynamic effects, the frame rate is typically less than 20 frames per second, resulting in a poor user experience.

[0011] Existing instrument image display technology solutions, based on resource-constrained domestic chip platforms, still face the following technical challenges that need to be addressed:

[0012] Insufficient compression efficiency and large storage space usage: Common image compression formats (such as PNG and JPG) or existing simple compression methods still have insufficient compression efficiency in instrumentation applications, resulting in large image files, occupying valuable Flash storage space, and increasing Flash read time.

[0013] Slow decoding speed and high CPU load: Even with precompiled binary data and optimized decoding methods, image decoding still accounts for a significant portion of the CPU load in instrument UIs with high resolution and complex dynamic effects. There is still room for improvement in decoding speed, especially on low-end chips.

[0014] Lack of optimization for instrument application scenarios: Existing technical solutions lack in-depth analysis and targeted optimization of the characteristics of instrument application scenarios. Instrument images usually have specific characteristics, such as:

[0015] Mainly vector graphic elements: Instrument icons, scales and other elements are usually vector graphics, with the characteristics of simple color areas and clear edges.

[0016] Monotonous colors: The colors of instrument UI are usually monotonous, with a limited variety of background and icon colors.

[0017] Partially transparent areas: Gauge icons and dynamic effect images often contain transparent areas to blend with the background.

[0018] Sequence frame animation requirements: There are a large number of sequence frame animations in the instrument, such as pointer rotation, indicator light flashing, etc., and there is a high degree of redundancy between sequence frames. Summary of the Invention

[0019] The instrument image compression and context-aware encoding method provided in this application can improve compression efficiency. The purpose of this application is to overcome the shortcomings of existing technical solutions and provide a more efficient, faster, and more adaptable image compression and decoding method for instrument products with limited resources. Specific goals include:

[0020] Improve compression efficiency: significantly improve the compression ratio of instrument images, reduce Flash storage space usage, and shorten Flash reading time.

[0021] Improved decoding speed: Significantly improves image decoding speed, reduces CPU load, ensures smooth operation of the instrument UI even on low-end chips, and achieves higher frame rates and more complex dynamic effects.

[0022] Optimization for instrument application scenarios: Fully utilize the characteristics of instrument images and design targeted compression and decoding algorithms to further improve performance.

[0023] Provides a more flexible solution: Provides configurable parameters and tools to facilitate developers to optimize and adjust according to different chip platforms and UI requirements.

[0024] In a first aspect, the present application provides an instrument image compression and context-aware encoding method, the method comprising: performing hierarchical sparse voxel representation on the instrument image to obtain a first image representation; performing context-aware encoding on the first image representation to obtain a first image encoding.

[0025] The instrument image is hierarchically sparsely voxelized to obtain a first image representation, including: hierarchically dividing the space of the instrument image using a quadtree data structure to construct a basic framework for hierarchical voxelization representation; the root node of the basic framework corresponds to the complete image area of ​​the instrument image, and each subsequent layer of nodes represents a four-equal sub-region division of the area covered by its parent node; according to the content complexity of the local area of ​​the image, the voxel size of each local area of ​​the image is adaptively determined to form corresponding voxels, and the voxels are used as leaf nodes in the quadtree; and attribute encoding is performed on each voxel to obtain the first image representation; wherein the attributes include color information and transparency information.

[0026] The content complexity is evaluated by at least one indicator among color variance, gradient amplitude and information entropy.

[0027] Among them, according to the content complexity of the local area of ​​the image, the voxel size of each local area of ​​the image is adaptively determined to form corresponding voxels, including: when the content complexity of the local area of ​​the image exceeds a threshold, the local area of ​​the image is subdivided; otherwise, the subdivision is stopped, and the voxel size of each local area of ​​the image is determined to form corresponding voxels.

[0028] Among them, attribute encoding is performed on each voxel, including: color information encoding according to the color attributes of the local area of ​​the image corresponding to each voxel; wherein, the color attributes include color change, color type, and appearance rendering; color change corresponds to average color encoding, color type corresponds to color palette encoding, and appearance rendering corresponds to spherical harmonic function coefficient encoding.

[0029] The performing of context-aware encoding on the first picture representation to obtain the first picture encoding includes: performing spatial context modeling, inter-frame context modeling, and parametric encoding on the first picture representation to obtain the first picture encoding.

[0030] The spatial context modeling of the first image representation includes: using the attribute values ​​of the encoded adjacent voxels to predict the attribute value of the current voxel; and adaptively selecting the optimal entropy coding model according to different contexts, and entropy coding the current voxel using the entropy coding model.

[0031] Among them, inter-frame context modeling is performed on the first picture representation, including: in response to the instrument picture being a sequence frame picture, motion compensation is performed on the voxels of the reference frame to obtain a predicted frame corresponding to the first picture representation; residual encoding is performed on the predicted frame; and key frames in the sequence frame picture are encoded, and only difference information is encoded for the remaining frames.

[0032] The parametric encoding of the first image representation includes: extracting vector parameters of vector graphic elements in the instrument image; the vector parameters include at least one of a straight line, an arc, a polygon and text; and encoding the extracted vector parameters.

[0033] After obtaining the first image code, the method further includes: decoding the first image code based on voxel rasterization at the instrument image display end, and displaying the decoded instrument image at the instrument image display end.

[0034] The present application has the following beneficial effects: Different from the prior art, the present application provides a method for compressing and context-aware encoding of an instrument image, comprising: performing hierarchical sparse voxel representation on the instrument image to obtain a first image representation; and performing context-aware encoding on the first image representation to obtain a first image encoding. This method can improve the compression efficiency of the instrument image. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without inventive efforts. Among them:

[0036] Figure 1 This is a flow chart of an embodiment of the instrument image compression and context-aware encoding method provided by the present application;

[0037] Figure 2 yes Figure 1 A schematic flow chart of an embodiment of step 11;

[0038] Figure 3 yes Figure 1 A flow chart of an embodiment of step 12;

[0039] Figure 4 yes Figure 1A schematic flow chart of another embodiment of step 12;

[0040] Figure 5 yes Figure 1 A schematic flow chart of another embodiment of step 12;

[0041] Figure 6 This is a structural diagram of an embodiment of a motorcycle adaptive instrument display compression system provided by the present application;

[0042] Figure 7 It is a structural diagram of an embodiment of a computer-readable storage medium provided by this application. DETAILED DESCRIPTION

[0043] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It will be understood that the specific embodiments described herein are only used to explain the present application, rather than to limit the present application. It should also be noted that, for ease of description, only some, rather than all, structures related to the present application are shown in the drawings. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0044] References herein to "embodiments" mean that a particular feature, structure, or characteristic described in connection with the embodiments may be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute an independent or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0045] See Figure 1 , Figure 1 This is a flow chart of an embodiment of a method for compressing and context-aware encoding of instrument images provided in this application. The method for compressing and context-aware encoding of instrument images includes:

[0046] Step 11: Perform hierarchical sparse voxel representation on the instrument image to obtain a first image representation.

[0047] In some embodiments, a corresponding system module may be used to perform hierarchical sparse voxel representation on an instrument image to be displayed to obtain a first image representation.

[0048] In some embodiments, see Figure 2 , step 11 can be the following process:

[0049] Step 111: Use a quadtree data structure to hierarchically divide the space of the instrument image and construct a basic framework for hierarchical voxel representation; the root node of the basic framework corresponds to the complete image area of ​​the instrument image, and each subsequent layer of nodes represents a four-equal sub-region division of the area covered by its parent node.

[0050] Because the instrument image is a two-dimensional image, a quadtree data structure is used to hierarchically partition the instrument image space. The quadtree constructs a hierarchical structure by recursively dividing a square area into four equal-sized sub-square areas.

[0051] Step 112: Adaptively determine the voxel size of each local image region based on the content complexity of the local image region, form corresponding voxels, and use the voxels as leaf nodes in the quadtree.

[0052] The content complexity is evaluated by at least one indicator among color variance, gradient amplitude and information entropy.

[0053] In some embodiments, when the content complexity of a local image region exceeds a threshold, the local image region is subdivided; otherwise, the subdivision is stopped, and the voxel size of each local image region is determined to form corresponding voxels.

[0054] Step 113: Attribute encoding is performed on each voxel to obtain a first image representation; wherein the attributes include color information and transparency information.

[0055] In some embodiments, encoding attributes for each voxel may be performed as follows:

[0056] Color information is encoded according to the color attributes of the local image area corresponding to each voxel; the color attributes include color change, color type, and appearance rendering; color change corresponds to average color encoding, color type corresponds to color palette encoding, and appearance rendering corresponds to spherical harmonic function coefficient encoding.

[0057] In one application scenario, a hierarchical sparse voxelization method is used to represent instrument images. This is a technology that converts traditional pixel matrix images into a sparse representation based on voxels (Volume Element). Unlike the pixel matrix that stores color information pixel by pixel, voxelization divides the image space into small volume units (degenerated into area units in two-dimensional images) and uniformly encodes the image attributes within each voxel. "Sparse" is reflected in the adaptive voxel division, using only fine voxels in areas with complex image content and coarse voxels in flat areas, thereby saving storage space. "Hierarchical" is reflected in the voxel organization structure. The hierarchical data structure of octree (degenerated into quadtree for two-dimensional images) can efficiently manage and index voxels of different sizes.

[0058] The specific steps are detailed as follows:

[0059] 1. Octree (quadtree) structure construction:

[0060] Principle: An octree is a tree-like data structure used to organize data in three-dimensional space. For two-dimensional images, we use its two-dimensional degenerate form, the quadtree. A quadtree constructs a hierarchical structure by recursively dividing a square region into four equally sized sub-square regions.

[0061] Image space partitioning is achieved by treating the two-dimensional image space where the instrument image is located as a square area, which serves as the root node of the quadtree. The root node represents the entire image area.

[0062] Recursive partitioning starts at the root node. Each layer of nodes (representing a square area) decides whether to subdivide based on the complexity of the image content within its coverage area. If subdivision is required, the current square area is divided into four equal-sized sub-square areas, each of which becomes a child of the current node. This partitioning process can be repeated recursively on multiple levels, forming a hierarchical structure.

[0063] The advantage of a hierarchical structure lies in its ability to represent different image regions using voxels of varying sizes. The root node represents the image overview, while deeper nodes represent image details. This hierarchical structure provides the foundation for subsequent adaptive voxel partitioning and efficient indexing.

[0064] The technical effect lies in using a quadtree data structure to hierarchically partition the two-dimensional image space, building a basic framework for hierarchical voxel representation. The root node of the quadtree corresponds to the complete image area, and each subsequent node represents a four-equal subregion division of the area covered by its parent node. This hierarchical structure allows the use of voxels of different sizes at different depth levels, thereby achieving multi-scale representation of image information from coarse to fine.

[0065] 2. Adaptive voxel partitioning:

[0066] Purpose: To achieve sparse representation, the voxel size of each region needs to be adaptively determined based on the complexity of the image content. In areas with rich image details or drastic color changes, smaller voxels are needed to capture fine details; in areas with flat image content or slow color changes, larger voxels can be used to reduce data redundancy.

[0067] The above-mentioned image content complexity can be evaluated by a variety of indicators, such as color variance, gradient magnitude and information entropy, as well as hybrid indicators.

[0068] Color Variance: Calculates the variance of pixel colors within the voxel coverage area. A larger variance indicates more dramatic color variations and higher complexity.

[0069] Gradient Magnitude: Calculates the average magnitude of pixel gradients within the voxel coverage area. Larger gradient magnitudes indicate more edges and details, and higher complexity.

[0070] Information Entropy: Calculates the information entropy of the pixel color distribution within the voxel coverage area. A larger information entropy indicates a more complex color distribution and higher complexity.

[0071] Hybrid metrics: Combined consideration of multiple metrics, such as color variance and gradient magnitude, to more comprehensively evaluate image complexity.

[0072] The adaptive voxel partitioning strategy is embodied in setting one or more thresholds based on the complexity evaluation index. When the complexity of the area covered by a voxel exceeds the threshold, it is subdivided; otherwise, subdivision stops and the voxel becomes a leaf node, representing a sparse voxel unit of the image.

[0073] The advantage of adaptability is that adaptive voxel partitioning can dynamically adjust the size and number of voxels according to the image content, minimize the number of voxels while ensuring image quality, and achieve efficient sparse representation.

[0074] The technical effect lies in: using an adaptive voxel partitioning strategy to dynamically adjust voxel size based on the content complexity of local image regions. Image complexity can be assessed using metrics such as color variance, gradient magnitude, or information entropy. When the complexity of the voxel coverage area exceeds a preset threshold, quadtree subdivision is performed; otherwise, subdivision is stopped, resulting in leaf node voxels. This adaptive strategy ensures that fine voxels are used in areas with rich image detail and coarse voxels are used in flat areas, thereby achieving content-adaptive sparse representation and effectively removing data redundancy.

[0075] 3. Voxel attribute encoding:

[0076] Purpose: Each voxel needs to store the color and transparency information of the area it covers, so that the image can be reconstructed at the decoder. Different encoding methods are suitable for different image characteristics and compression requirements.

[0077] Color information encoding methods include average color encoding, color palette encoding and spherical harmonic function coefficient encoding.

[0078] The average color coding method is as follows:

[0079] Applicable scenarios: Suitable for areas with gradual color changes, such as large areas with a single-color background.

[0080] Encoding method: Calculate the average color value (such as RGB average value) of all pixels in the voxel coverage area and store the average color value as the color attribute of the voxel.

[0081] Advantages: simple encoding, high compression rate, and can greatly reduce the amount of data.

[0082] Color Palette Coding is as follows:

[0083] Applicable scenarios: Suitable for instrument images with a limited number of colors, such as icons and vector graphic elements.

[0084] Encoding method: First, analyze the color distribution of the entire image or a group of images and construct a color palette containing a small number of representative colors. Then, for each voxel, determine the closest palette color within its coverage area and store the palette index of that color as the voxel's color attribute.

[0085] Advantages: It can effectively compress color information, especially when there are fewer types of colors, the compression rate is very high.

[0086] The spherical harmonics (SH) coefficients are encoded as follows:

[0087] Applicable scenarios: Suitable for instrument UI elements that require perspective-dependent effects. This refers to a rendering effect where the surface appearance of an object (such as color, brightness, and reflection) changes with the viewing angle. This effect can make the rendering more realistic and vivid, and more in line with the human eye's perception of objects in the real world.

[0088] Encoding method: Use low-order spherical harmonics (SH) to represent the color and illumination information within the voxel. Spherical harmonics are a set of orthogonal basis functions that can effectively represent direction-dependent illumination information. Voxels store low-order SH coefficients instead of directly storing color values: the SH coefficient stored in voxel υ is denoted as V sh For any viewing direction The color of voxel υ in this direction It can be calculated by the following formula:

[0089]

[0090] Indicates the viewing direction Below, the color value of voxel υ. This is a scalar value representing the intensity of the color on a certain color channel (for example, red channel R, green channel G, or blue channel B). In order to represent a color image, we need to encode the SH coefficient for each color channel.

[0091] V sh : Represents the spherical harmonics (SH) coefficient vector stored in voxel υ. This is a vector containing the low-order SH coefficients used to represent the voxel color and lighting information. The dimension of the vector depends on the order L of the spherical harmonics. For example, if a spherical harmonic of order L = 2 is used, the dimension of the Vsh vector is (L+1) 2 =9.

[0092] Represents the spherical harmonics basis function vector, in a given viewing direction For the L-order spherical harmonics, is a dimension of (L+1) 2 A vector whose elements are the values ​​of spherical harmonic basis functions of different orders and degrees. Usually normalized to a unit vector.

[0093] : represents the vector dot product operation. The above formula shows that the voxel color is obtained by transforming the voxel's SH coefficient vector V sh and viewing direction The spherical harmonics basis function vector Obtained by performing a dot product operation.

[0094] The transparency encoding (Alpha Coding) method is as follows:

[0095] Encoding method: Calculate the average transparency value (Alpha value) of all pixels in the voxel coverage area and store the average transparency value as the transparency attribute of the voxel.

[0096] Importance: Transparency information is crucial for the correct rendering of icons and animated elements in the instrument UI. Transparency encoding ensures that instrument UI elements blend correctly with the background to achieve the expected visual effect.

[0097] The technical benefits lie in the following: Multiple voxel attribute encoding methods are designed for voxel units in hierarchical sparse voxelization to accommodate the diverse content characteristics and compression requirements of instrument images. Color information can be encoded using average color encoding, color palette encoding, or spherical harmonics (SH) coefficient encoding; transparency information is encoded using average transparency encoding. The appropriate combination of encoding methods can be flexibly selected based on the actual application scenario to optimize compression efficiency and decoding performance.

[0098] To summarize: Through the above three steps, the hierarchical sparse voxelized image representation method proposed in this application can effectively convert instrument images from pixel matrices into an efficient, sparse, and hierarchical representation, laying a solid foundation for subsequent context-aware encoding and efficient decoding.

[0099] Step 12: Perform context-aware encoding on the first picture representation to obtain a first picture encoding.

[0100] To further improve the compression efficiency of instrument images based on hierarchical sparse voxel representation, context-aware coding (CAC) technology was introduced. The core idea of ​​CAC is to use various contextual information (such as spatial, temporal, and semantic context) in the image to predict the current data to be encoded. Only the prediction residuals or parameters related to the context are encoded, thereby reducing data redundancy and improving the compression ratio.

[0101] In some embodiments, spatial context modeling, inter-frame context modeling, and parametric encoding are performed on the first picture representation to obtain the first picture encoding.

[0102] In some embodiments, see Figure 3 , performing spatial context modeling on the first image representation can be the following process:

[0103] Step 31: Use the encoded attribute values ​​of the adjacent voxels to predict the attribute value of the current voxel.

[0104] Step 32: Adaptively select the optimal entropy coding model according to different contexts, and use the entropy coding model to perform entropy coding on the current voxel.

[0105] In order to further improve the compression efficiency, context-aware coding technology is introduced to make full use of the context information of instrument images for compression.

[0106] Spatial Context Modeling is implemented as follows:

[0107] Principle: Adjacent regions in an image often exhibit strong correlations. In instrument images, the color, transparency, and other attribute values ​​of adjacent voxels often exhibit similarity or gradual variations. Spatial context modeling aims to leverage this spatial correlation for predictive coding and adaptive entropy coding. Predictive coding and context-adaptive entropy coding (CAE) are primarily employed.

[0108] Predictive coding is primarily used, for example, to predict the attribute value of the current voxel using the attribute values ​​of encoded neighboring voxels. Because the predicted value is highly correlated with the true value, the variance and information entropy of the predicted residual (true value - predicted value) are typically much smaller than those of the original attribute value, allowing for more efficient encoding of the predicted residual.

[0109] The adjacent voxel selection is reflected in that the adjacent voxels can be: adjacent voxels at the same level and voxels at adjacent levels.

[0110] Adjacent voxels at the same level: for example, the encoded voxels to the left, above, and above the left of the current voxel.

[0111] Adjacent level voxels: for example, encoded voxels in the previous level (coarser level) that overlap or are adjacent to the current voxel in spatial position.

[0112] The prediction method can be a simple linear prediction, for example:

[0113] Mean prediction: Use the average of the attribute values ​​of adjacent voxels as the predicted value of the current voxel.

[0114] Median prediction: Use the median of the attribute values ​​of adjacent voxels as the predicted value of the current voxel.

[0115] Weighted average prediction: weighted average prediction is performed based on the spatial distance or correlation between adjacent voxels and the current voxel.

[0116] Residual Coding: Prediction residuals can be encoded using a variety of coding methods, such as:

[0117] Scalar quantization + entropy coding: The residual is scalar quantized and then entropy coding (such as arithmetic coding or Huffman coding) is used for lossless compression.

[0118] Vector quantization: Multiple residual values ​​are combined into a vector and then vector quantized to further improve compression efficiency.

[0119] Context-Adaptive Entropy Coding (CAE): The efficiency of entropy coding depends on the characteristics of the data distribution. The probability distribution of the encoded data may vary for different contexts (for example, different values ​​of adjacent voxel attributes). Context-Adaptive Entropy Coding aims to adaptively select the optimal entropy coding model based on the context, thereby maximizing entropy coding efficiency. This involves context modeling and entropy coding model selection.

[0120] Context modeling is achieved by using the encoded information of neighboring voxels as the context of the current voxel. The context information can be: the attribute values ​​of neighboring voxels and the encoding mode of neighboring voxels.

[0121] Attribute values ​​of neighboring voxels: for example, the color, transparency, and size of neighboring voxels.

[0122] Coding mode of neighboring voxels: For example, whether neighboring voxels use predictive coding, palette coding, etc.

[0123] Entropy coding model selection: Adaptively select the entropy coding model based on different contexts, such as probability model parameter adaptation and code table switching.

[0124] The probability model parameter adaptation is reflected in the fact that for adaptive entropy encoders such as arithmetic coding, the parameters of the probability model are dynamically adjusted according to context information.

[0125] Code table switching is reflected in the switching of different code tables according to context information for code table-based entropy encoders such as Huffman coding.

[0126] The technical effects are as follows: In order to improve the efficiency of spatial dimension compression, the present invention introduces spatial context modeling technology. On the one hand, a predictive coding method is adopted to predict the attribute value of the current voxel using the attribute values ​​of adjacent voxels, and the prediction residual is encoded. The prediction method can select mean prediction, median prediction or weighted average prediction, and adjacent voxels can come from the same level or adjacent levels. On the other hand, a context adaptive entropy coding (CAE) method is adopted to construct a context model based on the encoded information of adjacent voxels, and adaptively select the optimal entropy coding model based on the context model, such as arithmetic coding or Huffman coding, to further improve coding efficiency.

[0127] In some embodiments, see Figure 4 , performing inter-frame context modeling on the first picture representation can be the following process:

[0128] Step 41: In response to the instrument picture being a sequence frame picture, motion compensation is performed on voxels of a reference frame to obtain a predicted frame corresponding to the first picture representation.

[0129] Step 42: Perform residual coding on the predicted frame.

[0130] Step 43: Encode the key frames in the sequence frame pictures, and only encode the difference information for the remaining frames.

[0131] In some embodiments, the inter-frame context modeling (for sequential frame animation) is performed as follows: specifically including motion compensation, residual coding, and key frame coding.

[0132] Principle: Instrument UIs contain numerous sequential frame animations, such as rotating pointers and flashing indicator lights. There is often a high degree of temporal redundancy between adjacent frames. Inter-frame context modeling aims to exploit this temporal redundancy for compression.

[0133] Motion compensation (MC-Motion Compensation) involves motion estimation and motion compensation.

[0134] Motion estimation involves estimating the motion vectors of voxels between adjacent frames. A motion vector represents the displacement of a voxel between two frames.

[0135] Motion estimation can be performed using a variety of algorithms, such as: Block matching: This method divides the current frame into several blocks, searches for the block most similar to the current block in the reference frame, and calculates the displacement of the block as a motion vector. Optical flow: This method estimates the motion vector of each pixel in the image, providing a more detailed description of the image's motion.

[0136] Motion compensation involves performing motion compensation on the voxels of the reference frame based on the estimated motion vector to obtain a predicted frame for the current frame. The purpose of motion compensation is to eliminate redundant information caused by inter-frame motion.

[0137] The idea behind residual coding is that there is still a difference between the motion-compensated predicted frame and the actual frame. This difference is called the residual. Because motion compensation has eliminated most of the inter-frame redundancy, the information entropy of the residual is typically much smaller than that of the original frame, allowing for more efficient encoding of the residual.

[0138] The residual coding method can be encoded using the same coding method as the spatial prediction residual, such as scalar quantization + entropy coding or vector quantization.

[0139] Key frame coding involves key frame selection, difference frame coding and key frame interval.

[0140] Idea: Sequence frame animation usually has a certain periodicity or repetitiveness. Key frame encoding aims to use this temporal periodicity or repetitiveness for compression.

[0141] Keyframe selection involves selecting a small number of keyframes for complete encoding. Keyframes can be the start frame of a sequence animation, periodically repeated frames, or frames with significant content changes.

[0142] Difference frame coding is embodied in that for the remaining frames (difference frames), only the difference information between the frame and the most recent key frame is encoded. The difference information can be a motion vector, a residual, or a change in a voxel attribute value.

[0143] The keyframe interval reflects that the interval between keyframes can be adjusted according to the characteristics of the animation and the compression requirements. The smaller the keyframe interval, the higher the image quality, but the lower the compression ratio; the larger the keyframe interval, the higher the compression ratio, but the image quality may decrease.

[0144] The technical effect is that: for the instrument sequence frame animation, this application introduces inter-frame context modeling technology. On the one hand, the motion compensation (MC) method is used to estimate the inter-frame motion vector, perform motion compensation on the voxels, eliminate temporal redundancy, and encode the motion compensation residual. Motion estimation can use block matching method or optical flow method. On the other hand, the key frame encoding method is used to select a small number of key frames for complete encoding, and the remaining frames encode the difference information with the key frames, further improving the compression efficiency of the sequence frame animation.

[0145] In some embodiments, see Figure 5 , parameterized encoding of the first picture representation may be performed as follows:

[0146] Step 51: extracting vector parameters of the vector graphic elements in the instrument image; the vector parameters include at least one of a straight line, an arc, a polygon, and text.

[0147] Step 52: Encode the extracted vector parameters.

[0148] The method of parametric coding (for vector graphic elements) is as follows: specifically, it includes vector parameter coding and procedural generation.

[0149] The principle is as follows: Instrument UIs contain numerous vector graphics elements, such as lines, arcs, polygons, and text. Vector graphics elements are structured and parameterized. Parametric encoding leverages these parametric characteristics for efficient encoding, avoiding pixel-level voxelization and encoding.

[0150] Vector parameter coding includes vector graphics recognition, parameter extraction and vector parameter coding.

[0151] Vector graphics recognition involves identifying the vector graphics elements in the instrument image on the encoding side. This can be accomplished using image processing algorithms such as edge detection, contour extraction, and shape matching.

[0152] Parameter extraction involves extracting parameter information from the identified vector graphic elements, for example, a straight line: starting point coordinates (x1, y1), end point coordinates (x2, y2).

[0153] Arc: center coordinates (cx, cy), radius r, starting angle θ1, ending angle θ2.

[0154] Polygon: a sequence of vertex coordinates [(x1, y1), (x2, y2), ..., (xn, yn)].

[0155] Text: text content (string), font, font size, color, position, etc.

[0156] Vector parameter encoding is embodied in: encoding the extracted vector parameter information. The vector parameter can be compressed using scalar quantization + entropy coding or vector quantization.

[0157] Procedural Generation includes decoding-side generation and avoiding pixel-level decoding.

[0158] Decoding-side generation involves procedurally generating vector graphics elements based on the decoded vector parameter information using predefined rules and algorithms. For example, a line drawing algorithm generates a line based on its start and end point coordinates; an arc drawing algorithm generates an arc based on its center, radius, angle, and other parameters.

[0159] Avoiding pixel-level decoding is reflected in the following: procedural generation avoids pixel-level voxelization and decoding of vector graphic elements, significantly improving the decoding efficiency and rendering quality of vector graphic elements.

[0160] The technical effect is to introduce parametric encoding technology for common vector graphics elements in instrument images. Firstly, a vector parameter encoding method is used to identify and extract parameter information of vector graphics elements at the encoding end, such as the starting and ending points of a line, the center point and radius of an arc, and then encode the vector parameters. Secondly, a procedural generation method is used at the decoding end to procedurally generate vector graphics based on the vector parameter information using predefined rules and algorithms, avoiding pixel-level decoding and improving the decoding efficiency and rendering quality of vector graphics elements.

[0161] Summary: Context-aware coding technology is an important component of this application's technical solution. It achieves efficient compression of instrument images by fully leveraging their spatial and temporal redundancy and structural characteristics. Different context modeling methods can be flexibly combined to adapt to different instrument UI elements and animation scenarios, further improving compression performance and decoding efficiency.

[0162] In some embodiments, after obtaining the first image code, the first image code is decoded based on voxel rasterization at the instrument image display end, and the decoded instrument image is displayed at the instrument image display end.

[0163] In some embodiments, decoding speed is crucial to achieve a smooth UI experience on resource-constrained instrument products. This application adopts an efficient decoding method based on rasterization and optimizes it for hierarchical sparse voxel representation, aiming to maximize decoding efficiency and reduce CPU load.

[0164] The comparison between traditional pixel-level decoding and voxel rasterization is as follows:

[0165] The traditional image decoding process is usually pixel-level, that is, the decoder calculates the color value of each pixel pixel by pixel and then writes the pixel data to the frame buffer. This pixel-level decoding method is computationally intensive and inefficient, especially with high-resolution images and complex dynamic effects, which significantly increases the CPU load. Voxel rasterization is an object-space rendering method that directly converts voxelized geometric objects (voxel units in this invention) into pixels on the screen, avoiding the redundant calculations of pixel-by-pixel decoding.

[0166] The voxel rasterization process includes voxel projection, depth sorting, and pixel filling.

[0167] Voxel projection involves projecting the voxel units of the hierarchical sparse voxel representation into screen space. The projection process takes into account camera parameters (e.g., focal length, optical center, distortion coefficients) and viewport transformations. The projected voxels may cover multiple pixels in screen space.

[0168] Depth sorting is reflected in the following aspects: Since voxels are three-dimensional objects (in two-dimensional images, they can be considered as area units with depth information), in order to ensure the correct occlusion relationship, it is necessary to perform depth sorting on the voxels projected into the screen space. Depth sorting can use a variety of algorithms, such as:

[0169] Z-buffer algorithm: Maintain a depth buffer (Z-buffer) to record the depth value of each pixel. When rasterizing each voxel, compare the voxel's depth value with the depth value already in the Z-buffer. Only when the voxel's depth value is less than the depth value in the Z-buffer, the pixel color and Z-buffer are updated.

[0170] Painter's Algorithm: Sorts voxels by depth value from far to near, and then draws the voxels in the sorted order. The voxels drawn later will overwrite the voxels drawn earlier.

[0171] Morton code-based sorting: Morton codes are used to sort voxels for efficient depth sorting. This invention uses direction-dependent Morton code sorting to ensure the correct rendering order at all viewing angles.

[0172] The sorting based on Morton code is described as follows:

[0173] Depth sorting is a key step in the voxel rasterization process. It determines the order in which voxels are drawn on the screen and directly affects the correctness of the occlusion relationships in the rendered results. Traditional pixel-by-pixel depth sorting algorithms (such as the Z-buffer algorithm) require maintaining a depth buffer and performing pixel-level depth comparisons and updates, which is computationally intensive. Morton code-based sorting is a more efficient depth sorting method. It leverages the spatial indexing properties of Morton codes (also known as Z-order curves) to map voxels in three-dimensional space to one-dimensional space and sort them according to the size of the Morton codes, thereby approximately achieving depth sorting.

[0174] The principle of Morton code (Z-order curve) is as follows:

[0175] Space-Filling Curve: A Morton code is a space-filling curve that maps points in a multidimensional space (such as two-dimensional or three-dimensional space) to a one-dimensional space while preserving the spatial proximity of the points as much as possible. The Z-order curve is a common space-filling curve, named for its shape resembling the letter "Z."

[0176] Bit Interleaving: The Morton code generation process is based on bit interleaving. For a point (x, y) in two-dimensional space, its Morton code is obtained by interleaving the binary bits of the x and y coordinates. For example, if x = 5 (binary 101) and y = 2 (binary 010), the bit interleaving process of its Morton code is as follows:

[0177] x:101;

[0178] y:010;

[0179] Morton code: 100110 (binary 110110, decimal 54).

[0180] For a point (x, y, z) in three-dimensional space, its Morton code is obtained by interleaving the binary bits of the x, y, and z coordinates.

[0181] Spatial indexing properties: Morton codes have excellent spatial indexing properties. Spatially adjacent points tend to have adjacent Morton codes. Therefore, sorting by Morton code size can approximately reflect the spatial relationship of points, allowing for approximate depth sorting.

[0182] The depth sorting process based on Morton code is as follows:

[0183] Calculate the depth of the voxel center point: For each voxel to be sorted, calculate the depth value (Z coordinate) of its center point in the camera coordinate system.

[0184] Calculate the Morton code of the voxel center point: Convert the 3D coordinates (x, y, z) of the voxel center point to a Morton code. Typically, only the integer portion of the x, y, and z coordinates is used for Morton code calculation.

[0185] Sort by Morton code: Sort voxels based on the size of the Morton code at the voxel center. Voxels with smaller Morton codes are placed at the front, and voxels with larger Morton codes are placed at the back.

[0186] Rasterization drawing: Rasterize and draw voxels in sequence according to the order of Morton code sorting.

[0187] Furthermore, direction-dependent Morton code sorting can be adopted.

[0188] Direction-dependent Morton Order Sorting is described as follows:

[0189] Problem: Basic Morton code sorting is only an approximate depth sorting method. It can cause sorting errors at certain viewing angles, leading to incorrect occlusion relationships. For example, when the viewing angle is nearly parallel to the Z axis, Morton code sorting may not guarantee the correct depth order.

[0190] Direction-dependence is reflected in the following: To address the limitations of basic Morton code sorting, this invention adopts direction-dependent Morton code sorting. The core idea is to select different Morton code sorting methods based on the observation direction (the direction of the camera optical axis) to ensure more accurate depth sorting under various viewing angles.

[0191] The observation direction classification is reflected in the following: Based on the positive and negative signs of the camera optical axis, the observation direction is divided into eight cases (corresponding to the eight quadrants of 3D space). For example, when the camera optical axis points to +X, +Y, +Z, it belongs to the (+, +, +) type; when the camera optical axis points to -X, +Y, +Z, it belongs to the (-, +, +) type, and so on.

[0192] Direction-dependent Morton code generation is reflected in the following: for each type of observation direction, an optimized Morton code generation method is predefined. Different Morton code generation methods are mainly reflected in the order of bit interleaving and the positive and negative signs of the coordinate axes. For example, for an observation direction of type (+, +, +), the standard Morton code generation method (xyz sequential bit interleaving) can be used; for an observation direction of type (-, +, +), the bit interleaving order or the positive and negative signs of the coordinate axes can be adjusted to make the Morton code sorting more consistent with the depth order under that perspective.

[0193] The runtime Morton code selection is reflected in the following: at runtime, according to the current camera's optical axis direction, the observation direction type is determined, and the corresponding direction-dependent Morton code generation method is selected. The Morton code of the voxel is calculated, and then sorted and rasterized.

[0194] The advantages are as follows:

[0195] More accurate depth sorting: Direction-dependent Morton code sorting can dynamically adjust the sorting method according to the viewing angle, more accurately reflecting the depth order of voxels, reducing sorting errors and improving rendering quality.

[0196] High efficiency: Direction-dependent Morton code sorting still maintains the high efficiency of Morton code sorting, with low computational complexity and easy hardware acceleration.

[0197] The technical effects are as follows: In order to achieve efficient and accurate depth sorting, this application adopts a sorting method based on Morton codes, and introduces direction-dependent optimization for different viewing angles. Basic Morton code sorting achieves approximate depth sorting by calculating the Morton code of the voxel center point and sorting it. In order to solve the sorting errors that may occur in basic Morton code sorting under specific viewing angles, the present invention proposes a direction-dependent Morton code sorting technology. This technology classifies the observation direction into eight types according to the positive and negative signs of the camera optical axis direction, and predefines an optimized Morton code generation method for each type, including the bit interleaving order and the adjustment of the positive and negative signs of the coordinate axes. During runtime, the corresponding Morton code generation method is selected according to the current camera optical axis direction, and the voxel Morton code is calculated and sorted to ensure more accurate depth sorting at various viewing angles and improve rendering quality.

[0198] Direction-dependent Morton code sorting is a key component of the present invention's efficient decoding process. While ensuring depth sorting accuracy, it leverages the efficiency of Morton codes, avoids complex pixel-level depth comparisons and updates, and provides strong support for real-time voxel rasterization.

[0199] Pixel filling involves filling the pixel area covered by each voxel projected into screen space based on its color and transparency attributes. The pixel filling process also requires consideration of color blending: If multiple voxels project onto the same pixel, color blending, such as alpha blending, is required based on the voxel's transparency.

[0200] And Texture Mapping (optional): If the voxels store texture information, texture mapping can be applied during the pixel filling process to increase image details and realism.

[0201] The advantages of voxel rasterization are as follows:

[0202] Avoid pixel-level decoding: Voxel rasterization renders directly at the voxel level, avoiding the traditional pixel-level decoding process and significantly reducing the amount of computation.

[0203] Efficient transparency handling: Voxel rasterization naturally supports transparency blending and can efficiently handle transparent elements commonly found in instrument UIs.

[0204] Hardware acceleration friendly: Voxel rasterization can make full use of the parallel processing capabilities of hardware acceleration units such as GPU to further improve rendering speed.

[0205] The technical effects are as follows: In order to improve the decoding speed, the present invention adopts an efficient decoding method based on rasterization. Voxel rasterization projects the voxel units represented by hierarchical sparse voxelization directly into the screen space for rendering, avoiding the traditional pixel-level decoding process. The rasterization process includes three steps: voxel projection, depth sorting, and pixel filling. Depth sorting adopts a sorting algorithm based on direction-dependent Morton codes to ensure the correct occlusion relationship. The pixel filling process supports color mixing and texture mapping. Voxel rasterization can significantly reduce the amount of calculation, efficiently handle transparency, and make full use of hardware acceleration capabilities.

[0206] Furthermore, the logic of pre-calculation and caching is adopted in this application.

[0207] Pre-computation and Caching are introduced as follows:

[0208] Applicable scenarios: For static elements in the instrument UI, such as background images, static icons, static text, etc., these elements usually do not change during the UI operation.

[0209] The pre-calculation process is reflected in: during the initialization phase of the instrument software, these static elements are pre-decoded and pre-rendered, and the decoded pixel data or rendering results are cached in the memory.

[0210] Caching strategies include: Frame buffer caching: Pixel data for static elements is cached directly in the frame buffer. During UI rendering, pixel data can be read directly from the frame buffer, avoiding repeated decoding and rendering. Texture caching: Static elements are cached as texture objects in GPU memory. During UI rendering, hardware-accelerated texture mapping units can be used for fast rendering.

[0211] Runtime advantage: When the UI is running, for static elements, pre-calculated results are read directly from the cache without the need for real-time decoding and rendering, which significantly reduces runtime decoding overhead, lowers CPU and GPU load, and improves the overall performance of the UI.

[0212] The technical effect is as follows: To further reduce runtime decoding overhead, this invention utilizes pre-computation and caching technology for static elements in the instrument UI, such as background images and static icons. During the initialization phase of the instrument software, static elements are pre-decoded and pre-rendered, and the decoded results are cached in memory or GPU memory. During UI runtime, static elements are read directly from the cache, avoiding repeated decoding and rendering, significantly improving decoding efficiency and reducing resource consumption.

[0213] Furthermore, hardware acceleration support (Hardware Acceleration Support) is also adopted in this application, such as adopting a hardware acceleration unit and a corresponding hardware acceleration strategy.

[0214] Hardware acceleration units are reflected in: Modern chip platforms usually integrate various hardware acceleration units to accelerate graphics and image processing tasks, such as:

[0215] Image Processing Unit (IPU): A hardware unit dedicated to image processing that can efficiently perform image filtering, color space conversion, scaling, rotation, and other operations.

[0216] General-Purpose GPU: It has powerful parallel computing capabilities and can accelerate various graphics rendering and compute-intensive tasks, including voxel rasterization, texture mapping, shader calculations, etc.

[0217] Dedicated hardware decoder: Some chip platforms integrate hardware decoders to accelerate the decoding of specific image formats (such as JPEG and PNG).

[0218] The hardware acceleration strategy is reflected in: fully utilizing the hardware acceleration capabilities of the chip platform to accelerate the voxel rasterization and decoding process:

[0219] GPU-accelerated voxel rasterization: The voxel rasterization process (voxel projection, depth sorting, pixel filling) is handed over to the GPU for parallel processing, fully utilizing the parallel computing capabilities of the GPU.

[0220] IPU accelerated image processing: Use the IPU to accelerate image pre-processing operations such as color space conversion, scaling, filtering, etc.

[0221] Hardware decoder acceleration: If the chip platform provides a hardware decoder, you can use the hardware decoder to accelerate the decoding of specific image formats.

[0222] The technical benefits are as follows: To maximize decoding performance, this invention fully leverages the chip platform's hardware acceleration capabilities. Hardware acceleration includes accelerating image preprocessing operations using the image processing unit (IPU), accelerating voxel rasterization rendering using a general-purpose GPU, and accelerating the decoding of specific image formats using dedicated hardware decoders. This hardware acceleration significantly improves decoding speed, reduces CPU load, and enables a smoother instrument UI experience.

[0223] Summary: Efficient decoding is a key component of the technical solution of this application. It achieves high-speed decoding and smooth rendering of instrument images through the synergy of multiple technical means such as voxel rasterization, pre-computation and caching, and hardware acceleration support, laying a solid foundation for realizing high-performance UI on resource-constrained instrument products.

[0224] Furthermore, this application provides a complete tool chain to facilitate developers to use the technical solutions of this application: such as image preprocessing tools, voxel conversion tools and decoding libraries.

[0225] Image preprocessing tools: Integrates DiffPic and AutoClip functions for extracting frame differences and cropping transparent areas.

[0226] Voxel conversion tool: Convert images in PNG, JPG and other formats into ictd format with hierarchical sparse voxel representation. This tool can be configured with parameters.

[0227] For example: Octree level depth: controls the voxel refinement and compression ratio. Context encoding: selects the appropriate spatial context and inter-frame context encoding. Color encoding: selects the appropriate color encoding method (average color, palette, SH coefficient). Target platform parameters: optimize for different chip platforms.

[0228] Decoding library: Provides an efficient ICTD format decoding library that can be integrated into the instrument software.

[0229] By adopting the technical solution of this application, the following significant technical effects can be achieved:

[0230] Significantly improve compression efficiency: The combination of hierarchical sparse voxel representation and context-aware coding can significantly improve the compression ratio of instrument images, which is expected to be 30%-70% higher than existing technical solutions, greatly reducing Flash storage space usage and shortening Flash reading time.

[0231] Significantly improve decoding speed: The efficient decoding method based on rasterization and hardware acceleration support can significantly improve image decoding speed, which is expected to be 50%-200% higher than the existing technical solutions, significantly reduce CPU load, and improve the smoothness and responsiveness of the instrument UI.

[0232] Smoother instrument UI experience: Higher frame rates and faster response speeds provide a smoother and more natural instrument UI experience, especially in complex dynamic effects and high-resolution scenes.

[0233] Reduce hardware costs: More efficient compression and decoding methods reduce the requirements for chip platform performance, making it possible to achieve a smooth instrument UI on low-end domestic chips, thereby reducing hardware costs.

[0234] Reduce development difficulty and shorten development cycle: The complete tool chain and configurable parameters reduce the difficulty for developers to adapt to different chip platforms and shorten the development cycle.

[0235] Good scalability and flexibility: The hierarchical structure and modular design make the technical solution scalable and flexible, allowing for easy functional expansion and performance optimization to adapt to new requirements for future instrument UI development.

[0236] Furthermore, this application adopts a collaborative mechanism of four major technical means:

[0237] The core technical solution of this application consists of four key technologies: hierarchical sparse voxelized image representation, context-aware encoding, efficient decoding, and a toolchain. These four components do not exist in isolation, but rather work together to build an efficient and high-performance instrument image compression and decoding system. Ultimately, they achieve high compression efficiency, high decoding speed, low resource usage, and high-quality rendering.

[0238] 1. Hierarchical sparse voxel image representation (basic framework):

[0239] Core role: As the basic framework and data representation of the entire technical solution, hierarchical sparse voxelized image representation provides a structured and content-adaptive data foundation for subsequent context-aware encoding and efficient decoding.

[0240] Collaborative mechanism: The basis for efficient data compression: Through hierarchical structure and adaptive voxel partitioning, spatial redundant information in instrument images is effectively removed, and pixel matrix images are converted into a more compact voxel representation, laying the foundation for subsequent context-aware encoding.

[0241] The key to supporting efficient decoding: Voxel representation organizes image data into independent voxel units, allowing the decoder to perform raster rendering based on voxels, avoiding the traditional pixel-level decoding process and creating conditions for efficient decoding.

[0242] Hardware-acceleration-friendly data structure: Voxelized representation organizes image data into a regular voxel grid, which is easy for hardware acceleration units such as GPUs to perform parallel processing, facilitating hardware-accelerated decoding.

[0243] 2. Context-aware coding (compression engine):

[0244] Core Function: As the compression engine of the entire technical solution, context-aware coding further explores and utilizes spatial context redundancy, temporal context redundancy, and structured feature redundancy in instrument images based on hierarchical sparse voxel representation to maximize compression efficiency.

[0245] Synergy mechanism:

[0246] Reliance on voxel representation: Various context-aware coding techniques (predictive coding, CAE, motion compensation, keyframe coding, parametric coding) are all built on hierarchical sparse voxel representation, using voxels as the basic unit for context modeling and data prediction.

[0247] Improve voxel attribute encoding efficiency: Through spatial context modeling, predictive coding and CAE can more efficiently encode voxel attribute values ​​such as color and transparency, reducing the amount of data required for attribute encoding.

[0248] Compressed sequence frame animation: Through inter-frame context modeling, motion compensation, residual coding and key frame coding, instrument sequence frame animation can be efficiently compressed to reduce the amount of animation data.

[0249] Compressed vector graphic elements: Through parametric encoding, vector parameter encoding and procedural generation, vector graphic elements in the instrument UI can be efficiently compressed, avoiding inefficient pixel-level encoding of vector graphic elements.

[0250] 3. Efficient decoding (rendering engine):

[0251] Core Function: As the rendering engine of the entire technical solution, efficient decoding is responsible for quickly decoding and rendering compressed hierarchical sparse voxel data to the screen, ensuring the smoothness and responsiveness of the instrument UI.

[0252] Collaborative mechanism: Rasterization using voxel representation: Voxel rasterization technology directly renders data represented by hierarchical sparse voxel representation, avoiding the intermediate link of pixel-level decoding and improving decoding speed.

[0253] Benefit from pre-computation and caching: Pre-computation and caching techniques can reduce runtime decoding overhead and further improve decoding efficiency, especially when processing static instrument elements.

[0254] Rely on hardware acceleration: Hardware acceleration supports making full use of the chip platform's GPU, IPU and other hardware acceleration units to accelerate voxel rasterization and decoding processes, thereby maximizing decoding performance.

[0255] 4. Tool chain (development support):

[0256] Core role: As the development support for the entire technical solution, the tool chain provides a complete set of tools to facilitate developers to apply the technical solution of the present invention to actual instrument product development and optimize and adjust it according to specific needs.

[0257] Collaborative mechanism: Serving the compression process: The image preprocessing tool and voxel conversion tool support converting instrument images in various common formats into the ictd format with hierarchical sparse voxel representation, and support configuring various compression parameters, such as octree hierarchy depth, context encoding method, color encoding method, etc.

[0258] Serving the decoding process: The decoding library provides efficient ICTD format decoding function, is easy to integrate into the instrument software, and supports hardware accelerated decoding.

[0259] Parameter configuration and optimization: The parameter configuration function provided by the tool chain allows developers to flexibly adjust compression and decoding parameters according to different chip platforms and UI requirements to obtain optimal performance and effects.

[0260] These four technical means work together to produce the technical effects of the present invention, such as high compression efficiency, high decoding speed, low resource usage, and high-quality rendering.

[0261] High compression efficiency: Hierarchical sparse voxel representation + context-aware coding => effectively removes spatial redundancy, temporal redundancy, and structural feature redundancy => significantly improves compression ratio, reduces storage space usage, and shortens Flash read time.

[0262] High decoding speed: Voxel rasterization + pre-computation and caching + hardware acceleration support => avoid pixel-level decoding, reduce runtime calculations, and fully utilize hardware acceleration capabilities => significantly improve decoding speed, reduce CPU load, and achieve a smooth UI experience.

[0263] Low resource usage: Hierarchical sparse voxel representation => reduces memory usage; sliding window mechanism => limits GPU memory usage; efficient decoding => reduces CPU and GPU load => reduces hardware resource requirements, supporting deployment on resource-constrained instrument products.

[0264] High-quality rendering: Hierarchical sparse voxel representation => preserves image details; direction-dependent Morton code sorting => ensures correct occlusion relationships; SH coefficient encoding => supports view-dependent effects => achieves high-quality, realistic instrument UI rendering effects.

[0265] In summary, the four technical components of hierarchical sparse voxelized image representation, context-aware encoding, efficient decoding, and toolchain form an organic whole. Hierarchical sparse voxelized representation is the foundation, context-aware encoding is the engine, efficient decoding is the guarantee, and the toolchain is the support. They rely on and promote each other, jointly realizing the efficiency, high performance, and practicality of the technical solution of this invention. They solve the bottleneck problem of existing instrument image display technology solutions on resource-constrained platforms and provide more advanced technical options for the development of future instrument UIs.

[0266] See Figure 6 , Figure 6 Schematic diagram of an embodiment of an instrument image compression and context-aware coding system provided by the present application. The instrument image compression and context-aware coding system 60 includes a processor 61 and a memory 62 coupled to the processor 61;

[0267] The memory 62 is used to store computer programs, and the processor 61 is used to execute the computer programs to implement the following method:

[0268] The instrument image is hierarchically sparsely voxelized to obtain a first image representation; and the first image representation is context-aware encoded to obtain a first image encoding.

[0269] In some embodiments, the processor 61 is further configured to execute a computer program to implement the method of any of the above embodiments.

[0270] See Figure 7 , Figure 7 1 is a schematic diagram of the structure of an embodiment of a computer-readable storage medium provided by the present application. The computer-readable storage medium 70 is used to store a computer program 71. When the computer program 71 is executed by a processor, it is used to implement the following method:

[0271] The instrument image is hierarchically sparsely voxelized to obtain a first image representation; and the first image representation is context-aware encoded to obtain a first image encoding.

[0272] In some embodiments, when the computer program 71 is executed by a processor, it is also used to implement the method of any of the above embodiments.

[0273] In the several embodiments provided in this application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the modules or units is merely a logical functional division. In actual implementation, other division methods may be used, such as combining or integrating multiple units or components into another system, or ignoring or not implementing certain features.

[0274] If the integrated units in the above other embodiments are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) or a processor to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0275] The above description is only an implementation method of the present application and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the description and drawings of this application, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A method for instrument image compression and context-aware encoding, characterized in that: The method comprises: Perform hierarchical sparse voxel representation on the instrument image to obtain a first image representation; Context-aware encoding is performed on the first picture representation to obtain a first picture encoding.

2. The method according to claim 1, characterized in that The step of performing hierarchical sparse voxel representation on the instrument image to obtain a first image representation includes: A quadtree data structure is used to hierarchically divide the space of the instrument image to construct a basic framework for hierarchical voxel representation; the root node of the basic framework corresponds to the complete image area of ​​the instrument image, and each subsequent layer of nodes represents a four-equal sub-region division of the area covered by its parent node; Adaptively determining the voxel size of each local area of ​​the image according to the content complexity of the local area of ​​the image, forming corresponding voxels, and using the voxels as leaf nodes in the quadtree; and performing attribute encoding on each voxel to obtain the first image representation; wherein the attributes include color information and transparency information.

3. The method according to claim 2, characterized in that The content complexity is evaluated by at least one indicator among color variance, gradient amplitude and information entropy.

4. The method according to claim 2, characterized in that Adaptively determining the voxel size of each local area of ​​the image according to the content complexity of the local area of ​​the image to form corresponding voxels includes: When the content complexity of the local image region exceeds a threshold, the local image region is subdivided; otherwise, the subdivision is stopped, and then the voxel size of each local image region is determined to form corresponding voxels.

5. The method according to claim 2, characterized in that The attribute encoding of each voxel includes: Color information is encoded according to the color attributes of the local area of ​​the image corresponding to each voxel; wherein the color attributes include color change, color type, and appearance rendering; the color change corresponds to average color encoding, the color type corresponds to color palette encoding, and the appearance rendering corresponds to spherical harmonic function coefficient encoding.

6. The method according to claim 1, characterized in that The performing context-aware encoding on the first picture representation to obtain a first picture encoding includes: Spatial context modeling, inter-frame context modeling, and parametric encoding are performed on the first picture representation to obtain a first picture encoding.

7. The method according to claim 6, characterized in that The performing spatial context modeling on the first image representation includes: Use the attribute values ​​of the encoded adjacent voxels to predict the attribute value of the current voxel; And according to different contexts, an optimal entropy coding model is adaptively selected, and the entropy coding model is used to perform entropy coding on the current voxel.

8. The method according to claim 6, characterized in that The performing inter-frame context modeling on the first picture representation includes: In response to the instrument picture being a sequence frame picture, performing motion compensation on voxels of a reference frame to obtain a predicted frame corresponding to the first picture representation; Performing residual coding on the predicted frame; And the key frames in the sequence frame pictures are encoded, and only the difference information is encoded for the remaining frames.

9. The method according to claim 6, characterized in that The parametrically encoding the first picture representation includes: Extracting vector parameters of the vector graphic elements in the instrument image; the vector parameters include at least one of a straight line, an arc, a polygon, and text; Encode the extracted vector parameters.

10. The method according to claim 1, characterized in that After obtaining the first picture code, the method further includes: The first image code is decoded at the instrument image display end based on voxel rasterization, and the decoded instrument image is displayed at the instrument image display end.

Images