Image decoding method, system, device, storage medium and program product

Abstract

The invention discloses an image decoding method, system and device and a storage medium, and relates to the technical field of image coding and decoding. The method comprises the following steps: obtaining a plurality of compressed data by calculating a shader; for a first work group of the plurality of work groups, color values of respective pixels in the first image block are calculated by threads in the first work group based on first compressed data of the plurality of compressed data. According to the scheme, ASTC block decoding is realized by adopting a calculation shader program, and the method is not limited by a special hardware unit any more, and has higher universality.

Description

Technical Field

[0001] This application relates to the field of image encoding and decoding technology, and in particular to an image decoding method, system, device, storage medium, and program product. Background Technology

[0002] ASTC (Adaptive Scalable Texture Compression) is a standardized texture compression format that supports compressing texture image blocks of different sizes into fixed 128-bit ASTC blocks.

[0003] In related technologies, it is necessary to directly sample the ASTC block through a dedicated hardware unit to decode the texture image block. However, this image decoding method relies on a dedicated hardware unit and has poor versatility. Summary of the Invention

[0004] This application provides an image decoding method, system, device, storage medium, and program product. The technical solutions provided by this application are as follows: According to one aspect of the embodiments of this application, an image decoding method is provided, the method comprising: Multiple compressed data are obtained through a compute shader, which includes multiple workgroups. The compressed data is obtained by converting the ASTC blocks of the image blocks. Different compressed data correspond to different image blocks, and each compressed data is obtained by a thread in a different workgroup. For the first workgroup among multiple workgroups, the color value of each pixel in the first image block is calculated by the thread in the first workgroup based on the first compressed data among multiple compressed data; wherein, the image block corresponding to the first compressed data is the first image block.

[0005] According to one aspect of the embodiments of this application, an image decoding method is provided, the method comprising: Obtain the compressed file, which includes the ASTC blocks of multiple image patches; The ASTC blocks of multiple image blocks are converted into formats that the computation shader supports, resulting in multiple compressed data, with different compressed data corresponding to different image blocks; Multiple workgroups in the scheduled compute shader decode multiple compressed data sets, with different compressed data sets decoded by threads in different workgroups. These multiple compressed data sets are used by threads in multiple workgroups to calculate the color value of each pixel in multiple image patches.

[0006] According to one aspect of the embodiments of this application, an image decoding system is provided, the system including a first processor, a first memory, a second processor, and a second memory; wherein, the first memory stores a compressed file, the compressed file including ASTC blocks of multiple image blocks respectively; The first processor is used to convert the ASTC blocks of multiple image blocks into a format that the computation shader supports, to obtain multiple compressed data, with different compressed data corresponding to different image blocks; and to write the multiple compressed data into the second memory. The first processor is also used to schedule multiple workgroups in the compute shader to decode multiple compressed data; The second processor, in response to the scheduling of the first processor, retrieves multiple compressed data from the second memory through multiple workgroups, wherein different compressed data are retrieved by threads in different workgroups; The second processor is also configured to, for the first work group among multiple work groups, calculate the color value of each pixel in the first image block based on the first compressed data among multiple compressed data through the threads in the first work group; wherein the image block corresponding to the first compressed data is the first image block.

[0007] According to one aspect of the embodiments of this application, a computer device is provided, the computer device including a processor and a memory, the memory storing a computer program, the computer program being loaded and executed by the processor to implement the above-described image decoding method.

[0008] According to one aspect of the embodiments of this application, a computer-readable storage medium is provided, which stores a computer program that is loaded and executed by a processor to implement the above-described image decoding method.

[0009] According to one aspect of the embodiments of this application, a computer program product is provided, which includes a computer program executed by a processor to implement the above-described image decoding method.

[0010] The technical solutions provided in this application have at least the following beneficial effects: By using computation shaders, each workgroup acquires compressed data, allowing one workgroup to calculate the color values ​​of pixels within a single image patch, while different workgroups calculate the color values ​​of pixels within different image patches. On one hand, since the ASTC blocks of the image patch are converted into a format that supports direct acquisition by the computation shader (i.e., compressed data), the above scheme supports ASTC block decoding using computation shader programs, no longer limited to dedicated hardware units, and thus has greater versatility. On the other hand, by utilizing the parallel nature of the computation shader workgroups, multiple image patches can be decoded simultaneously, improving the throughput of the image decoding process. Attached Figure Description

[0011] Figure 1 This is a schematic diagram of an image decoding system provided in one embodiment of this application; Figure 2 This is a flowchart of an image decoding method provided in one embodiment of this application; Figure 3 This is a flowchart of an image decoding method provided in another embodiment of this application; Figure 4 This is a flowchart of the decoding process of a thread in a workgroup in a compute shader provided in one embodiment of this application; Figure 5 This is a flowchart of the decoding process of an image decoding system provided in one embodiment of this application; Figure 6 This is a structural block diagram of a computer device provided in one embodiment of this application. Detailed Implementation

[0012] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0013] ASTC is one of the efficient texture compression formats standardized by the Khronos organization. It uses a fixed 128-bit encoding method for each compressed block and makes a trade-off between image quality and bitrate by using variable block size (e.g., from 4×4 to 12×12).

[0014] In related technologies, ASTC decoding methods typically include: Method 1, direct sampling by the hardware texture decoding unit in the GPU (Graphics Processing Unit); Method 2, decoding via a proprietary decoding path driven by the GPU manufacturer; Method 3, uploading the uncompressed texture after software decoding on the CPU (Central Processing Unit).

[0015] However, Method 1 cannot be applied to GPUs / platforms lacking ASTC hardware support, resulting in poor versatility. Method 2 relies on driver decoding configured by each GPU manufacturer, leading to poor controllability and cross-platform consistency. Method 3 uses software decoding on the CPU side, which has limited throughput, high power consumption, and poor real-time decoding performance.

[0016] The technical solution provided in this application provides a method for decoding ASTC blocks based on GPU computing shaders. This method does not depend on specific hardware decoding units or drivers from specific vendors, and can achieve high throughput. This will be described in more detail in the following embodiments.

[0017] Please refer to Figure 1 The diagram illustrates an image decoding system 100 provided in one embodiment of this application. The system 100 includes a first processor 10, a first memory 20, a second processor 30, and a second memory 40.

[0018] In some embodiments, the system 100 is disposed in a computer device, which may be an electronic device such as a mobile phone, tablet computer, multimedia playback device, PC (Personal Computer), wearable device, in-vehicle terminal device, VR (Virtual Reality) device, AR (Augmented Reality) device, MR (Mixed Reality) device, etc., and this application does not limit it.

[0019] In some embodiments, the first processor 10 is a CPU. Optionally, the first processor 10 is coupled to a first memory 20, which, exemplarily, is the CPU's memory.

[0020] In some embodiments, the second processor 30 is a GPU. Optionally, the second processor 30 is coupled to the second memory 40, which, exemplarily, is GPU memory, such as video memory.

[0021] In this embodiment, the first processor 10 only handles basic format conversion, file header parsing, and scheduling, while the computational shader in the second processor 30 performs parallel decoding of the ASTC block. This approach ensures controllability and cross-platform consistency while reducing the load on the first processor 10, improving the throughput of the decoding process, and facilitating reuse across different graphics APIs (Application Programming Interfaces). The following embodiments will describe this from the perspectives of both the first processor 10 and the second processor 30.

[0022] Please refer to Figure 2 The diagram illustrates a flowchart of an image decoding method according to an embodiment of this application. The execution entity for each step of the method is a first processor, such as a CPU. The method includes at least one of the following steps 210-230.

[0023] Step 210: Obtain the compressed file, which includes the ASTC blocks of each of the multiple image blocks.

[0024] An ASTC block of an image block refers to an ASTC format block obtained by encoding an image block.

[0025] In some embodiments, the compressed file is an ASTC file, which includes an ASTC file header and ASTC blocks for each of the multiple image blocks. In some embodiments, the multiple image blocks all belong to a first image, that is, the multiple image blocks are image blocks in the same image. Optionally, the first image is a texture image, and the image blocks are texture image blocks.

[0026] Step 220: Convert the ASTC blocks of each of the multiple image blocks into a format that the computation shader supports, to obtain multiple compressed data.

[0027] Different compressed data correspond to different image blocks. That is to say, image blocks, ASTC blocks and compressed data are mutually corresponding. One ASTC block is converted from one image block, and one compressed data is converted from one ASTC block.

[0028] In some embodiments, for the portion of the compressed file after the file header, a set number of bytes is used as the byte length when reading the ASTC block, thereby reading the ASTC blocks of each of the multiple image blocks separately. For example, if the ASTC block is 16 bytes (128 bits), the set number of bytes is 16 bytes.

[0029] In some embodiments, the compute shader supports processing formats including unsigned integers of a set length, the set length being determined according to actual needs, such as 32 bits. Step 220 includes: for an ASTC block of any one of a plurality of image blocks, reinterpreting the ASTC block as at least one unsigned integer of a set length as compressed data. For example, for a 16-byte ASTC block, the ASTC block is reinterpreted as four 32-bit unsigned integers (uvec4) to obtain compressed data.

[0030] Step 230: Schedule multiple workgroups in the compute shader to decode multiple compressed data.

[0031] In this process, different compressed data are decoded by threads in different workgroups, and multiple compressed data are used by threads in multiple workgroups to calculate the color value of each pixel in multiple image patches. That is to say, step 230 includes: scheduling one workgroup in the computation shader to decode one compressed data, and multiple workgroups to decode different compressed data respectively, thereby completing the decoding of multiple compressed data.

[0032] The process of decoding compressed data is the process of calculating the color value of the pixel in the image block corresponding to the compressed data. For a more detailed explanation of this process, please refer to the embodiment on the second processor side below, which will not be repeated here.

[0033] The technical solution provided in this application, by scheduling computation shaders, allows one workgroup to decode compressed data at a time, enabling one workgroup to calculate the color value of a pixel in an image block, while different workgroups calculate the color values ​​of pixels in different image blocks. On one hand, since the ASTC block of the image block is converted into a format that supports computation shader processing, the above solution supports ASTC block decoding using computation shader programs, no longer limited to dedicated hardware units, and has greater versatility. On the other hand, by utilizing the parallel nature of computation shader workgroups, multiple image blocks can be decoded simultaneously, improving the throughput of the image decoding process.

[0034] In some embodiments, the plurality of image blocks are obtained by dividing a first image block; that is, the compressed file is a compressed file of the first image. The method further includes at least one of the following steps S11 to S14 (not shown in the figures).

[0035] S11, parse the header of the compressed file to determine the size of the first image and the size of the image blocks.

[0036] Below is a brief introduction to the composition of the file header of a compressed file.

[0037] In some embodiments, the header of a compressed file includes at least one of the following fields: magic number field, image size field, and block size field.

[0038] The magic number field is used to indicate the type of compressed file. For example, the magic number field of an ASTC compressed file is a fixed magic number 0x5CA1AB13, which is stored in little-endian and is represented in the file as the byte sequence 13 AB A1 5C, used to identify ASTC compressed files.

[0039] The image size field indicates the size of the first image, which may include the number of pixels covered in the first direction (xsize), the number of pixels covered in the second direction (ysize), and the number of pixels covered in the third direction (zsize). The first, second, and third directions are perpendicular to each other. xsize can also be called the width of the first image, ysize can also be called the height of the first image, and zsize can also be called the depth of the first image. Optionally, xsize, ysize, and zsize are each stored using 3 bytes (e.g., little-endian 24 bits). Optionally, if the first image is a 2D image, zsize is 1 (some files use 0 to represent 1). For example, if xsize = 1024, ysize = 512, and zsize = 1, then the first image includes a total of 1024 × 512 pixels.

[0040] The block size field indicates the size of the image block, which may include the number of pixels covered in the first direction (block_x), the number of pixels covered in the second direction (block_y), and the number of pixels covered in the third direction (block_z). ​​Here, block_x can also be referred to as the width of the image block, block_y as the height of the image block, and block_z as the depth of the image block. Optionally, block_x, block_y, and block_z are each stored using 1 byte. Optionally, when the image block is a 2D image block, block_z is 1. For example, if block_x=8, block_y=8, and block_z=1, then the image block includes a total of 8×8 pixels. For example, since each ASTC block is fixed at 128 bits, the average bitrate per pixel is approximately 128 / (block_x) block_y block_z) bpp (bit-per-texel, bits per pixel); for example, for an 8×8 image block, the bit rate is 2 bpp.

[0041] Additionally, it should be noted that when the first image is a texture image, the pixels mentioned in this application may also be referred to as texels.

[0042] In some embodiments, a compressed file is read from the first memory described above.

[0043] In some embodiments, step S11 includes the following steps S111 to S113 (not shown in the figures).

[0044] Step S111: Parse the header of the compressed file to determine the magic number field, image size field, and block size field.

[0045] Step S112: Verify the magic number field.

[0046] The purpose of verifying the magic number field is to verify whether the compressed file is of the expected type. For example, you can determine whether the compressed file is an ASTC compressed file that needs to be decompressed by verifying whether the magic number field is a fixed magic number of 0x5CA1AB13.

[0047] Step S113: If the magic number segment verification passes, determine the size of the first image based on the image size field, and determine the size of the image block based on the block size field.

[0048] Step S12: Calculate grid dimension information based on the size of the first image and the size of the image patch.

[0049] Grid dimension information is used to indicate how the first image is divided into multiple image blocks.

[0050] In some embodiments, the grid dimension information includes the number of image blocks in at least two directions, where the number of image blocks in a certain direction refers to the number of image blocks covered by the first image in that direction.

[0051] Optionally, the grid dimension information includes the number of image blocks in the first direction, the number of image blocks in the second direction, and the number of image blocks in the third direction. The number of image blocks in the first direction is blocksX = ceil(xsize / block_x), the number of image blocks in the second direction is blocksY = ceil(ysize / block_y), and the number of image blocks in the third direction is blocksZ = ceil(zsize / block_z), where ceil represents rounding up.

[0052] S13, create an input object based on grid dimension information. The input object is used to hold multiple compressed data.

[0053] Furthermore, the above method also includes: uploading multiple compressed data to the input object at once or in batches, and ensuring that the compressed data in the input object are byte-aligned (e.g., all are uvec4) and endian (e.g., all are big-endian or little-endian).

[0054] In some embodiments, the input object is an integer image (e.g., uimage2D, format RGBA32UI), and each compressed data can be stored as an element in the integer image.

[0055] In some embodiments, multiple compressed data are organized into a two-dimensional or three-dimensional array according to grid dimension information and then written to the input object.

[0056] For example, when the grid dimension information is blocksX×blocksY, the size of the integer image is blocksX×blocksY, and the compressed data of each uvec4 is organized into an array of size blocksX×blocksY. Accordingly, the compressed data of each uvec4 is stored as an element in the integer image.

[0057] Of course, the input object can also be a TBO (Texture Buffer Object), an SSBO (Shader Storage Buffer Object), etc., and this application does not limit it.

[0058] In some embodiments, an input object is created in a second memory.

[0059] S14, create an output object according to the size of the first image. The output object is used to carry the decoding results of multiple compressed data.

[0060] In some embodiments, an output image with the same size as the first image is created as the output object. Since the decoding result of multiple compressed data completely contains the pixel values ​​of each pixel in the first image, a pixel in the output image is used to carry the pixel value of a pixel in the first image.

[0061] The output image (such as image2D) can be in the format of RGBA8, RGBA16F, RGBA32F, or other FBO (FrameBuffer Object) color appendages, or image2D (Array), etc. This application does not limit the format.

[0062] After the pixel values ​​of each pixel in the first image are completely stored in the output image, the output image can be directly used as the input texture of the subsequent rendering pipeline on the second processor side, or it can be read back to the first processor side when needed. This application does not limit its specific use.

[0063] In some embodiments, an output object is created in a second memory.

[0064] Both the input and output objects mentioned above can be accessed by the fragment shader.

[0065] In the above embodiment, the first processor prepares the input and output objects by parsing the file header of the compressed file, which provides the necessary guarantee for the second processor's compressed data decoding process. Moreover, the first processor only needs to undertake the file header parsing and resource creation work, and the load is low.

[0066] In some embodiments, the method further includes setting the number of threads included in the workgroup based on the size of the image patch.

[0067] In some embodiments, the number of pixels included in an image block is determined based on the size of the image block, and the number of threads included in the workgroup is set to be consistent with the number of pixels included in the image block.

[0068] For example, when the size of the image block is block_x×block_y, the number of threads included in the workgroup is set to block_x×block_y.

[0069] In the above embodiment, the first processor sets the number of threads in the workgroup according to the size of the image block, so that one thread in the workgroup corresponds to one pixel in the image block, thereby ensuring that pixel-level parallel decoding can be achieved on the second processor side.

[0070] In some embodiments, the method further includes: generating an error message or a degradation strategy when the size of the image patch is a size that does not support processing, or when the image patch division method indicated by the grid dimension information does not support processing, wherein the degradation strategy is used to indicate the use of other image decoding schemes.

[0071] Please refer to Figure 3 The diagram illustrates a flowchart of an image decoding method provided in another embodiment of this application. The execution entity for each step of the method is a second processor, such as a GPU. The method includes at least one of the following steps 310-320.

[0072] Step 310: Obtain multiple compressed data sets through the compute shader.

[0073] The computation shader consists of multiple workgroups. Compressed data is obtained by converting the ASTC blocks of image blocks. Different compressed data correspond to different image blocks, and each compressed data is obtained by a thread in a different workgroup.

[0074] In some embodiments, a compute shader retrieves multiple compressed data sets from an input object located in a second memory. For example, a thread in one compute shader workgroup can retrieve one of the multiple compressed data sets by executing the texelFetch / imageLoad function once; multiple threads in multiple workgroups can retrieve multiple compressed data sets by executing the texelFetch / imageLoad function.

[0075] For example, the aforementioned workgroups have identifiers, and different workgroups have different identifiers. The identifier of a workgroup is consistent with the block coordinates of the image block corresponding to the workgroup. Therefore, based on the identifier of the workgroup, each workgroup can obtain compressed data corresponding to an image block. The block coordinates of the image block are used to indicate the image block from the first image divided into multiple image blocks.

[0076] Step 320: For the first workgroup among multiple workgroups, calculate the color value of each pixel in the first image block based on the first compressed data among multiple compressed data through the thread in the first workgroup.

[0077] The image block corresponding to the first compressed data is the first image block.

[0078] The first working group can be any one of multiple working groups, and this application does not limit this.

[0079] The first compressed data is the compressed data assigned to the first working group for decoding. Optionally, the block coordinates of the first image block corresponding to the first compressed data are consistent with the identifier of the first working group.

[0080] Optionally, the color value of a pixel may include components from multiple color channels.

[0081] The technical solution provided in this application uses a computation shader to acquire compressed data in a way that one workgroup is used to calculate the color value of a pixel in an image block, while different workgroups are used to calculate the color values ​​of pixels in different image blocks. On one hand, since the ASTC block of the image block is converted into a format that supports direct acquisition by the computation shader (i.e., compressed data format), the above solution supports ASTC block decoding using computation shader programs, no longer limited to dedicated hardware units, and has greater versatility. On the other hand, by utilizing the parallel nature of the computation shader workgroups, multiple image blocks can be decoded simultaneously, improving the throughput of the image decoding process.

[0082] In some embodiments, the number of threads in the first workgroup is equal to the number of pixels included in the first image block, and each thread in the first workgroup is used to calculate the color value of a pixel in the first image block.

[0083] In other words, one thread in the first workgroup is used to calculate the color value of a pixel in the first image block, and different threads in the first workgroup are used to calculate the color values ​​of different pixels in the first image block.

[0084] For example, threads in the first workgroup have identifiers, and different threads in the same workgroup have different identifiers. The identifier of a thread and the coordinates of the pixel corresponding to the thread within the image block are consistent. Therefore, each thread can calculate the pixel value of the pixel corresponding to that thread based on the thread identifier.

[0085] In the above embodiment, the threads in the first working group can decode the color values ​​of different pixels in the first image block in parallel. Therefore, this solution, through the architecture of the working group and threads of the computing shader, can not only achieve parallel decoding at the image block level, but also parallel decoding at the pixel level, thereby improving the throughput of the image decoding process.

[0086] In some embodiments, the first compressed data includes a configuration field, a weight quantization value field, and an endpoint quantization value field, and the inverse quantization data of the first compressed data includes weights in the weight grid and color endpoint pairs of the first image patch. Step 320 includes the following steps 322-328 ( Figure 3 (not shown in the figure) The steps mentioned in the various embodiments below are all executed by threads in the first working group, and will not be repeated hereafter.

[0087] Step 322: Based on the configuration fields, determine the size of the weighted grid, the weight quantization mode information, and the CEM (Color Endpoint Mode) information.

[0088] A grid cell in the weight grid is used to carry a weight. The weight quantization mode information is used to indicate the target weight quantization mode used in the process of quantizing the weights in the weight grid into a weight quantization value field. The CEM information is used to indicate the target encoding method used for the color endpoints of the first image block. The color endpoints of the first image block are used to interpolate and determine the color values ​​of the pixels in the first image block.

[0089] In some embodiments, threads in the first workgroup extract information or fields (such as the configuration field, weight quantization value field, and endpoint quantization value field) from the first compressed data using a bit extraction function. This bit extraction function supports extracting a set length of bits from the first compressed data in a single operation. Since the set length can be set and changed as needed by technicians, this bit extraction function can reduce memory accesses and branching. For example, in the case of uvec4 first compressed data, this bit extraction function can be a bitfieldExtract / bitfieldReverse function that supports bit extraction across 32-bit boundaries.

[0090] Below is a brief introduction to the composition of the configuration fields.

[0091] The configuration field can be considered as a field located in the header of the first compressed data, which is used to describe the encoding mode of the first image block. It may include at least one of the following: block mode field, number of partitions information, seed value of the first image block, blank extended block identifier, CEM information, and CCS (Color Component Selector) information.

[0092] The block mode field includes at least one of the following: the size of the weight grid, the weight quantization mode information, and the biplane indication information.

[0093] The size of the weighted grid includes the number of grid cells it covers in each direction. For example, a weighted grid with a size of 3×3 contains 9 grid cells and supports 9 weights.

[0094] The weight quantization mode information indicates the target weight quantization mode used in the process of quantizing the weights in the weight grid into the weight quantization value field. For example, the weight quantization mode information is an index of the target weight quantization mode. Different weight quantization modes represent different quantization methods for the weights and have different indices.

[0095] The dual-plane indication information is used to indicate whether a dual-weighted plane is used in the encoding process for the first image block.

[0096] In the case where the encoding process of the first image block does not employ a dual-weight plane, the color values ​​of the pixels in the first image block are encoded with the same weights for the components in different color channels.

[0097] In the case where the encoding process of the first image block does not employ a dual-weight plane, the color value of a pixel in the first image block contains a component of a color channel that is encoded with a separate weight.

[0098] The partition count information is used to indicate the number of partitions included in the first image block. Each partition in the first image block has a separate pair of color endpoints.

[0099] The seed value of the first image block is used to calculate the partition index of each pixel in the first image block. The partition index of a pixel is used to indicate the partition to which the pixel belongs.

[0100] The blank extension block identifier is a special code used to indicate whether the first image block is an empty block with a constant color value or an extension block that extends beyond the boundary of the first image.

[0101] CEM information is used to indicate the target encoding method used for the color endpoint pairs of the first image patch. The target encoding method is the encoding method used to encode the color endpoint pairs of the first image patch into the endpoint quantization value field.

[0102] CCS information is used to indicate color channels encoded with separate weights when a dual-weight plane is used in the encoding process of the first image block.

[0103] In some embodiments, before step 322, the aforementioned blank extension identifier is parsed first. If the blank extension identifier indicates that the first image block is not an empty block / extended block, then step 322 is executed. If the blank extension identifier indicates that the first image block is an empty block / extended block, the first image block can be filled with the set color value or marked according to the specification.

[0104] Step 324: Based on the weight quantization mode information, dequantize from the weight quantization value field to obtain the weights in the weight grid.

[0105] In some embodiments, step 324 includes the following steps: 1. Find the target weight quantization parameter corresponding to the target weight quantization mode from the weight quantization parameter lookup table.

[0106] The weight quantization parameter lookup table is used to record the correspondence between weight quantization modes and weight quantization parameters.

[0107] For example, the weight quantization parameter lookup table records the index of each weight quantization mode and the weight quantization parameter corresponding to each index. Therefore, the target weight quantization parameter can be found from the weight quantization parameter lookup table based on the index of the target weight quantization mode contained in the weight quantization mode information.

[0108] Optionally, the target weight quantization parameters include at least one of the following: the number of bits occupied by the quantized value of each weight in the weight quantization value field, and encoding information. The encoding information is used to indicate the encoding method of the weights in the weight grid, such as trit / quint (3-ary / 5-ary) encoding.

[0109] 2. Based on the target weight quantization parameters, the weights in the weight grid are obtained by dequantization from the weight quantization value field.

[0110] In some embodiments, the step includes at least one of the following steps (1) to (3).

[0111] (1) According to the target weight quantization parameter, the number of bits occupied by each weight in the weight quantization value field, and extract the quantization value of each weight in the weight grid from the weight quantization value field.

[0112] Optionally, according to the encoding method indicated by the encoding method information, dequantization calculation is performed on the quantized value of the weight extracted from the weight quantization value field to obtain the weight in the weight grid.

[0113] (2) Determine the inverse quantization lookup table corresponding to the encoding method indicated by the encoding method information. The inverse quantization lookup table corresponding to a certain encoding method records the correspondence between the weights and the weight quantization values ​​under that encoding method. The inverse quantization result of the quantization value in the weight quantization value field can be retrieved from the inverse quantization lookup table corresponding to the encoding method. For example, in the case of trit / quint encoding, the compact 3 / 5 base symbol sequence in the weight quantization value field can be restored to the weight value by using the inverse quantization lookup table corresponding to trit / quint encoding.

[0114] Optionally, the above inverse quantization results can be directly used as weights in the weight grid.

[0115] (3) Map the dequantization result obtained from the dequantization lookup table to the standard weight range to obtain the weight in the weight grid.

[0116] The standard weight range is set by technical personnel as needed, such as 0~64 (0 represents an actual weight of 0, 1 represents an actual weight of 1 / 64, and so on, 64 represents an actual weight of 64 / 64=1), and this application does not limit it.

[0117] Optionally, the process of mapping the dequantization result to the standard weight range can also be accomplished by looking up a table. For example, since the weight values ​​that can be retrieved in the dequantization lookup table corresponding to the encoding method are predictable, a standardized lookup table that records the mapping relationship between these weight values ​​and weight values ​​within the standard range can be pre-established. Then, based on the dequantization result, the weights in the weight grid are retrieved from the standardized lookup table. These weights are all within the standard weight range, thereby minimizing the computational load of the image decoding process.

[0118] In some embodiments, when the dual-plane indication information indicates that the encoding process for the first image block uses a dual-weight plane, the weights in the two weight grids (i.e., two sets of weights) can be dequantized from the weight quantization value field. For example, if the weight grid size is 3×3 and the encoding process uses a dual-weight plane, 18 weights can be dequantized from the weight quantization value field, which can then be filled into the two weight grids respectively. In the above case, the method further includes the following step: determining the target color channel indicated by the CCS information from each color channel according to the CCS information in the configuration field. Wherein, the first weight grid in the two weight grids corresponds to the target color channel, and the second weight grid corresponds to the remaining color channels other than the target color channel.

[0119] Of course, if the dual-plane instruction information indicates that the encoding process for the first image block uses a dual-weight plane, a single weight grid can still be used. However, each grid cell in the weight grid needs to be filled with two weights, one of which corresponds to the target color channel and the other corresponds to the other color channels.

[0120] For details on the specific role of the weights in the weight grid, and how the correspondence between the weight grid / weights and color channels is specifically reflected, please refer to the examples below, which will not be elaborated here.

[0121] In the above embodiment, the target weight quantization parameters required for the weight dequantization process are obtained by using a pre-configured weight quantization parameter lookup table, thereby improving the dequantization efficiency of the process and thus increasing the throughput of the entire image decoding process.

[0122] In some embodiments, the aforementioned weight quantization value field is extracted as follows: the length of the configuration field is calculated, and the length of the configuration field is used as an offset relative to the header of the first compressed data to determine the starting position of the weight quantization value field in the first compressed data. The weight quantization value field is then extracted from this starting position, or the quantization value in the weight quantization value field is extracted.

[0123] Step 326: Based on CEM information, dequantize from the endpoint quantization value field to obtain the color endpoint pair of the first image block.

[0124] In some embodiments, step 326 includes at least one of the following steps: 1. Extract the quantization values ​​of the color endpoint pairs of the first image block from the endpoint quantization value field according to the target encoding method.

[0125] In some embodiments, the endpoint quantization value field is extracted as follows: the remaining bits in the first compressed data, excluding the weight quantization value field of the configuration field, are determined as the endpoint quantization value field. That is, the length of the endpoint quantization value field is equal to the length of the first compressed data minus the sum of the lengths of the configuration field and the weight quantization value field.

[0126] Since the starting position and length of the quantized value of the color endpoint pair obtained under a certain encoding method are determined by the specification of that encoding method, the quantized value of the color endpoint pair of the first image block can be directly extracted from the endpoint quantized value field based on the target encoding method indicated by the CEM information.

[0127] Since the target encoding method indicated by CEM information is the encoding method for color endpoint pairs, the target encoding method can also be directly called the target color endpoint mode, i.e., the target CEM. The target CEM (target encoding method) can include variants such as L / LA / RGB / RGBA, LDR / HDR and direct / base+offset / delta / scale, which are not limited in this application.

[0128] It should be noted that a color endpoint pair of the first image block includes two color values, which are used for interpolation to determine the color value of the pixels in the first image block, hence the name "color endpoint".

[0129] In some embodiments, the first image block includes at least two partitions, and the color endpoint pairs of the first image block include color endpoint pairs for each of the at least two partitions. The color endpoint pairs of the partitions are used for interpolation to determine the color values ​​of pixels in the partitions. This step includes: determining the quantized values ​​of the color endpoint pairs for each of the at least two partitions from the endpoint quantization value field according to the target encoding scheme.

[0130] 2. Find the target color quantization parameter corresponding to the target encoding method from the color quantization parameter lookup table.

[0131] The color quantization parameter lookup table is used to record the correspondence between encoding methods and color quantization parameters.

[0132] The above encoding method refers to the encoding method for color values, and the above color quantization parameters are parameters used to quantize color values. The target color quantization parameters corresponding to the target encoding method are the parameters used to quantize the endpoint color pairs of the first image block under the target encoding method.

[0133] For example, the color quantization parameter lookup table records the indexes of each encoding method (or each CEM), and the color quantization parameters corresponding to each encoding method's index. Therefore, the target color quantization parameter can be found from the color quantization parameter lookup table based on the encoding method index contained in the CEM information.

[0134] In some embodiments, the endpoint quantization value field, in addition to the quantization values ​​of the color endpoint pairs of the first image block, also includes some configuration information. This configuration information is also used to configure the encoding method of the color endpoint pairs of the first image block, i.e., to configure the target encoding method. Therefore, when looking up the target color quantization parameter from the color quantization parameter lookup table, the input needs to include not only CEM information but also the configuration information in the endpoint quantization value field.

[0135] Optionally, based on the target color quantization parameters, the quantization values ​​of the color endpoint pairs of the first image block are dequantized to obtain the color endpoint pairs of the first image block.

[0136] 3. Based on the target color quantization parameters and the quantization values ​​of the color endpoint pairs of the first image block, the color endpoint pairs of the first image block are retrieved from the color inverse quantization lookup table.

[0137] The color inverse quantization lookup table is used to record the correspondence between the quantized values ​​and the color values ​​under different color quantization parameters.

[0138] In some embodiments, the color inverse quantization lookup table is used to record the correspondence between the combination of the color quantization parameter and the quantized value of the color value, and the color value itself. Therefore, by using the combination of the target color quantization parameter and the quantized value of the color endpoint pair of the first image block as input to the query, the color endpoint pair of the first image block can be found from the color inverse quantization lookup table.

[0139] In some embodiments, the color inverse quantization lookup table includes multiple sub-lookup tables corresponding to different color quantization parameters. Each sub-lookup table records the correspondence between the quantized color value and the color value under the corresponding color quantization parameter. First, based on the target color quantization parameter, a target sub-lookup table corresponding to the target color quantization parameter is determined from the multiple sub-lookup tables. Then, based on the color endpoint pairs of the first image patch, the color endpoint pairs of the first image patch can be found from the target sub-lookup table.

[0140] In some embodiments, the color endpoint pairs of at least two partitions are retrieved from the color dequantization lookup table based on the target color quantization parameter and the quantization values ​​of the color endpoint pairs of at least two partitions.

[0141] In some embodiments, the color endpoint pair of the first image patch obtained from the color inverse quantization lookup table is not the final color endpoint pair used; it can be referred to as the uncorrected color endpoint pair. The method further includes: correcting the uncorrected color endpoint pair according to the target encoding scheme or target CEM indicated by the CEM information to obtain the color endpoint pair of the first image patch. Different encoding schemes require different correction rules, such as blue-contract, bit-transfer-signed, HDR / LDR scaling and offset rules, etc. Therefore, this application does not limit the correction rule.

[0142] In the above embodiment, after extracting the quantization value based on the CEM information, the color endpoint pairs of the first image block are obtained entirely through table lookup. This breaks down the complex calculation and mapping process into multiple table lookup processes, which helps to improve throughput. Furthermore, since no calculation is required, it is also convenient to verify the accuracy of the final table lookup result.

[0143] Step 328: For the first pixel in the first image block, calculate the color value of the first pixel based on the position of the first pixel in the first image block, the weight in the weighted grid, and the color endpoint pair of the first image block.

[0144] The first pixel can be any pixel in the first image block, and this application does not limit it.

[0145] In some embodiments, the color value of a first pixel in a first image block is calculated by a first thread in a first working group, based on the position of the first pixel in the first image block, the weight in the weighted grid, and the color endpoint pair of the first image block.

[0146] The first thread is the thread in the first workgroup corresponding to the first pixel. In some embodiments, the position of the first pixel in the first image block, such as the coordinates of the first pixel within the first image block, is determined according to the identifier of the first thread. For example, the identifier of the first thread is the coordinates of the first pixel within the first image block.

[0147] In some embodiments, the color value of the first pixel is written to the corresponding position in the output object. Furthermore, to ensure visibility of subsequent pipelines, a memory barrier or API synchronization can be inserted into the output object by the first processor. The corresponding position of the first pixel in the output object refers to the position in the output object that corresponds to the position of the first pixel in the first image. For example, if the output object is an output image, and the coordinates of the first pixel in the first image are (x1, y1), then the color value of the first pixel should also be written to position (x1, y1) in the output image.

[0148] In some embodiments, the method further includes: inputting the color value of the first pixel into a transfer function, which maps the color value of the first pixel from a linear space to an sRGB (standard Red Green Blue) color space, thereby converting the color value of the first pixel into an sRGB signal that supports display on a standard sRGB display. Optionally, the sRGB signal is written to the corresponding position in the output object.

[0149] In the above embodiment, the threads in the compute shader workgroup perform configuration field parsing, weight and color endpoint pair decoding on the compressed data, thereby decoding the pixel color values ​​in the image patch. This process is independent of specific hardware and drivers and can be implemented based on the functionality of the compute shader, ensuring the versatility of this solution.

[0150] In some embodiments, step 328 above includes the following steps: 1. Based on the position of the first pixel in the first image block, interpolate the weights in the weight grid to obtain the weight of the first pixel.

[0151] In some embodiments, the step includes: mapping the position of the first pixel in the first image block to a weight grid; obtaining weights from at least two grid cells in the weight grid that are closest to the mapped position of the first pixel, thus obtaining at least two weights; and summing the at least two weights based on the mapped position of the first pixel and the positions of the at least two grid cells in the weight grid to obtain the weight of the first pixel. For example, in the case of 4-neighborhood weighting, the at least two grid cells are the four grid cells (top left, bottom left, top right, and bottom right) in the weight grid that are closest to the mapped position of the first pixel, and the at least two weights are four weights.

[0152] In some embodiments, the position of the grid cell in the weighted grid is represented by coordinate values ​​within the standard coordinate range. Mapping the position of the first pixel in the first image block to the weighted grid means normalizing the coordinates of the first pixel in the first image block to the standard coordinate range to obtain the mapped coordinate values. These mapped coordinate values ​​can reflect the mapped position of the first pixel in the weighted grid.

[0153] The above standard coordinate range is set by technical personnel as needed. For example, it can be set to be consistent with the above standard weight range, such as both the standard coordinate range and the standard weight range being 0~64.

[0154] In some embodiments, the dual-plane indication information indicates that the encoding process for the first image block employs a dual-weight plane. Based on the position of the first pixel in the first image block, weights in the first weight grid are interpolated to obtain a first weight for the first pixel, and weights in the second weight grid are interpolated to obtain a second weight for the first pixel. Alternatively, based on the position of the first pixel in the first image block, weights in the weight grid corresponding to the target color channel are interpolated to obtain a first weight for the first pixel, and weights in the weight grid corresponding to the remaining color channels are interpolated to obtain a second weight for the first pixel.

[0155] 2. Based on the weight of the first pixel, interpolate the color endpoints of the first image block to obtain the color value of the first pixel.

[0156] In some embodiments, the number of partitions indicated by the partition quantity information is 1, and the first image block has only one color endpoint pair. The color value of the first pixel is obtained by interpolating the color endpoint pair of the first image block using the weight of the first pixel. The interpolation formula is as follows: W×F1+(1-W2)×F2, where W is the weight of the first pixel, F1 is one color value in the color endpoint pair, and F2 is the other color value in the color endpoint pair.

[0157] In some embodiments, a first weight of the first pixel is used to interpolate the components of the color endpoint pairs of the first image block in the target color channel to obtain the components of the color value of the first pixel in the target color channel. A second weight of the first pixel is used to interpolate the components of the color endpoint pairs of the first image block in the remaining color channels to obtain the components of the color value of the first pixel in the remaining color channels.

[0158] In the above embodiment, the weight of each pixel in the first image block is not encoded separately in the first compressed data, but is encoded on a coarser grid cell. In the decoding stage, the pixel weight is obtained by interpolation from the grid cell according to the pixel position, thereby ensuring that this application can be applied to decode compressed data with high compression ratio.

[0159] In some embodiments, the first image block includes at least two partitions, and the color endpoint pairs of the first image block include color endpoint pairs for each of the at least two partitions. That is, the number of partitions indicated by the partition number information is greater than 1. In this case, the above method further includes the following steps S21-S22 (not shown in the figures).

[0160] S21, determine the seed value of the first image block from the configuration field.

[0161] The seed value of the first image block is used to calculate the partition index of each pixel in the first image block. The partition index of a pixel is used to indicate the partition to which the pixel belongs.

[0162] In computer science, a "seed value" is an initial value used to generate a pseudo-random number sequence. In the field of video encoding and decoding technology, it can be combined with the pixel position in an image block and converted into a pixel partition index through calculation. However, the process of mapping partition indexes based on seed values ​​and pixel positions involves relatively complex calculations, which can affect decoding efficiency. Therefore, this application solves the above problem by pre-calculating the partition indexes corresponding to each combination of query values ​​(combinations of pixel positions and seed values) and flattening the combination of query values ​​and partition indexes into a partition lookup table.

[0163] S22, based on the position of the first pixel in the first image block and the seed value of the first image block, the partition index of the first pixel is obtained from the partition lookup table. The partition lookup table is used to record the correspondence between the query value combination and the partition index.

[0164] In some embodiments, a partition lookup table is obtained from the R8UI texture buffer of the second processor.

[0165] The query value combination is a combination of pixel position and seed value, and the partition index of the first pixel is used to indicate that the first pixel belongs to the target partition in at least two partitions.

[0166] In some embodiments, the query value combination is mapped to lookup coordinates (seed, localID) in a partition lookup table, thereby retrieving the partition index of the first pixel from the location indicated by the lookup coordinates in the partition lookup table. Here, localID represents the coordinates of the first pixel within the first image block.

[0167] In some embodiments, the partition information retrieved from the partition lookup table is of fixed length, such as 2-bit or 8-bit partition information, which includes the partition index of the first pixel. Based on the number of partitions, the valid bits are extracted from the partition information to obtain the partition index of the first pixel. For example, when the number of partitions is 4, the partition information includes two valid bits; when the number of partitions is 8, the partition information includes three valid bits.

[0168] The above-mentioned step of "interpolating the color endpoint pairs of the first image block based on the weight of the first pixel to obtain the color value of the first pixel" includes: interpolating the color endpoint pairs of the target partition based on the weight of the first pixel to obtain the color value of the first pixel.

[0169] In some embodiments, a first weight of the first pixel is used to interpolate the components of the color endpoint pairs of the target partition in the target color channel to obtain the components of the color value of the first pixel in the target color channel. A second weight of the first pixel is used to interpolate the components of the color endpoint pairs of the target partition in the remaining color channels to obtain the components of the color value of the first pixel in the remaining color channels.

[0170] In the above embodiment, the partition to which the first pixel belongs is determined by looking up the partition lookup table. This method can avoid performing complex partition function calculations inside the computation shader, thereby reducing the complexity and branch divergence of image decoding performed by the computation shader.

[0171] In some embodiments, prior to step 328, the method further includes: if the size of the first image is not divisible by the size of the image block, determining the global coordinates of the first pixel based on the coordinates of the first pixel in the image block, the size of the image block, and the block coordinates of the first image block. That is, the global coordinates of the first pixel are pixelID = blockID. The expression `blockDim + localID` represents the coordinates of the first image block, where `blockID` is the block coordinates, `blockDim` is the size of the image block, and `localID` is the coordinates of the first pixel within the first image block. The global coordinates of the first pixel refer to its coordinates in the coordinate system used by the first image. Based on these global coordinates, it can be determined whether the first pixel exceeds the boundary of the first image. If the first pixel exceeds the boundary, there is no need to perform the color value calculation step described above for the first pixel; in the output object, either no color value is written for the first pixel, or a pre-defined color value is written.

[0172] In some embodiments, after step 326, the method further includes: Write the weights in the weighted grid and the color endpoints of the first image patch to the shared buffer area of ​​the first workgroup.

[0173] The data stored in the shared cache area of ​​the first workgroup can be accessed by the various threads included in the first workgroup.

[0174] Optionally, the weights in the aforementioned weighted grid and the color endpoint pairs of the first image block are calculated by a first thread in the first working group. The method further includes: obtaining the weights in the weighted grid and the color endpoint pairs of the first image block from the shared cache area of ​​the first working group by a second thread in the first working group; and, for a second pixel in the first image block, calculating the color value of the second pixel based on the position of the second pixel in the first image block, the weights in the weighted grid, and the color endpoint pairs of the first image block.

[0175] In the above embodiment, threads in the same workgroup share data such as weights in the weighted grid and color endpoints of the first image block. This utilizes the shared memory feature of threads in the workgroup to avoid repeated parsing of the same data between threads, thereby further improving the decoding efficiency of the computation shader.

[0176] In addition, it should be noted that the various lookup tables mentioned in this application can be generated by the first processor in the initial stage and uploaded to the Texture Nuffer Object (TBO) in the second memory for use by the compute shader, or they can be directly embedded in the compute shader program. This application does not limit this.

[0177] In summary, the technical solution provided in this application requires input objects, lookup tables, output objects, etc., during the image decoding process. The binding relationship between these objects and the computation shader can be abstracted into a unified interface layer for execution management on the first processor side. Therefore, this solution is not limited by the graphics API used by the second processor (GPU) and can be ported to the computation pipeline of various graphics APIs such as OpenGL / Vulkan / DirectX, while keeping the core logic of the computation shader unchanged.

[0178] The following example illustrates the decoding process of a thread in a workgroup within a compute shader. Please refer to [link / reference]. Figure 4 The decoding process includes at least one of the following steps S31 to S38.

[0179] S31, Obtain the first compressed data from the input object.

[0180] S32, parse the configuration fields of the first compressed data to determine the size of the weight grid, weight quantization mode information and CEM information.

[0181] S33, based on the weight quantization mode information, dequantizes the weights in the weight grid from the weight quantization value field of the first compressed data.

[0182] S34, based on CEM information, dequantize from the endpoint quantization value field of the first compressed data to obtain the color endpoint pairs of each of the at least two partitions included in the first image.

[0183] S35: Based on the position of the first pixel in the first image block, interpolate the weights in the weight grid to obtain the weight of the first pixel.

[0184] S36, the query finds that the partition to which the first pixel belongs is the target partition among at least two partitions.

[0185] S37, based on the weight of the first pixel, interpolate the color endpoints of the target partition to obtain the color value of the first pixel.

[0186] S38 writes the color value of the first pixel to the output object.

[0187] An exemplary embodiment of this application also provides an image decoding system. This image decoding system, as... Figure 1 As shown, the system 100 includes a first processor 10, a first memory 20, a second processor 30, and a second memory 40. The first memory 20 stores compressed files, each containing an ASTC block of multiple image blocks.

[0188] The first processor 10 is used to convert the ASTC blocks of multiple image blocks into a format supported by the computation shader to obtain multiple compressed data, with different compressed data corresponding to different image blocks; and to write the multiple compressed data into the second memory 40.

[0189] The first processor 10 is also used to schedule multiple workgroups in the compute shader to decode multiple compressed data.

[0190] The second processor 30 is configured to, in response to the scheduling of the first processor 10, retrieve multiple compressed data from the second memory 40 through multiple workgroups, wherein different compressed data are retrieved by threads in different workgroups.

[0191] The second processor 30 is further configured to, for the first work group among multiple work groups, calculate the color value of each pixel in the first image block based on the first compressed data among multiple compressed data through a thread in the first work group; wherein the image block corresponding to the first compressed data is the first image block.

[0192] It should be noted that, in the above image decoding system, details not described in detail, such as other functions of the first processor and the second processor, can be found in the method embodiments of the first processor side and the second processor side described above, and will not be repeated here.

[0193] The following is a concrete example illustrating the decoding process of an image decoding system. Please refer to [link / reference]. Figure 5 The decoding process includes at least one of the following steps S41 to S46, and the decoding process is described using the example of a CPU as the first processor and a GPU as the second processor.

[0194] S41, the CPU retrieves the compressed file from the first memory.

[0195] S42, the CPU parses the header of the compressed file to determine the size of the first image and the size of the image blocks; and converts the ASTC blocks of each of the multiple image blocks in the compressed file into a format that the computation shader can process, thereby obtaining multiple compressed data.

[0196] S43, the CPU determines the grid dimension information based on the size of the first image and the size of the image block; creates an input object in the second memory according to the grid dimension information, creates an input object in the second memory according to the size of the first image; and uploads multiple compressed data to the input object.

[0197] S44, the GPU obtains and decodes multiple compressed data through the compute shader to get the color value of each pixel in the first image.

[0198] S45, the GPU writes the color values ​​of each pixel in the first image to the output object.

[0199] S46, CPU reads back or saves the output object.

[0200] In summary, the technical solutions provided in this application have the following technical effects: No dedicated hardware decoder required: Real-time decoding of ASTC blocks is achieved on GPUs that only support general-purpose compute shaders.

[0201] Cross-platform consistency: It does not rely on vendor-driven proprietary paths, its behavior is controllable, and regression verification can be performed using reference implementations and test vectors. Furthermore, this solution is entirely controlled by the application side, facilitating cross-platform reproduction and debugging.

[0202] High throughput and low CPU usage: Decoding calculations are executed in parallel on the GPU, significantly reducing the bottleneck and power consumption caused by CPU software decoding.

[0203] Debuggable and maintainable: Key aspects such as compressed data parsing and lookup tables are clearly defined, allowing for problem location by block / pixel and expansion to more block sizes / formats.

[0204] Flexible output: Supports output to multiple formats such as RGBA8, RGBA16F, and RGBA32F, and is easy to connect to subsequent rendering / image processing pipelines.

[0205] An embodiment of this application also provides a computer device that includes the image decoding system described above.

[0206] Please refer to Figure 6 The diagram illustrates a structural block diagram of a computer device provided in one embodiment of this application.

[0207] Typically, computer device 600 includes a processor 601 and a memory 602.

[0208] Processor 601 may include one or more processing cores, such as a quad-core processor, an octa-core processor, etc. Processor 601 may be implemented using at least one hardware form selected from DSP (Digital Signal Processing), FPGA (Field Programmable Gate Array), and PLA (Programmable Logic Array). Processor 601 may also include a main processor and a coprocessor. The main processor, also known as the CPU, is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, processor 601 may integrate a GPU, which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, processor 601 may also include an AI processor, which is used to handle computational operations related to machine learning.

[0209] The memory 602 may include one or more computer-readable storage media, which may be tangible and non-transitory. The memory 602 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In some embodiments, the non-transitory computer-readable storage media in the memory 602 stores a computer program that is loaded and executed by the processor 601 to implement the image decoding method on the first processor side described above, and / or to implement the image decoding method on the second processor side described above.

[0210] Those skilled in the art will understand that Figure 6 The structure shown does not constitute a limitation on the computer device 600, and may include more or fewer components than shown, or combine certain components, or use different component arrangements.

[0211] In some embodiments, a computer-readable storage medium is also provided, which stores a computer program loaded and executed by a processor to implement the image decoding method on the first processor side and / or to implement the image decoding method on the second processor side.

[0212] Optionally, the computer-readable storage medium may include: ROM (Read-Only Memory), RAM (Random-Access Memory), SSD (Solid State Drives), or optical disc, etc. The random access memory may include ReRAM (Resistance Random Access Memory) and DRAM (Dynamic Random Access Memory).

[0213] In some embodiments, a computer program product is also provided, comprising a computer program stored in a computer-readable storage medium, wherein a processor reads from and executes the computer program to implement the image decoding method on the first processor side described above, and / or to implement the image decoding method on the second processor side described above.

[0214] It should be understood that "multiple" as used herein refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. Furthermore, the step numbers described herein are merely illustrative of one possible execution order. In some other embodiments, the steps may not be executed in numerical order, such as two steps with different numbers being executed simultaneously, or two steps with different numbers being executed in the reverse order of the illustration. This application does not limit this.

[0215] The above are merely exemplary embodiments of this application and are not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application shall be included within the protection scope of this application.

Claims

1. An image decoding method, characterized in that, The method includes: Multiple compressed data are obtained through a compute shader, wherein the compute shader includes multiple workgroups, and the compressed data is obtained by converting an image patch into an adaptive scalable texture compression ASTC block. Different compressed data correspond to different image patches, and each compressed data is obtained by a thread in a different workgroup. For the first workgroup among the plurality of workgroups, the color value of each pixel in the first image block is calculated based on the first compressed data among the plurality of compressed data through the thread in the first workgroup; wherein, the image block corresponding to the first compressed data is the first image block.

2. The method according to claim 1, characterized in that, The number of threads in the first workgroup is equal to the number of pixels included in the first image block, and each thread in the first workgroup is used to calculate the color value of a pixel in the first image block.

3. The method according to claim 1 or 2, characterized in that, The first compressed data includes a configuration field, a weight quantization value field, and an endpoint quantization value field; The step of calculating the color value of each pixel in the first image block based on the first compressed data among the plurality of compressed data includes: Based on the configuration fields, the size of the weight grid, weight quantization mode information, and color endpoint mode (CEM) information are determined. A grid cell in the weight grid is used to carry a weight. The weight quantization mode information is used to indicate the target weight quantization mode used in the process of quantizing the weights in the weight grid into the weight quantization value field. The CEM information is used to indicate the target encoding method used for the color endpoint pairs of the first image block. The color endpoint pairs of the first image block are used to interpolate and determine the color values ​​of the pixels in the first image block. Based on the weight quantization mode information, the weights in the weight grid are obtained by dequantization from the weight quantization value field; Based on the CEM information, the color endpoint pairs of the first image block are obtained by dequantization from the endpoint quantization value field; For a first pixel in the first image block, the color value of the first pixel is calculated based on the position of the first pixel in the first image block, the weight in the weighted grid, and the color endpoint pair of the first image block.

4. The method according to claim 3, characterized in that, The step of dequantizing from the weight quantization value field to obtain the weights in the weight grid based on the weight quantization mode information includes: The target weight quantization parameter corresponding to the target weight quantization mode is found in the weight quantization parameter lookup table, wherein the weight quantization parameter lookup table is used to record the correspondence between the weight quantization mode and the weight quantization parameter; Based on the target weight quantization parameters, the weights in the weight grid are obtained by dequantization from the weight quantization value field.

5. The method according to claim 3, characterized in that, The step of obtaining the color endpoint pair of the first image patch by dequantization from the endpoint quantization value field based on the CEM information includes: According to the target encoding method, extract the quantization values ​​of the color endpoint pairs of the first image block from the endpoint quantization value field; The target color quantization parameter corresponding to the target encoding method is found in the color quantization parameter lookup table, wherein the color quantization parameter lookup table is used to record the correspondence between encoding methods and color quantization parameters; Based on the target color quantization parameters and the quantization values ​​of the color endpoint pairs of the first image block, the color endpoint pairs of the first image block are retrieved from the color inverse quantization lookup table. The color inverse quantization lookup table is used to record the correspondence between the quantization values ​​of color values ​​and the color values ​​under different color quantization parameters.

6. The method according to claim 3, characterized in that, The step of calculating the color value of the first pixel based on the position of the first pixel in the first image patch, the weight in the weighted grid, and the color endpoint pairs of the first image patch includes: Based on the position of the first pixel in the first image block, the weights in the weight grid are interpolated to obtain the weight of the first pixel; Based on the weight of the first pixel, the color endpoints of the first image block are interpolated to obtain the color value of the first pixel.

7. The method according to claim 6, characterized in that, The first image block includes at least two partitions, and the color endpoint pairs of the first image block include the color endpoint pairs of each of the at least two partitions; The method further includes: The seed value of the first image block is determined from the configuration field. The seed value of the first image block is used to calculate the partition index of each pixel in the first image block. The partition index of the pixel is used to indicate the partition to which the pixel belongs. Based on the position of the first pixel in the first image block and the seed value of the first image block, the partition index of the first pixel is obtained from the partition lookup table. The partition lookup table is used to record the correspondence between the query value combination and the partition index. The query value combination is a combination of pixel position and seed value. The partition index of the first pixel is used to indicate that the first pixel belongs to the target partition among the at least two partitions. The step of interpolating the color endpoint pairs of the first image block based on the weight of the first pixel to obtain the color value of the first pixel includes: Based on the weight of the first pixel, the color endpoint pairs of the target partition are interpolated to obtain the color value of the first pixel.

8. The method according to claim 3, characterized in that, The method further includes: The weights in the weighted grid and the color endpoints of the first image patch are written to the shared cache area of ​​the first workgroup, wherein the data stored in the shared cache area of ​​the first workgroup can be accessed by the threads included in the first workgroup.

9. An image decoding method, characterized in that, The method includes: Obtain a compressed file, the compressed file comprising multiple image blocks, each with its own ASTC block; Each of the multiple image blocks is converted into an ASTC block that the computation shader supports, resulting in multiple compressed data, with different compressed data corresponding to different image blocks; Multiple workgroups in the computation shader are scheduled to decode the multiple compressed data, wherein different compressed data are decoded by threads in different workgroups, and the multiple compressed data are used by threads in the multiple workgroups to calculate the color value of each pixel in the multiple image blocks.

10. The method according to claim 9, characterized in that, The plurality of image blocks are obtained by dividing the first image; the method further includes: The file header of the compressed file is parsed to determine the size of the first image and the size of the image blocks; Based on the size of the first image and the size of the image block, the grid dimension information is calculated, and the grid dimension information is used to indicate the division method of the first image into the plurality of image blocks; Based on the grid dimension information, an input object is created, which is used to carry the multiple compressed data. An output object is created based on the dimensions of the first image, and the output object is used to carry the decoding results of the multiple compressed data. Both the input object and the output object can be accessed by the compute shader.

11. The method according to claim 10, characterized in that, The method further includes: The number of threads included in the workgroup is set based on the size of the image patch.

12. An image decoding system, characterized in that, The system includes a first processor, a first memory, a second processor, and a second memory; wherein, the first memory stores compressed files, and the compressed files include ASTC blocks of multiple image blocks respectively; The first processor is configured to convert the ASTC blocks of each of the plurality of image blocks into a format supported by the computation shader to obtain a plurality of compressed data, wherein different compressed data correspond to different image blocks; and to write the plurality of compressed data into the second memory. The first processor is further configured to schedule multiple workgroups in the compute shader to decode the multiple compressed data; The second processor is configured to, in response to scheduling by the first processor, retrieve the plurality of compressed data from the second memory through the plurality of workgroups, wherein different compressed data are retrieved by threads in different workgroups; The second processor is further configured to, for the first workgroup among the plurality of workgroups, calculate the color value of each pixel in the first image block based on the first compressed data among the plurality of compressed data, through a thread in the first workgroup; wherein the image block corresponding to the first compressed data is the first image block.

13. A computer device, characterized in that, The computer device includes a processor and a memory, the memory storing a computer program that is loaded and executed by the processor to implement the method as claimed in any one of claims 1 to 8, and / or to implement the method as claimed in any one of claims 9 to 11.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that is loaded and executed by a processor to implement the method as claimed in any one of claims 1 to 8, and / or to implement the method as claimed in any one of claims 9 to 11.

15. A computer program product, characterized in that, The computer program product includes a computer program that is executed by a processor to implement the method as claimed in any one of claims 1 to 8, and / or to implement the method as claimed in any one of claims 9 to 11.

Images