Image decoding methods, systems, devices and storage media

Abstract

This application discloses an image decoding method, system, device, and storage medium, relating to the field of image encoding and decoding technology. The method includes: running a fragment shader to acquire multiple compressed data sets; and determining the color values ​​of each fragment in the drawing area of ​​the fragment shader based on the multiple compressed data sets, using these as the color values ​​of each pixel in a first image. This method uses a fragment shader program in the graphics rendering pipeline to achieve complete image decoding. It can be applied to any device that supports a conventional graphics rendering pipeline, requiring no computational pipeline or dedicated hardware units, and thus possesses strong versatility.

Description

Technical Field

[0001] This application relates to the field of image encoding and decoding technology, and in particular to an image decoding method, system, device and storage medium. Background Technology

[0002] ASTC (Adaptive Scalable Texture Compression) is a standardized texture compression format that supports compressing texture image blocks of different sizes into fixed 128-bit compressed blocks.

[0003] In related technologies, it is necessary to directly sample the compressed block through a dedicated hardware unit to decode the texture image block. However, this image decoding method relies on a dedicated hardware unit and has poor versatility. Summary of the Invention

[0004] This application provides an image decoding method, system, device, and storage medium. The technical solutions provided by this application are as follows:

[0005] According to one aspect of the embodiments of this application, an image decoding method is provided, applied to a first processor, the method comprising:

[0006] Run the fragment shader so that the fragment shader acquires multiple compressed data, wherein the compressed data is obtained by converting compressed blocks of image blocks, different compressed data correspond to different image blocks, and the image blocks corresponding to the multiple compressed data are all obtained by dividing the first image;

[0007] Based on the multiple compressed data, the color value of each fragment in the drawing area of ​​the fragment shader is determined to obtain the color value of each pixel in the first image; wherein the size of the drawing area is the same as the size of the first image.

[0008] According to one aspect of the embodiments of this application, an image decoding method is provided, applied to a second processor, the method comprising:

[0009] A compressed file is obtained, the compressed file comprising compressed blocks for multiple image blocks, each of which is obtained by dividing the first image;

[0010] Each of the multiple image blocks is converted into a format supported by the fragment shader to obtain multiple compressed data, with different compressed data corresponding to different image blocks;

[0011] The multiple compressed data are used by the fragment shader to determine the color value of each fragment in the drawing area, which serves as the color value of each pixel in the first image. The size of the drawing area is the same as the size of the first image.

[0012] According to one aspect of the embodiments of this application, an image decoding system is provided, the system including a first processor, a first memory, a second processor, and a second memory; wherein, the first memory stores a compressed file, the compressed file including compressed blocks of multiple image blocks, each of the multiple image blocks being divided from the first image;

[0013] The first processor is configured to convert the compressed blocks of the plurality of image blocks into a format supported by the fragment shader to obtain a plurality of compressed data, wherein different compressed data correspond to different image blocks; and write the plurality of compressed data into the second memory;

[0014] The second processor is configured to run the fragment shader such that the fragment shader retrieves the plurality of compressed data from the second memory;

[0015] The second processor is further configured to determine, based on the plurality of compressed data, the color value of each fragment in the drawing area of ​​the fragment shader, as the color value of each pixel in the first image; wherein the size of the drawing area is consistent with the size of the first image.

[0016] According to one aspect of the embodiments of this application, a computer device is provided, the computer device including a processor and a memory, the memory storing a computer program, the computer program being loaded and executed by the processor to implement the above-described image decoding method.

[0017] According to one aspect of the embodiments of this application, a computer-readable storage medium is provided, wherein a computer program is stored in the computer-readable storage medium, the computer program being loaded and executed by a processor to implement the above-described image decoding method.

[0018] According to one aspect of the embodiments of this application, a computer program product is provided, the computer program product including a computer program that is executed by a processor to implement the above-described image decoding method.

[0019] The technical solutions provided in this application have at least the following beneficial effects:

[0020] The fragment shader decodes the color values ​​of each pixel in the first image by coloring fragments within the drawing area. Since the multiple compressed data referenced in the coloring process are all converted from compressed blocks of image blocks (i.e., the compressed blocks of image blocks are converted to a format compatible with the fragment shader), and the image blocks corresponding to these compressed data are all derived from the first image, and the size of the drawing area is consistent with the size of the first image, this scheme ensures that the color values ​​of each pixel in the first image are completely decoded even when the fragment shader has completely colored the drawing area. In summary, the above method uses the fragment shader program in the graphics rendering pipeline to achieve complete image decoding. It can be applied to any device that supports a conventional graphics rendering pipeline, requiring no computational pipeline or dedicated hardware units, and thus possesses extremely high versatility. Attached Figure Description

[0021] Figure 1 This is a schematic diagram of an image decoding system provided in one embodiment of this application;

[0022] Figure 2 This is a flowchart of an image decoding method provided in one embodiment of this application;

[0023] Figure 3 This is a flowchart of an image decoding method provided in another embodiment of this application;

[0024] Figure 4 This is a flowchart of a decoding method provided in one embodiment of this application;

[0025] Figure 5 This is a flowchart of a decoding method provided in another embodiment of this application;

[0026] Figure 6 This is a flowchart of the decoding process of an image decoding system provided in one embodiment of this application;

[0027] Figure 7 This is a structural block diagram of a computer device provided in one embodiment of this application. Detailed Implementation

[0028] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0029] ASTC is one of the efficient texture compression formats standardized by the Khronos organization. It uses a fixed 128-bit encoding method for each compressed block and makes a trade-off between image quality and bitrate by using variable block size (e.g., from 4×4 to 12×12).

[0030] In related technologies, ASTC decoding methods typically include: Method 1, direct sampling by the hardware texture decoding unit in the GPU (Graphics Processing Unit); Method 2, decoding via a proprietary decoding path driven by the GPU manufacturer; Method 3, uploading the uncompressed texture after software decoding on the CPU (Central Processing Unit).

[0031] However, Method 1 cannot be applied to GPUs / platforms lacking ASTC hardware support, resulting in poor versatility. Method 2 relies on driver decoding configured by each GPU manufacturer, leading to poor controllability and cross-platform consistency. Method 3 uses software decoding on the CPU side, which has limited throughput, high power consumption, and poor real-time decoding performance.

[0032] The technical solution provided in this application provides a method for compressed block decoding based on GPU fragment shaders. This method does not rely on specific hardware decoding units or specific vendor drivers, nor does it require the use of computing pipelines, such as computing shaders within computing pipelines. It can be applied to older devices or embedded devices that only support graphics rendering pipelines, thereby significantly expanding the deployment coverage of compressed block decoding. This will be described in more detail in the following embodiments.

[0033] Please refer to Figure 1 The diagram illustrates an image decoding system 100 provided in one embodiment of this application. The system 100 includes a first processor 10, a first memory 20, a second processor 30, and a second memory 40.

[0034] In some embodiments, the system 100 is disposed in a computer device, which may be an electronic device such as a mobile phone, tablet computer, multimedia playback device, PC (Personal Computer), wearable device, in-vehicle terminal device, VR (Virtual Reality) device, AR (Augmented Reality) device, MR (Mixed Reality) device, etc., and this application does not limit it.

[0035] In some embodiments, the first processor 10 is a CPU. Optionally, the first processor 10 is coupled to a first memory 20, which, exemplarily, is the CPU's memory.

[0036] In some embodiments, the second processor 30 is a GPU. Optionally, the second processor 30 is coupled to the second memory 40, which, exemplarily, is GPU memory, such as video memory.

[0037] In this embodiment, the first processor 10 only handles basic format conversion and file header parsing, while the fragment shader in the second processor 30 decodes the compressed blocks. This solution simultaneously satisfies real-time performance, compatibility, and maintainability. The following embodiments will describe this from both the first processor 10 and the second processor 30 perspectives.

[0038] Please refer to Figure 2 The diagram illustrates a flowchart of an image decoding method according to an embodiment of this application. The execution entity for each step of the method is a first processor, such as a CPU. The method includes at least one of the following steps 210-220.

[0039] Step 210: Obtain the compressed file, which includes compressed blocks of multiple image blocks.

[0040] Multiple image blocks are derived from the first image; that is to say, the compressed file is a compressed file of the first image.

[0041] In some embodiments, the compressed file is an ASTC file, which includes an ASTC file header and ASTC blocks for each of multiple image blocks. Optionally, the first image is a texture image, the image blocks are texture image blocks, and the compressed block of the image block is the ASTC block of the image block. The technical solution provided by the embodiments of this application enables real-time decoding of ASTC textures on a GPU that does not have an ASTC hardware decoding unit. This will be described in more detail in the following embodiments.

[0042] Step 220: Convert the compressed blocks of the multiple image blocks into the format supported by the fragment shader to obtain multiple compressed data.

[0043] Different compressed data correspond to different image blocks. That is to say, image blocks, compressed blocks, and compressed data are mutually corresponding. One compressed block is converted from one image block, and one compressed data is converted from one compressed block.

[0044] In some embodiments, for the portion of the compressed file after the file header, a set number of bytes is used as the byte length when reading the compressed blocks, so that the compressed blocks of multiple image blocks are read separately. For example, if the compressed file is an ASTC file and the compressed blocks are ASTC blocks, since the ASTC block is 16 bytes (128 bits), the byte count is set to 16 bytes.

[0045] In some embodiments, the fragment shader supports processing formats including unsigned integers of a set length, the set length being determined according to actual needs, such as 32 bits. Step 220 includes: for a compressed block of any one of the plurality of image blocks, reinterpreting the compressed block as at least one unsigned integer of a set length to obtain compressed data. For example, for a 16-byte ASTC block, the compressed block is reinterpreted as four 32-bit unsigned integers (uvec4) to obtain compressed data.

[0046] Multiple compressed data are used by the fragment shader to determine the color values ​​of each fragment in the drawing area, which serve as the color values ​​of each pixel in the first image. The size of the drawing area is consistent with the size of the first image.

[0047] The process by which the fragment shader determines the color value of each fragment in the drawing area to obtain the color value of each pixel in the first image is also the process of decoding and restoring the first image. For a more detailed explanation of this process, please refer to the embodiment on the second processor side below, which will not be repeated here.

[0048] The drawing area of ​​a fragment shader is the region used to hold the fragments that the fragment shader needs to process. This can be the full-screen area of ​​a computer device's screen or a window area. The size of the drawing area includes the number of fragments covered in the first direction, the number of fragments covered in the second direction, and the number of fragments covered in the third direction. The first, second, and third directions are perpendicular to each other. Optionally, when the drawing area is planar, the number of fragments covered in the third direction is 1.

[0049] The size of the first image includes the number of pixels covered in the first direction (xsize), the number of pixels covered in the second direction (ysize), and the number of pixels covered in the third direction (zsize). The first, second, and third directions are perpendicular to each other. xsize can also be called the width of the first image, ysize can also be called the height of the first image, and zsize can also be called the depth of the first image. Optionally, xsize, ysize, and zsize are each stored using 3 bytes (e.g., little-endian 24 bits). Optionally, if the first image is a 2D image, zsize is 1 (some files use 0 to represent 1). For example, if xsize = 1024, ysize = 512, and zsize = 1, then the first image includes a total of 1024 × 512 pixels.

[0050] The size of the drawing area is the same as the size of the first image. That is, the number of fragments covered by the drawing area in the first direction is equal to xsize, the number of fragments covered in the second direction is equal to ysize, and the number of fragments covered in the third direction is equal to zsize.

[0051] The technical solution provided in this application uses a fragment shader to decode the color values ​​of each pixel in a first image by coloring fragments within the drawing area. Since the multiple compressed data referenced in the coloring process are all converted from compressed blocks of image blocks (i.e., the compressed blocks of image blocks are converted to a format compatible with the fragment shader), and the image blocks corresponding to these compressed data are all divided from the first image, and the size of the drawing area is consistent with the size of the first image, this solution can ensure that the color values ​​of each pixel in the first image are completely decoded even when the fragment shader has completely colored the drawing area. In summary, the above method uses a fragment shader program in the graphics rendering pipeline to achieve complete image decoding. It can be applied to any device that supports a conventional graphics rendering pipeline, requiring no computational pipeline or dedicated hardware units, and has extremely high versatility.

[0052] In some embodiments, the above method further includes at least one of the following steps S11 to S12 (not shown in the figures).

[0053] Step S11: Parse the header of the compressed file to determine the size of the first image and the size of the image blocks.

[0054] Below is a brief introduction to the composition of the file header of a compressed file.

[0055] In some embodiments, the header of a compressed file includes at least one of the following fields: magic number field, image size field, and block size field.

[0056] The magic number field is used to indicate the type of compressed file. For example, the magic number field of an ASTC compressed file is a fixed magic number 0x5CA1AB13, which is stored in little-endian and is represented in the file as the byte sequence 13 AB A1 5C, used to identify ASTC compressed files.

[0057] The image size field is used to indicate the size of the first image, which may include xsize, ysize and zsize as described above.

[0058] The block size field indicates the size of the image block, which may include the number of pixels covered in the first direction (block_x), the number of pixels covered in the second direction (block_y), and the number of pixels covered in the third direction (block_z). ​​Here, block_x can also be referred to as the width of the image block, block_y as the height of the image block, and block_z as the depth of the image block. Optionally, block_x, block_y, and block_z are each stored using 1 byte. Optionally, when the image block is a 2D image block, block_z is 1. For example, if block_x=8, block_y=8, and block_z=1, then the image block includes a total of 8×8 pixels. For example, when the compressed block is an ASTC block, since each compressed block is fixed at 128 bits, the average bitrate per pixel is approximately 128 / (block_x). block_y block_z) bpp (bit-per-texel, bits per pixel); for example, for an 8×8 image block, the bit rate is 2 bpp.

[0059] Additionally, it should be noted that when the first image is a texture image, the pixels mentioned in this application may also be referred to as texels.

[0060] In some embodiments, a compressed file is read from the first memory described above.

[0061] In some embodiments, step S11 includes the following steps S111 to S113 (not shown in the figures).

[0062] Step S111: Parse the header of the compressed file to determine the magic number field, image size field, and block size field.

[0063] Step S112: Verify the magic number field.

[0064] The purpose of verifying the magic number field is to verify whether the compressed file is of the expected type. For example, you can determine whether the compressed file is an ASTC compressed file that needs to be decompressed by verifying whether the magic number field is a fixed magic number of 0x5CA1AB13.

[0065] Step S113: If the magic number segment verification passes, determine the size of the first image based on the image size field, and determine the size of the image block based on the block size field.

[0066] Step S12: Set the size of the drawing area to the size of the first image.

[0067] In some embodiments, the dimensions of the first image are written to a UniformBuffer Object (UBO) in a second memory to set the size of the drawing area to the dimensions of the first image. The UBO is used to pass data to the fragment shader.

[0068] The size of the image block is used to map the position of each fragment in the drawing area to the position of each pixel in the first image in its respective image block. The position of each pixel in its respective image block and the inverse quantization data in multiple compressed data are used to determine the color value of each fragment.

[0069] For example, the size of the image block is used to map the position of the first fragment in the drawing area to the first image block to which the first pixel corresponding to the first fragment belongs in the first image, and to map the position of the first pixel in the first image block. The position of the first pixel in the first image block and the inverse quantization data of the first compressed data in the plurality of compressed data are used to determine the color value of the first fragment. The image block corresponding to the first compressed data is the first image block, and the inverse quantization data of the first compressed data is obtained based on the first compressed data. For a detailed explanation of the above mapping and calculation process, please refer to the description on the second processor side, which takes the first fragment as an example, and it will not be repeated here.

[0070] In some embodiments, the size of the image block is written into the above UBO, thereby passing the size of the image block to the fragment shader.

[0071] In the above embodiment, the first processor determines the size of the first image and the size of the image blocks inside the second image by parsing the file header of the compressed file, thus providing the necessary data guarantee for the fragment shader to establish the fragment-pixel mapping relationship.

[0072] In some embodiments, the above method further includes at least one of the following steps S21 to S24 (not shown in the figures).

[0073] S21, Calculate grid dimension information based on the size of the first image and the size of the image patch.

[0074] The grid dimension information is used to indicate how the first image is divided into multiple image blocks.

[0075] In some embodiments, the grid dimension information includes the number of image blocks in at least two directions, where the number of image blocks in a certain direction refers to the number of image blocks covered by the first image in that direction.

[0076] Optionally, the grid dimension information includes the number of image blocks in the first direction, the number of image blocks in the second direction, and the number of image blocks in the third direction. The number of image blocks in the first direction is blocksX = ceil(xsize / block_x), the number of image blocks in the second direction is blocksY = ceil(ysize / block_y), and the number of image blocks in the third direction is blocksZ = ceil(zsize / block_z), where ceil represents rounding up.

[0077] S22, create an input object based on the mesh dimension information. The input object is used to carry the input data of the fragment shader.

[0078] In some embodiments, an input object is created in a second memory.

[0079] S23, write multiple compressed data into the input object.

[0080] In some embodiments, multiple compressed data are written to the input object at once or in batches, and the byte alignment (e.g., all uvec4) and endianness (e.g., all big-endian or little-endian) of the compressed data in the input object are ensured.

[0081] In some embodiments, the input object is an integer image (e.g., uimage2D, format RGBA32UI), and each compressed data can be stored as an element in the integer image.

[0082] In some embodiments, multiple compressed data are organized into a two-dimensional or three-dimensional array according to grid dimension information and then written to the input object.

[0083] For example, when the grid dimension information is blocksX×blocksY, the size of the integer image is blocksX×blocksY, and the compressed data of each uvec4 is organized into an array of size blocksX×blocksY. Accordingly, the compressed data of each uvec4 is stored as an element in the integer image.

[0084] Of course, the input object can also be a TBO (Texture Buffer Object), an SSBO (Shader Storage Buffer Object), etc., and this application does not limit it.

[0085] S24. Create an output object according to the size of the first image. The output object is used to carry the decoding result of the fragment shader on the input data.

[0086] In some embodiments, an output image with the same size as the first image is created as the output object. Since the decoding result of multiple compressed data completely contains the pixel values ​​of each pixel in the first image, a pixel in the output image is used to carry the pixel value of a pixel in the first image.

[0087] The output image can be in the format of RGBA8, RGBA16F, RGBA32F, or other FBO (Frame Buffer Object) color appendages, or image2D (Array), etc. This application does not limit the format.

[0088] After the pixel values ​​of each pixel in the first image are completely stored in the output image, the output image can be directly used as the input texture of the subsequent rendering pipeline on the second processor side, or it can be read back to the first processor side when needed. This application does not limit its specific use.

[0089] In some embodiments, an output object is created in a second memory.

[0090] Both the input and output objects mentioned above can be accessed by the fragment shader.

[0091] In the above embodiment, the first processor completes the preparation of the input and output objects, providing the necessary guarantee for the second processor's compressed data decoding process. Moreover, the first processor only needs to undertake the work of parsing the file header and creating resources, resulting in a low load.

[0092] In some embodiments, the method further includes writing at least one of the following information into the UBO to pass it to the fragment shader: mesh dimension information, layer index of the first image, and sRGB (standard Red Green Blue) switch.

[0093] The layer index of the first image is used to indicate the layer to which the first image belongs. At the same time, when the layer index is passed through UBO, the output object should also be marked as belonging to the layer indicated by the layer index.

[0094] The sRGB switch indicates whether the decoded color values ​​need to be mapped to the sRGB color space.

[0095] In some embodiments, the method further includes: generating an error message or a degradation strategy when the size of the image patch is a size that does not support processing, or when the image patch division method indicated by the grid dimension information does not support processing, wherein the degradation strategy is used to indicate the use of other image decoding schemes.

[0096] Please refer to Figure 3The diagram illustrates a flowchart of an image decoding method provided in another embodiment of this application. The execution entity for each step of the method is a second processor, such as a GPU. The method includes at least one of the following steps 310-320.

[0097] Step 310: Run the fragment shader so that the fragment shader acquires multiple compressed data.

[0098] The compressed data is obtained by converting image blocks into compressed blocks. Different compressed data correspond to different image blocks, and the image blocks corresponding to multiple compressed data are all obtained by dividing the first image.

[0099] In some embodiments, a fragment shader is run such that it retrieves multiple compressed data sets from an input object located in a second memory. For example, this is achieved by executing the fragment shader's texelFetch / imageLoad functions to retrieve multiple compressed data sets from the input object.

[0100] Step 320: Based on multiple compressed data, determine the color value of each fragment in the drawing area of ​​the fragment shader, and use it as the color value of each pixel in the first image.

[0101] The size of the drawing area is the same as the size of the first image.

[0102] In other words, the number of fragments in the drawing area is the same as the number of pixels in the first image, and there is a one-to-one correspondence between the fragments in the drawing area and the pixels in the first image. Therefore, calculating the color value of each fragment in the drawing area is equivalent to obtaining the color value of each pixel in the first image.

[0103] It should be noted that the above step of "determining the color value of each fragment in the drawing area of ​​the fragment shader based on multiple compressed data" is also implemented by running the fragment shader program, and will not be repeated hereafter.

[0104] Optionally, the fragment shader can use 16F / 32F as an intermediate field for data storage and representation during the decoding process.

[0105] Optionally, the color value of a pixel or fragment may include components of multiple color channels.

[0106] The technical solution provided in this application uses a fragment shader to decode the color values ​​of each pixel in a first image by coloring fragments within the drawing area. Since the multiple compressed data referenced in the coloring process are all converted from compressed blocks of image blocks (i.e., the compressed blocks of image blocks are converted to a format compatible with the fragment shader), and the image blocks corresponding to these compressed data are all divided from the first image, and the size of the drawing area is consistent with the size of the first image, this solution can ensure that the color values ​​of each pixel in the first image are completely decoded even when the fragment shader has completely colored the drawing area. In summary, the above method uses a fragment shader program in the graphics rendering pipeline to achieve complete image decoding. It can be applied to any device that supports a conventional graphics rendering pipeline, requiring no computational pipeline or dedicated hardware units, and has extremely high versatility.

[0107] In some embodiments, step 320 includes the following steps 322-326 ( Figure 3 (not shown in the image), the steps mentioned in the various embodiments below are all executed by the fragment shader, and will not be described again hereafter.

[0108] Step 322: For the first fragment in each fragment, based on the position of the first fragment in the drawing area and the size of the image block, determine the first image block to which the first pixel corresponding to the first fragment belongs, and the position of the first pixel in the first image block.

[0109] The correspondence between the first fragment and the first pixel means that the position of the first fragment in the drawing area corresponds to the position of the first pixel in the first image. For ease of understanding, this correspondence can be interpreted as follows: when the drawing area and the first image are overlapped in the same coordinate system, the coordinates of the first fragment in that coordinate system are the same as the coordinates of the first pixel in that coordinate system. For example, if the size of both the drawing area and the first image is 1024×512, and the first fragment is the fragment in the first row and third column of the drawing area, then the first pixel is the pixel in the first row and third column of the first image.

[0110] In some embodiments, step 322 includes steps (1) and (2) below.

[0111] (1) Perform element-wise quotient operation on the first coordinate vector and the block size vector to obtain the block coordinates of the first image block.

[0112] Wherein, the first coordinate vector is the vector representation of the position of the first fragment in the drawing area, the block size vector is the vector representation of the size of the image block, and the block coordinates of the first image block are used to indicate the first image block from the various image blocks obtained by dividing the first image.

[0113] Optionally, each element in the block size vector represents the number of pixels that an image block covers in one direction.

[0114] (2) Perform element-wise modulo operation on the first coordinate vector and the block size vector to obtain the coordinates of the first pixel in the first image block.

[0115] The coordinates of the first pixel in the first image block indicate the position of the first pixel in the first image block.

[0116] For example, the first coordinate vector is p = ivec2(gl_FragCoord.xy) – origin, the block coordinates of the first image block are blockID = p / ivec2(Bx,By), and the coordinates of the first pixel in the first image block are localID = p%ivec2(Bx,By). Here, ivec2(gl_FragCoord.xy) is the coordinate of the first fragment in the window coordinate system where the drawing area is located, and origin is a set offset value, usually 0. ivec2(Bx,By) is the block size vector. It should be noted that in this example, the first image is a 2D image; therefore, only the first direction x and the second direction y are considered. If the first image is a 3D image, the above formula can be extended by the third direction z.

[0117] Using the above scheme, the first image block to which the first pixel corresponding to the first fragment belongs, and the position of the first pixel in the first image block, can be determined by element-wise operation instructions on the vector, which has low implementation complexity.

[0118] Step 324: Based on the position of the first pixel in the first image block and the inverse quantization data of the first compressed data in multiple compressed data, the color value of the first fragment is obtained.

[0119] The image block corresponding to the first compressed data is the first image block, and the inverse quantization data of the first compressed data is obtained based on the first compressed data.

[0120] In some embodiments, the dequantized data of the first compressed data is the data obtained by performing dequantization on the first compressed data. For the specific dequantization process, please refer to the embodiments below, which will not be described in detail here.

[0121] Step 326: Write the color value of the first fragment as the color value of the first pixel into the output object.

[0122] In some embodiments, the color value of the first pixel is written to the corresponding position in the output object. Furthermore, to ensure visibility of subsequent pipelines, a memory barrier or API synchronization can be inserted into the output object by the first processor. The corresponding position of the first pixel in the output object refers to the position in the output object that corresponds to the position of the first pixel in the first image. For example, if the output object is an output image, and the coordinates of the first pixel in the first image are (x1, y1), then the color value of the first pixel should also be written to position (x1, y1) in the output image.

[0123] In some embodiments, the method further includes: inputting the color value of the first pixel into a transfer function according to the indication of the sRGB switch, wherein the transfer function maps the color value of the first pixel from linear space to sRGB (standard Red Green Blue) color space, thereby converting the color value of the first pixel into an sRGB signal that supports display on a standard sRGB display. Optionally, the sRGB signal is written to the corresponding position in the output object.

[0124] In some embodiments, the method further includes: first determining, based on the layer index of the first image, an output object belonging to the layer indicated by the layer index from a second memory; and then writing the color value of the first pixel into the corresponding position in the output object.

[0125] In the above embodiment, by utilizing the correspondence between fragments in the drawing area and pixels in the first image, the position of the first fragment in the drawing area can be used to infer the image block to which the corresponding first pixel belongs and the position of the first pixel within that image block. This allows for the calculation of fragment color values ​​based on compressed data. This scheme leverages the correspondence between fragments and pixels when the drawing area and the first image are of the same size, enabling the use of a fragment shader to calculate fragment color values ​​and thus achieve compressed block decoding.

[0126] In addition, this application provides two decoding methods on the fragment shader side. The first decoding method uses a two-stage decoding approach: first, the fragment shader pre-decodes the inverse quantized data of each compressed data segment, and then directly uses the pre-decoded inverse quantized data during the fragment calculation stage. The second decoding method involves the fragment shader performing steps such as block location, compressed data parsing, inverse quantization, interpolation, and writing for each fragment. These two decoding methods will be described in detail in the following embodiments.

[0127] Please refer to Figure 4 The diagram illustrates a flowchart of a decoding method provided in one embodiment of this application. This decoding method includes at least one of the following steps 410-460.

[0128] Step 410: Based on multiple compressed data, determine the inverse quantization data for each of the multiple compressed data.

[0129] In the following embodiments, the first compressed data among multiple compressed data will be used as an example to illustrate how to determine the inverse quantization data of the compressed data, which will not be elaborated here.

[0130] Step 420: Write the dequantized data of each of the multiple compressed data sets into an intermediate storage object.

[0131] The intermediate storage object can be any object used to store data generated by the fragment shader; this application does not limit this.

[0132] Step 430: Based on the position of the first fragment in the drawing area and the size of the image block, determine the first image block to which the first pixel belongs, and the position of the first pixel in the first image block.

[0133] Step 440: If it is determined that the first pixel belongs to the first image block, the inverse quantization data of the first compressed data is obtained from the intermediate storage object.

[0134] Step 450: Based on the position of the first pixel in the first image block and the inverse quantization data of the first compressed data in multiple compressed data, the color value of the first fragment is obtained.

[0135] Step 460: Write the color value of the first fragment as the color value of the first pixel into the output object.

[0136] In the above embodiments, when it is determined that the first pixel belongs to the first image block, since the first compressed data corresponds to the first image block, the inverse quantization data of the first compressed data can be directly obtained from the intermediate storage object without having to repeatedly determine the inverse quantization data of the first compressed data. Therefore, the above embodiments can avoid repeated parsing of compressed data, which helps to improve the throughput and energy efficiency of the image decoding process.

[0137] Please refer to Figure 5 The diagram illustrates a flowchart of a decoding method provided in another embodiment of this application. This decoding method includes at least one of the following steps 510-540.

[0138] Step 510: Based on the position of the first fragment in the drawing area and the size of the image block, determine the first image block to which the first pixel belongs, and the position of the first pixel in the first image block.

[0139] Step 520: If it is determined that the first pixel belongs to the first image block, the inverse quantization data of the first compressed data is determined based on the first compressed data.

[0140] Step 530: Based on the position of the first pixel in the first image block and the inverse quantization data of the first compressed data in multiple compressed data, the color value of the first fragment is obtained.

[0141] Step 540: Write the color value of the first fragment as the color value of the first pixel into the output object.

[0142] In the above embodiment, after determining that the first pixel belongs to the first image block, the inverse quantization data of the first compressed data is then specifically determined. In this decoding process, the inverse quantization data of the compressed data does not need to be saved in advance, which helps to save memory space on the second processor side.

[0143] In some embodiments, the first compressed data includes a configuration field, a weight quantization value field, and an endpoint quantization value field. The inverse quantization data of the first compressed data includes weights in the weight grid and color endpoint pairs of the first image patch. The inverse quantization data of the first compressed data is determined through the following steps S31-S33. That is, the above steps 410 or 520 may include the following steps S31-S33 (not shown in the figures).

[0144] Step S31: Based on the configuration fields, determine the size of the weighted grid, the weight quantization mode information, and the CEM (Color Endpoint Mode) information.

[0145] A grid cell in the weight grid is used to carry a weight. The weight quantization mode information is used to indicate the target weight quantization mode used in the process of quantizing the weights in the weight grid into a weight quantization value field. The CEM information is used to indicate the target encoding method used for the color endpoints of the first image block. The color endpoints of the first image block are used to interpolate and determine the color values ​​of the pixels in the first image block.

[0146] In some embodiments, the fragment shader extracts information or fields (such as the configuration field, weight quantization value field, and endpoint quantization value field mentioned above) from the first compressed data using a bit extraction function. This bit extraction function supports extracting a set length of bits from the first compressed data in a single operation. Since the set length can be set and changed as needed by an engineer, this bit extraction function can reduce memory accesses and branching. For example, in the case of uvec4 first compressed data, this bit extraction function can be a bitfieldExtract / bitfieldReverse function that supports bit extraction across 32-bit boundaries.

[0147] Below is a brief introduction to the composition of the configuration fields.

[0148] The configuration field can be considered as a field located in the header of the first compressed data, which is used to describe the encoding mode of the first image block. It may include at least one of the following: block mode field, number of partitions information, seed value of the first image block, blank extended block identifier, CEM information, and CCS (Color Component Selector) information.

[0149] The block mode field includes at least one of the following: the size of the weight grid, the weight quantization mode information, and the biplane indication information.

[0150] The size of the weighted grid includes the number of grid cells it covers in each direction. For example, a weighted grid with a size of 3×3 contains 9 grid cells and supports 9 weights.

[0151] The weight quantization mode information indicates the target weight quantization mode used in the process of quantizing the weights in the weight grid into the weight quantization value field. For example, the weight quantization mode information is an index of the target weight quantization mode. Different weight quantization modes represent different quantization methods for the weights and have different indices.

[0152] The dual-plane indication information is used to indicate whether a dual-weighted plane is used in the encoding process for the first image block.

[0153] In the case where the encoding process of the first image block does not employ a dual-weight plane, the color values ​​of the pixels in the first image block are encoded with the same weights for the components in different color channels.

[0154] In the case where the encoding process of the first image block does not employ a dual-weight plane, the color value of a pixel in the first image block contains a component of a color channel that is encoded with a separate weight.

[0155] The partition count information is used to indicate the number of partitions included in the first image block. Each partition in the first image block has a separate pair of color endpoints.

[0156] The seed value of the first image block is used to calculate the partition index of each pixel in the first image block. The partition index of a pixel is used to indicate the partition to which the pixel belongs.

[0157] The blank extension block identifier is a special code used to indicate whether the first image block is an empty block with a constant color value or an extension block that extends beyond the boundary of the first image.

[0158] CEM information is used to indicate the target encoding method used for the color endpoint pairs of the first image patch. The target encoding method is the encoding method used to encode the color endpoint pairs of the first image patch into the endpoint quantization value field.

[0159] CCS information is used to indicate color channels encoded with separate weights when a dual-weight plane is used in the encoding process of the first image block.

[0160] In some embodiments, before step S31, the aforementioned blank extension identifier is parsed first. If the blank extension identifier indicates that the first image block is not an empty block / extended block, then step S31 is executed. If the blank extension identifier indicates that the first image block is an empty block / extended block, the first image block can be filled with the set color value or marked according to the specification.

[0161] Step S32: Based on the weight quantization mode information, dequantize from the weight quantization value field to obtain the weights in the weight grid.

[0162] In some embodiments, step S32 includes the following steps:

[0163] 1. Find the target weight quantization parameter corresponding to the target weight quantization mode from the weight quantization parameter lookup table.

[0164] The weight quantization parameter lookup table is used to record the correspondence between weight quantization modes and weight quantization parameters.

[0165] For example, the weight quantization parameter lookup table records the index of each weight quantization mode and the weight quantization parameter corresponding to each index. Therefore, the target weight quantization parameter can be found from the weight quantization parameter lookup table based on the index of the target weight quantization mode contained in the weight quantization mode information.

[0166] Optionally, the target weight quantization parameters include at least one of the following: the number of bits occupied by the quantized value of each weight in the weight quantization value field, and encoding information. The encoding information is used to indicate the encoding method of the weights in the weight grid, such as trit / quint (3-ary / 5-ary) encoding.

[0167] 2. Based on the target weight quantization parameters, the weights in the weight grid are obtained by dequantization from the weight quantization value field.

[0168] In some embodiments, the step includes at least one of the following steps (1) to (3).

[0169] (1) According to the target weight quantization parameter, the number of bits occupied by each weight in the weight quantization value field, and extract the quantization value of each weight in the weight grid from the weight quantization value field.

[0170] Optionally, according to the encoding method indicated by the encoding method information, dequantization calculation is performed on the quantized value of the weight extracted from the weight quantization value field to obtain the weight in the weight grid.

[0171] (2) Determine the inverse quantization lookup table corresponding to the encoding method indicated by the encoding method information. The inverse quantization lookup table corresponding to a certain encoding method records the correspondence between the weights and the weight quantization values ​​under that encoding method. The inverse quantization result of the quantization value in the weight quantization value field can be retrieved from the inverse quantization lookup table corresponding to the encoding method. For example, in the case of trit / quint encoding, the compact 3 / 5 base symbol sequence in the weight quantization value field can be restored to the weight value by using the inverse quantization lookup table corresponding to trit / quint encoding.

[0172] Optionally, the above inverse quantization results can be directly used as weights in the weight grid.

[0173] (3) Map the dequantization result obtained from the dequantization lookup table to the standard weight range to obtain the weight in the weight grid.

[0174] The standard weight range is set by technical personnel as needed, such as 0~64 (0 represents an actual weight of 0, 1 represents an actual weight of 1 / 64, and so on, 64 represents an actual weight of 64 / 64=1), and this application does not limit it.

[0175] Optionally, the process of mapping the dequantization result to the standard weight range can also be accomplished by looking up a table. For example, since the weight values ​​that can be retrieved in the dequantization lookup table corresponding to the encoding method are predictable, a standardized lookup table that records the mapping relationship between these weight values ​​and weight values ​​within the standard range can be pre-established. Then, based on the dequantization result, the weights in the weight grid are retrieved from the standardized lookup table. These weights are all within the standard weight range, thereby minimizing the computational load of the image decoding process.

[0176] In some embodiments, when the dual-plane indication information indicates that the encoding process for the first image block uses a dual-weight plane, the weights in the two weight grids (i.e., two sets of weights) can be dequantized from the weight quantization value field. For example, if the weight grid size is 3×3 and the encoding process uses a dual-weight plane, 18 weights can be dequantized from the weight quantization value field, which can then be filled into the two weight grids respectively. In the above case, the method further includes the following step: determining the target color channel indicated by the CCS information from each color channel according to the CCS information in the configuration field. Wherein, the first weight grid in the two weight grids corresponds to the target color channel, and the second weight grid corresponds to the remaining color channels other than the target color channel.

[0177] Of course, if the dual-plane instruction information indicates that the encoding process for the first image block uses a dual-weight plane, a single weight grid can still be used. However, each grid cell in the weight grid needs to be filled with two weights, one of which corresponds to the target color channel and the other corresponds to the other color channels.

[0178] For details on the specific role of the weights in the weight grid, and how the correspondence between the weight grid / weights and color channels is specifically reflected, please refer to the examples below, which will not be elaborated here.

[0179] In the above embodiment, the target weight quantization parameters required for the weight dequantization process are obtained by using a pre-configured weight quantization parameter lookup table, thereby improving the dequantization efficiency of the process and thus increasing the throughput of the entire image decoding process.

[0180] In some embodiments, the aforementioned weight quantization value field is extracted as follows: the length of the configuration field is calculated, and the length of the configuration field is used as an offset relative to the header of the first compressed data to determine the starting position of the weight quantization value field in the first compressed data. The weight quantization value field is then extracted from this starting position, or the quantization value in the weight quantization value field is extracted.

[0181] Step S33: Based on CEM information, dequantize from the endpoint quantization value field to obtain the color endpoint pair of the first image block.

[0182] In some embodiments, step S33 includes at least one of the following steps:

[0183] 1. Extract the quantization values ​​of the color endpoint pairs of the first image block from the endpoint quantization value field according to the target encoding method.

[0184] In some embodiments, the endpoint quantization value field is extracted as follows: the remaining bits in the first compressed data, excluding the weight quantization value field of the configuration field, are determined as the endpoint quantization value field. That is, the length of the endpoint quantization value field is equal to the length of the first compressed data minus the sum of the lengths of the configuration field and the weight quantization value field.

[0185] Since the starting position and length of the quantized value of the color endpoint pair encoded under a certain encoding method are determined by the specification of that encoding method, the quantized value of the color endpoint pair of the first image block can be directly extracted from the endpoint quantized value field based on the target encoding method indicated by the CEM information.

[0186] Since the target encoding method indicated by the CEM information is the encoding method for color endpoint pairs, the target encoding method can also be directly called the target color endpoint mode, i.e., the target CEM. The target CEM (target encoding method) can include variants such as L / LA / RGB / RGBA, LDR / HDR and direct / base+offset / delta / scale, which are not limited in this application.

[0187] It should be noted that a color endpoint pair of the first image block includes two color values, which are used for interpolation to determine the color value of the pixels in the first image block, hence the name "color endpoint".

[0188] In some embodiments, the first image block includes at least two partitions, and the color endpoint pairs of the first image block include color endpoint pairs for each of the at least two partitions. The color endpoint pairs of the partitions are used for interpolation to determine the color values ​​of pixels in the partitions. This step includes: determining the quantized values ​​of the color endpoint pairs for each of the at least two partitions from the endpoint quantization value field according to the target encoding scheme.

[0189] 2. Find the target color quantization parameter corresponding to the target encoding method from the color quantization parameter lookup table.

[0190] The color quantization parameter lookup table is used to record the correspondence between encoding methods and color quantization parameters.

[0191] The above encoding method refers to the encoding method for color values, and the above color quantization parameters are parameters used to quantize color values. The target color quantization parameters corresponding to the target encoding method are the parameters used to quantize the endpoint color pairs of the first image block under the target encoding method.

[0192] For example, the color quantization parameter lookup table records the indexes of each encoding method (or each CEM), and the color quantization parameters corresponding to each encoding method's index. Therefore, the target color quantization parameter can be found from the color quantization parameter lookup table based on the encoding method index contained in the CEM information.

[0193] In some embodiments, the endpoint quantization value field, in addition to the quantization values ​​of the color endpoint pairs of the first image block, also includes some configuration information. This configuration information is also used to configure the encoding method of the color endpoint pairs of the first image block, i.e., to configure the target encoding method. Therefore, when looking up the target color quantization parameter from the color quantization parameter lookup table, the input needs to include not only CEM information but also the configuration information in the endpoint quantization value field.

[0194] Optionally, based on the target color quantization parameters, the quantization values ​​of the color endpoint pairs of the first image block are dequantized to obtain the color endpoint pairs of the first image block.

[0195] 3. Based on the target color quantization parameters and the quantization values ​​of the color endpoint pairs of the first image block, the color endpoint pairs of the first image block are retrieved from the color inverse quantization lookup table.

[0196] The color inverse quantization lookup table is used to record the correspondence between the quantized values ​​and the color values ​​under different color quantization parameters.

[0197] In some embodiments, the color inverse quantization lookup table is used to record the correspondence between the combination of the color quantization parameter and the quantized value of the color value, and the color value itself. Therefore, by using the combination of the target color quantization parameter and the quantized value of the color endpoint pair of the first image block as input to the query, the color endpoint pair of the first image block can be found from the color inverse quantization lookup table.

[0198] In some embodiments, the color inverse quantization lookup table includes multiple sub-lookup tables corresponding to different color quantization parameters. Each sub-lookup table records the correspondence between the quantized color value and the color value under the corresponding color quantization parameter. First, based on the target color quantization parameter, a target sub-lookup table corresponding to the target color quantization parameter is determined from the multiple sub-lookup tables. Then, based on the color endpoint pairs of the first image patch, the color endpoint pairs of the first image patch can be found from the target sub-lookup table.

[0199] In some embodiments, the color endpoint pairs of at least two partitions are retrieved from the color dequantization lookup table based on the target color quantization parameter and the quantization values ​​of the color endpoint pairs of at least two partitions.

[0200] In some embodiments, the color endpoint pair of the first image block obtained from the color inverse quantization lookup table is not the final color endpoint pair used; it can be referred to as the uncorrected color endpoint pair. The method further includes: correcting the uncorrected color endpoint pair according to the encoding scheme indicated by the CEM information, or CEM, to obtain the color endpoint pair of the first image block. Different encoding schemes require different correction rules, such as blue-contract, bit-transfer-signed, HDR / LDR scaling and offset rules, etc. Therefore, this application does not limit the correction rule.

[0201] In the above embodiment, after extracting the quantization value based on the CEM information, the color endpoint pairs of the first image block are obtained entirely through table lookup. This breaks down the complex calculation and mapping process into multiple table lookup processes, which helps to improve throughput. Furthermore, since no calculation is required, it is also convenient to verify the accuracy of the final table lookup result.

[0202] In some embodiments, step 324 above includes the following steps S41 to S42 (not shown in the figures).

[0203] Step S41: Based on the position of the first pixel in the first image block, interpolate the weights in the weight grid to obtain the weight of the first pixel.

[0204] In some embodiments, the step includes: mapping the position of the first pixel in the first image block to a weight grid; obtaining weights from at least two grid cells in the weight grid that are closest to the mapped position of the first pixel, thus obtaining at least two weights; and summing the at least two weights based on the mapped position of the first pixel and the positions of the at least two grid cells in the weight grid to obtain the weight of the first pixel. For example, in the case of 4-neighborhood weighting, the at least two grid cells are the four grid cells (top left, bottom left, top right, and bottom right) in the weight grid that are closest to the mapped position of the first pixel, and the at least two weights are four weights.

[0205] In some embodiments, the position of the grid cell in the weighted grid is represented by coordinate values ​​within the standard coordinate range. Mapping the position of the first pixel in the first image block to the weighted grid means normalizing the coordinates of the first pixel in the first image block to the standard coordinate range to obtain the mapped coordinate values. These mapped coordinate values ​​can reflect the mapped position of the first pixel in the weighted grid.

[0206] The above standard coordinate range is set by technical personnel as needed. For example, it can be set to be consistent with the above standard weight range, such as both the standard coordinate range and the standard weight range being 0~64.

[0207] In some embodiments, the dual-plane indication information indicates that the encoding process for the first image block employs a dual-weight plane. Based on the position of the first pixel in the first image block, weights in the first weight grid are interpolated to obtain a first weight for the first pixel, and weights in the second weight grid are interpolated to obtain a second weight for the first pixel. Alternatively, based on the position of the first pixel in the first image block, weights in the weight grid corresponding to the target color channel are interpolated to obtain a first weight for the first pixel, and weights in the weight grid corresponding to the remaining color channels are interpolated to obtain a second weight for the first pixel.

[0208] Step S42: Based on the weight of the first pixel, interpolate the color endpoints of the first image block to obtain the color value of the first fragment.

[0209] In some embodiments, the number of partitions indicated by the partition quantity information is 1, and the first image block has only one color endpoint pair. The color value of the first fragment can be obtained by interpolating the color endpoint pair of the first image block using the weight of the first pixel. The interpolation formula is as follows: W×F1+(1-W2)×F2, where W is the weight of the first pixel, F1 is one color value in the color endpoint pair, and F2 is the other color value in the color endpoint pair.

[0210] In some embodiments, a first weight of the first pixel is used to interpolate the components of the color endpoint pairs of the first image block in the target color channel to obtain the components of the color value of the first fragment in the target color channel. A second weight of the first pixel is used to interpolate the components of the color endpoint pairs of the first image block in the remaining color channels to obtain the components of the color value of the first fragment in the remaining color channels.

[0211] In the above embodiment, the weight of each pixel in the first image block is not encoded separately in the first compressed data, but is encoded on a coarser grid cell. In the decoding stage, the pixel weight is obtained by interpolation from the grid cell according to the pixel position, thereby ensuring that this application can be applied to decode compressed data with high compression ratio.

[0212] In some embodiments, the first image block includes at least two partitions, and the color endpoint pairs of the first image block include color endpoint pairs for each of the at least two partitions. That is, the number of partitions indicated by the partition number information is greater than 1. In this case, the above method further includes the following steps S51-S52 (not shown in the figures).

[0213] S51, determine the seed value of the first image patch from the configuration field.

[0214] The seed value of the first image block is used to calculate the partition index of each pixel in the first image block. The partition index of a pixel is used to indicate the partition to which the pixel belongs.

[0215] In computer science, a "seed value" is an initial value used to generate a pseudo-random number sequence. In the field of video encoding and decoding technology, it can be combined with the pixel position in an image block and converted into a pixel partition index through calculation. However, the process of mapping partition indexes based on seed values ​​and pixel positions involves relatively complex calculations, which can affect decoding efficiency. Therefore, this application solves the above problem by pre-calculating the partition indexes corresponding to each combination of query values ​​(combinations of pixel positions and seed values) and flattening the combination of query values ​​and partition indexes into a partition lookup table.

[0216] S52, based on the position of the first pixel in the first image block and the seed value of the first image block, the partition index of the first pixel is obtained from the partition lookup table. The partition lookup table is used to record the correspondence between the query value combination and the partition index.

[0217] In some embodiments, a partition lookup table is obtained from the R8UI texture buffer of the second processor.

[0218] The query value combination is a combination of pixel position and seed value, and the partition index of the first pixel is used to indicate that the first pixel belongs to the target partition in at least two partitions.

[0219] In some embodiments, the query value combination is mapped to lookup coordinates (seed, localID) in a partition lookup table, thereby retrieving the partition index of the first pixel from the location indicated by the lookup coordinates in the partition lookup table. Here, localID represents the coordinates of the first pixel within the first image block.

[0220] In some embodiments, the partition information retrieved from the partition lookup table is of fixed length, such as 2-bit or 8-bit partition information, which includes the partition index of the first pixel. Based on the number of partitions, the valid bits are extracted from the partition information to obtain the partition index of the first pixel. For example, when the number of partitions is 4, the partition information includes two valid bits; when the number of partitions is 8, the partition information includes three valid bits.

[0221] The above-mentioned step of "interpolating the color endpoint pairs of the first image block based on the weight of the first pixel to obtain the color value of the first fragment" includes: interpolating the color endpoint pairs of the target partition based on the weight of the first pixel to obtain the color value of the first fragment.

[0222] In some embodiments, a first weight of the first pixel is used to interpolate the components of the color endpoint pairs of the target partition in the target color channel to obtain the components of the color value of the first fragment in the target color channel. A second weight of the first pixel is used to interpolate the components of the color endpoint pairs of the target partition in the remaining color channels to obtain the components of the color value of the first fragment in the remaining color channels.

[0223] In the above embodiment, the partition to which the first pixel belongs is determined by looking up the partition lookup table. This method can avoid performing complex partition function calculations inside the fragment shader, thereby reducing the complexity and branch divergence of image decoding performed by the fragment shader.

[0224] In some embodiments, in addition to the final color value, the weights in the weighted grid mentioned in the above embodiments, the partitions to which the pixels belong, and the endpoint color pairs can all be output as visual information by the second processor, thereby facilitating technicians to verify the correctness of each item against the decoding specification.

[0225] In some embodiments, prior to step 324, the method further includes: if the size of the first image is not divisible by the size of the image block, determining the global coordinates of the first pixel based on the coordinates of the first pixel in the image block, the size of the image block, and the block coordinates of the first image block. That is, the global coordinates of the first pixel are pixelID = blockID. The expression `blockDim + localID` represents the coordinates of the first image block, where `blockID` is the block coordinates, `blockDim` is the size of the image block, and `localID` is the coordinates of the first pixel within the first image block. The global coordinates of the first pixel refer to its coordinates in the coordinate system used by the first image. Based on these global coordinates, it can be determined whether the first pixel exceeds the boundary of the first image. If the first pixel exceeds the boundary, there is no need to perform the color value calculation step for the first fragment; in the output object, the color value for the first pixel can be omitted or a pre-defined color value can be written.

[0226] In addition, it should be noted that the various lookup tables mentioned in this application can be generated by the first processor in the initial stage and uploaded to the Texture Nuffer Object (TBO) in the second memory for use by the fragment shader, or they can be directly embedded in the fragment shader program. This application does not limit this.

[0227] An exemplary embodiment of this application also provides an image decoding system. This image decoding system, as... Figure 1 As shown, the system 100 includes a first processor 10, a first memory 20, a second processor 30, and a second memory 40. The first memory 20 stores a compressed file, which includes compressed blocks of multiple image blocks, each of which is derived from the first image.

[0228] The first processor 10 is configured to convert the compressed blocks of the plurality of image blocks into a format supported by the fragment shader to obtain a plurality of compressed data, wherein different compressed data correspond to different image blocks; and to write the plurality of compressed data into the second memory 40.

[0229] The second processor 30 is configured to run the fragment shader such that the fragment shader retrieves the plurality of compressed data from the second memory 40.

[0230] The second processor 30 is further configured to determine, based on the plurality of compressed data, the color value of each fragment in the drawing area of ​​the fragment shader, as the color value of each pixel in the first image; wherein the size of the drawing area is consistent with the size of the first image.

[0231] It should be noted that, in the above image decoding system, details not described in detail, such as other functions of the first processor and the second processor, can be found in the method embodiments of the first processor side and the second processor side described above, and will not be repeated here.

[0232] The following is a concrete example illustrating the decoding process of an image decoding system. Please refer to [link / reference]. Figure 6 The decoding process includes the following steps S61 to S66.

[0233] S61, the CPU retrieves the compressed file from the first memory.

[0234] S62, the CPU parses the header of the compressed file to determine the size of the first image and the size of the image blocks; and converts the compressed blocks of multiple image blocks in the compressed file into the format supported by the fragment shader to obtain multiple compressed data.

[0235] S63, the CPU sets the size of the drawing area of ​​the fragment shader to the size of the first image, and writes the size of the image block to the second memory.

[0236] In S64, the GPU runs a fragment shader so that the fragment shader can acquire multiple compressed data sets.

[0237] In S65, the GPU uses the fragment shader to determine the first image block to which the first pixel belongs, and the position of the first pixel in the first image block, based on the position of the first fragment in the drawing area and the size of the image block.

[0238] In S66, the GPU obtains the color value of the first fragment based on the position of the first pixel in the first image block and the inverse quantization data of the first compressed data in multiple compressed data through the fragment shader. This color value is used as the color value of the first pixel, and the image block corresponding to the first compressed data is the first image block.

[0239] In summary, the technical solutions provided by the embodiments of this application have the following technical effects:

[0240] Computational pipeline freedom and hardware freedom: Real-time decoding is possible on devices that do not support computational shaders and have no hardware ASTC decoder, relying solely on the conventional rendering pipeline of vertex shaders-fragment shaders.

[0241] Cross-platform consistency: Get rid of driver-private paths, output is repeatable and easy to regress.

[0242] The project is easily integrated: it can be directly connected to existing FBO / post-processing links with low modification costs; it supports sRGB / LDR / HDR and multiple format outputs.

[0243] Adjustable performance: Two-stage pre-decoding significantly reduces redundant parsing, resulting in higher throughput and more controllable power consumption.

[0244] An embodiment of this application also provides a computer device that includes the image decoding system described above.

[0245] Please refer to Figure 7 The diagram illustrates a structural block diagram of a computer device provided in one embodiment of this application.

[0246] Typically, computer device 700 includes a processor 701 and a memory 702.

[0247] Processor 701 may include one or more processing cores, such as a quad-core processor, an octa-core processor, etc. Processor 701 may be implemented using at least one hardware form selected from DSP (Digital Signal Processing), FPGA (Field Programmable Gate Array), and PLA (Programmable Logic Array). Processor 701 may also include a main processor and a coprocessor. The main processor, also known as the CPU, is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, processor 701 may integrate a GPU, which is responsible for rendering and drawing the content required to be displayed on the screen. In some embodiments, processor 701 may also include an AI processor, which is used to handle computational operations related to machine learning.

[0248] The memory 702 may include one or more computer-readable storage media, which may be tangible and non-transitory. The memory 702 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In some embodiments, the non-transitory computer-readable storage media in the memory 702 stores a computer program that is loaded and executed by the processor 701 to implement the image decoding method on the first processor side described above, and / or to implement the image decoding method on the second processor side described above.

[0249] Those skilled in the art will understand that Figure 7 The structure shown does not constitute a limitation on the computer device 700, and may include more or fewer components than shown, or combine certain components, or use different component arrangements.

[0250] In some embodiments, a computer-readable storage medium is also provided, wherein a computer program is stored therein, the computer program being loaded and executed by a processor to implement the image decoding method on the first processor side described above, and / or to implement the image decoding method on the second processor side described above.

[0251] Optionally, the computer-readable storage medium may include: ROM (Read-Only Memory), RAM (Random-Access Memory), SSD (Solid State Drives), or optical disc, etc. The random access memory may include ReRAM (Resistance Random Access Memory) and DRAM (Dynamic Random Access Memory).

[0252] In some embodiments, a computer program product is also provided, the computer program product including a computer program stored in a computer-readable storage medium, the processor reading from the computer-readable storage medium and executing the computer program to implement the image decoding method on the first processor side described above, and / or to implement the image decoding method on the second processor side described above.

[0253] It should be understood that "multiple" as used herein refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. Furthermore, the step numbers described herein are merely illustrative of one possible execution order. In some other embodiments, the steps may not be executed in numerical order, such as two steps with different numbers being executed simultaneously, or two steps with different numbers being executed in the reverse order of the illustration. This application does not limit this.

[0254] The above are merely exemplary embodiments of this application and are not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application shall be included within the protection scope of this application.

Claims

1. An image decoding method, characterized in that, Applied to a second processor, the method includes: Run the fragment shader so that the fragment shader acquires multiple compressed data, wherein the compressed data is obtained by converting the compressed blocks of image blocks into a format that the fragment shader supports processing, different compressed data correspond to different image blocks, and the image blocks corresponding to the multiple compressed data are all obtained by dividing the first image; For the first fragment among the fragments included in the drawing area of ​​the fragment shader, based on the position of the first fragment in the drawing area and the size of the image block, the first image block to which the first pixel corresponding to the first fragment belongs, and the position of the first pixel in the first image block are determined; Based on the position of the first pixel in the first image block and the inverse quantization data of the first compressed data in the plurality of compressed data, the color value of the first fragment is obtained; wherein, the image block corresponding to the first compressed data is the first image block, the inverse quantization data of the first compressed data is obtained based on the first compressed data, the color value of each fragment in the drawing area is used as the color value of each pixel in the first image, and the size of the drawing area is consistent with the size of the first image.

2. The method according to claim 1, characterized in that, The step of determining the first image block to which the first pixel corresponding to the first fragment belongs, and the position of the first pixel in the first image block, based on the position of the first fragment in the drawing area and the size of the image block, includes: An element-wise quotient operation is performed on the first coordinate vector and the block size vector to obtain the block coordinates of the first image block; wherein, the first coordinate vector is a vector representation of the position of the first fragment in the drawing area, the block size vector is a vector representation of the size of the image block, and the block coordinates of the first image block are used to indicate the first image block from the various image blocks obtained by dividing the first image. Perform an element-wise modulo operation on the first coordinate vector and the block size vector to obtain the coordinates of the first pixel in the first image block.

3. The method according to claim 1, characterized in that, Before determining the first image block to which the first pixel corresponding to the first fragment belongs, and the position of the first pixel in the first image block, and the position of the first pixel in the first image block, based on the position of the first fragment in the drawing area, the size of the first image, and the size of the image block, the method for determining the first image block to which the first fragment in the first image block belongs, and the position of the first pixel in the first image block, the method further includes: Based on the multiple compressed data, the inverse quantization data of each of the multiple compressed data is determined respectively; Write the dequantized data of each of the multiple compressed data into an intermediate storage object; Before obtaining the color value of the first fragment based on the position of the first pixel in the first image block and the inverse quantization data of the first compressed data in the plurality of compressed data, the method further includes: If it is determined that the first pixel belongs to the first image block, the inverse quantization data of the first compressed data is obtained from the intermediate storage object.

4. The method according to claim 1, characterized in that, After determining the first image block to which the first pixel corresponding to the first fragment belongs, and the position of the first pixel in the first image block, and the position of the first pixel in the first image block, based on the position of the first fragment in the drawing area, the size of the first image, and the size of the image block, the method further includes: If it is determined that the first pixel belongs to the first image block, the inverse quantization data of the first compressed data is determined based on the first compressed data.

5. The method according to claim 1, characterized in that, The first compressed data includes a configuration field, a weight quantization value field, and an endpoint quantization value field; the inverse quantization data of the first compressed data includes the weights in the weight grid and the color endpoint pairs of the first image patch. The method for obtaining the inverse quantization data of the first compressed data includes: Based on the configuration fields, the size of the weight grid, weight quantization mode information, and color endpoint mode (CEM) information are determined. A grid cell in the weight grid is used to carry a weight. The weight quantization mode information is used to indicate the target weight quantization mode used in the process of quantizing the weights in the weight grid into the weight quantization value field. The CEM information is used to indicate the target encoding method used for the color endpoint pair of the first image block. The color endpoint pair of the first image block is used to interpolate and determine the color value of the pixels in the first image block. Based on the weight quantization mode information, the weights in the weight grid are obtained by dequantization from the weight quantization value field; Based on the CEM information, the color endpoint pairs of the first image block are obtained by dequantization from the endpoint quantization value field.

6. The method according to claim 5, characterized in that, The step of dequantizing from the weight quantization value field to obtain the weights in the weight grid based on the weight quantization mode information includes: The target weight quantization parameter corresponding to the target weight quantization mode is found in the weight quantization parameter lookup table, wherein the weight quantization parameter lookup table is used to record the correspondence between the weight quantization mode and the weight quantization parameter; Based on the target weight quantization parameters, the weights in the weight grid are obtained by dequantization from the weight quantization value field.

7. The method according to claim 5, characterized in that, The step of obtaining the color endpoint pair of the first image patch by dequantization from the endpoint quantization value field based on the CEM information includes: According to the target encoding method, determine the quantization value of the color endpoint pair of the first image block from the endpoint quantization value field; The target color quantization parameter corresponding to the target encoding method is found in the color quantization parameter lookup table, wherein the color quantization parameter lookup table is used to record the correspondence between encoding methods and color quantization parameters; Based on the target color quantization parameters and the quantization values ​​of the color endpoint pairs of the first image block, the color endpoint pairs of the first image block are retrieved from the color inverse quantization lookup table. The color inverse quantization lookup table is used to record the correspondence between the quantization values ​​of color values ​​and the color values ​​under different color quantization parameters.

8. The method according to claim 1, characterized in that, The inverse quantization data of the first compressed data includes the weights in the weight grid and the color endpoint pairs of the first image patch; The process of obtaining the color value of the first fragment based on the position of the first pixel in the first image block and the inverse quantization data of the first compressed data in the plurality of compressed data includes: Based on the position of the first pixel in the first image block, the weights in the weight grid are interpolated to obtain the weight of the first pixel; Based on the weight of the first pixel, the color endpoints of the first image block are interpolated to obtain the color value of the first fragment.

9. The method according to claim 8, characterized in that, The first image block includes at least two partitions, and the color endpoint pairs of the first image block include the color endpoint pairs of each of the at least two partitions; The method further includes: The seed value of the first image block is determined from the first compressed data. The seed value of the first image block is used to calculate the partition index of each pixel in the first image block. The partition index of the pixel is used to indicate the partition to which the pixel belongs. Based on the position of the first pixel in the first image block and the seed value of the first image block, the partition index of the first pixel is obtained from the partition lookup table. The partition lookup table is used to record the correspondence between the query value combination and the partition index. The query value combination is a combination of pixel position and seed value. The partition index of the first pixel is used to indicate that the first pixel belongs to the target partition among the at least two partitions. The step of interpolating the color endpoint pairs of the first image block based on the weight of the first pixel to obtain the color value of the first fragment includes: Based on the weight of the first pixel, the color endpoint pairs of the target partition are interpolated to obtain the color value of the first fragment.

10. The method according to any one of claims 1 to 9, characterized in that, The first image is a texture image, and the compression block is an adaptive scalable texture compression ASTC block.

11. An image decoding method, characterized in that, Applied to a first processor, the method includes: Obtain a compressed file, the compressed file comprising compressed blocks for multiple image blocks, each of the multiple image blocks being divided from a first image; Each of the multiple image blocks is converted into a format supported by the fragment shader to obtain multiple compressed data, with different compressed data corresponding to different image blocks; Specifically, for the first fragment among the fragments included in the drawing area of ​​the fragment shader, the position of the first fragment in the drawing area and the size of the image block are used to determine the first image block to which the first pixel corresponding to the first fragment belongs in the first image, and the position of the first pixel in the first image block. The position of the first pixel in the first image block and the inverse quantization data of the first compressed data in the plurality of compressed data are used to obtain the color value of the first fragment. The image block corresponding to the first compressed data is the first image block. The inverse quantization data of the first compressed data is obtained based on the first compressed data. The color values ​​of each fragment in the drawing area are used as the color values ​​of each pixel in the first image. The size of the drawing area is consistent with the size of the first image.

12. The method according to claim 11, characterized in that, The method further includes: The file header of the compressed file is parsed to determine the size of the first image and the size of the image blocks; Set the size of the drawing area to the size of the first image.

13. The method according to claim 12, characterized in that, The method further includes: Based on the size of the first image and the size of the image block, the grid dimension information is calculated, and the grid dimension information is used to indicate the division method of the first image into the plurality of image blocks; Based on the mesh dimension information, an input object is created, which is used to carry the input data of the fragment shader; Write the plurality of compressed data into the input object; An output object is created according to the size of the first image. The output object is used to carry the decoding result of the fragment shader on the input data.

14. An image decoding system, characterized in that, The system includes a first processor, a first memory, a second processor, and a second memory; wherein, the first memory stores a compressed file, the compressed file including compressed blocks of multiple image blocks, each of the multiple image blocks being divided from a first image; The first processor is configured to convert the compressed blocks of the plurality of image blocks into a format supported by the fragment shader to obtain a plurality of compressed data, wherein different compressed data correspond to different image blocks; and write the plurality of compressed data into the second memory; The second processor is configured to run the fragment shader such that the fragment shader retrieves the plurality of compressed data from the second memory; The second processor is further configured to, for a first fragment among the fragments included in the drawing area of ​​the fragment shader, determine, based on the position of the first fragment in the drawing area and the size of the image block, a first image block to which a first pixel corresponding to the first fragment in the first image belongs, and the position of the first pixel in the first image block; The second processor is further configured to obtain the color value of the first fragment based on the position of the first pixel in the first image block and the inverse quantization data of the first compressed data in the plurality of compressed data; wherein, the image block corresponding to the first compressed data is the first image block, the inverse quantization data of the first compressed data is obtained based on the first compressed data, the color value of each fragment in the drawing area is used as the color value of each pixel in the first image, and the size of the drawing area is consistent with the size of the first image.

15. A computer device, characterized in that, The computer device includes a processor and a memory, the memory storing a computer program that is loaded and executed by the processor to implement the method as claimed in any one of claims 1 to 10, and / or to implement the method as claimed in any one of claims 11 to 13.

16. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that is loaded and executed by a processor to implement the method as claimed in any one of claims 1 to 10, and / or to implement the method as claimed in any one of claims 11 to 13.

17. A computer program product, characterized in that, The computer program product includes a computer program that is executed by a processor to implement the method as claimed in any one of claims 1 to 10, and / or to implement the method as claimed in any one of claims 11 to 13.

Images