Image decoding method and device, equipment and storage medium

Abstract

The invention provides an image decoding method and device, equipment and a storage medium, and relates to the technical field of computers. The method comprises the steps that a CPU analyzes image data to be decoded to obtain a plurality of code blocks, the code blocks are transmitted to a GPU, and the image data to be decoded are obtained in a wavelet transform coding mode; the GPU performs entropy decoding processing on the plurality of code blocks in parallel through the plurality of thread blocks to obtain sub-band coefficients; the GPU performs reverse wavelet transform on the sub-band coefficients to obtain wavelet reduction data; and the GPU performs pixel recovery processing on the wavelet recovery data to obtain decoded data. According to the scheme, the decoding efficiency and the resource utilization rate of image decoding can be improved by completing code block analysis in the CPU and executing entropy decoding and reverse wavelet transform in parallel in the GPU.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and more specifically, to an image decoding method, apparatus, device, and storage medium. Background Technology

[0002] In wavelet transform-based decoding, multiple data processing steps need to be performed sequentially on the image to be decoded. Under current computing architectures, some existing technologies typically complete all decoding processes of the image to be decoded solely within the CPU.

[0003] While decoding independently within the CPU is simple to implement and highly versatile, its limited parallel processing capabilities mean that decoding steps are typically executed sequentially, making it difficult to process multiple data segments simultaneously and resulting in low overall decoding efficiency. This is especially problematic when processing high-resolution images, where CPU throughput and memory bandwidth become limiting, hindering real-time processing and often leading to significantly increased decoding latency. Summary of the Invention

[0004] The purpose of this disclosure is to provide an image decoding method, an image decoding device, an image decoding equipment, and a computer-readable storage medium, which can improve the decoding efficiency and resource utilization of image decoding by completing code block parsing in the CPU and performing entropy decoding and inverse wavelet transform in parallel in the GPU.

[0005] Other features and advantages of this disclosure will become apparent from the following detailed description, or may be learned in part by practice of this disclosure.

[0006] According to a first aspect of the present disclosure, an image decoding method is provided, applied to an image decoding system, the image decoding system including a central processing unit (CPU) and a graphics processing unit (GPU), the method comprising: the CPU parsing image data to be decoded to obtain multiple code blocks, and transmitting the multiple code blocks to the GPU, wherein the image data to be decoded is obtained by wavelet transform encoding; the GPU performing entropy decoding processing on the multiple code blocks in parallel through multiple thread blocks to obtain subband coefficients; the GPU performing inverse wavelet transform on the subband coefficients to obtain wavelet restored data; and the GPU performing pixel recovery processing on the wavelet restored data to obtain decoded data.

[0007] In some example embodiments of this disclosure, based on the foregoing scheme, the step of performing entropy decoding processing on the multiple code blocks in parallel through multiple thread blocks to obtain subband coefficients includes: allocating the multiple code blocks to the multiple thread blocks; dividing the bit plane data in the code blocks into multiple threads within the corresponding thread blocks; loading decoding state data for context modeling into the shared memory of the thread blocks, and having the multiple threads perform entropy decoding processing on the bit plane data in parallel based on the decoding state data to obtain the subband coefficients.

[0008] In some example embodiments of this disclosure, based on the foregoing scheme, loading the decoding state data used for context modeling into the shared memory of the thread block includes: reading saliency state data, sign state data, and amplitude refinement state data from the code block and loading them into the shared memory of the thread block corresponding to the code block, so that the thread block can perform context modeling in the entropy decoding process; wherein, the saliency state data represents the marking information of whether each coefficient in the code block reaches saliency, the sign state data represents the marking information of the positive and negative sign attributes of each coefficient in the code block, and the amplitude refinement state data represents the marking information of the bit parsing status of each coefficient amplitude in the code block during the refinement process.

[0009] In some example embodiments of this disclosure, based on the foregoing scheme, the step of performing entropy decoding processing on the bit plane data in parallel by the multiple threads based on the decoding state data to obtain the subband coefficients includes: reading the decoding state data from the shared memory by the multiple threads respectively; performing context modeling based on the decoding state data by each thread, and performing bit value parsing on each bit position of the bit plane data in parallel in the entropy decoding channel according to the result of the context modeling; and generating the subband coefficients based on the result of the bit value parsing.

[0010] In some example embodiments of this disclosure, based on the foregoing scheme, the step of transmitting the plurality of code blocks to the GPU includes: allocating corresponding video memory in the GPU according to the size of the plurality of code blocks; and transmitting the plurality of code blocks to the video memory via asynchronous data transmission.

[0011] In some example embodiments of this disclosure, based on the foregoing scheme, writing the plurality of code blocks into the video memory via asynchronous data transmission includes: writing the plurality of code blocks into fixed page memory to obtain fixed page code block data; transmitting the fixed page code block data to the video memory via asynchronous data transmission, and storing the fixed page code block data in the video memory in a block-based storage manner.

[0012] In some example embodiments of this disclosure, based on the foregoing scheme, the inverse wavelet transform of the subband coefficients includes: when the wavelet transform encoding is lossy encoding, performing dequantization processing on the subband coefficients based on the quantization step size corresponding to the subband coefficients; loading the dequantized subband coefficients from video memory to shared memory; and using different thread blocks to perform inverse wavelet transform on the dequantized subband coefficients in the shared memory in the row direction and column direction, respectively.

[0013] In some example embodiments of this disclosure, based on the foregoing scheme, loading the dequantized subband coefficients from video memory to shared memory includes: dividing the dequantized subband coefficients into row direction coefficients and column direction coefficients; and loading the row direction coefficients and column direction coefficients in the video memory into the shared memory respectively.

[0014] In some example embodiments of this disclosure, based on the aforementioned scheme, the subband coefficients corresponding to a code block are processed by two thread blocks, the two thread blocks including a first thread block and a second thread block; the step of using different thread blocks to perform inverse wavelet transforms on the dequantized subband coefficients in the shared memory in the row direction and column direction respectively includes: using the first thread block to perform inverse wavelet transforms on the row direction coefficients in the shared memory in the row direction to obtain the row direction transform result; and using the second thread block to perform inverse wavelet transforms on the row direction transform result and the column direction coefficients in the column direction.

[0015] In some example embodiments of this disclosure, based on the foregoing scheme, the step of performing pixel restoration processing on the wavelet-reconstructed data to obtain decoded data includes: determining the number of thread blocks required for the pixel restoration processing based on the image width corresponding to the wavelet-reconstructed data and the number of threads in a single thread block in the row direction of the GPU; determining pixel restoration thread blocks based on the number of thread blocks; calling the pixel restoration kernel function required for the pixel restoration processing, and executing the pixel restoration kernel function in parallel on the threads within the pixel restoration thread block to perform pixel restoration processing on the wavelet-reconstructed data; wherein, the processing object of the thread is the pixel column of the wavelet-reconstructed data.

[0016] In some example embodiments of this disclosure, based on the foregoing scheme, determining the number of thread blocks required for pixel restoration processing based on the image width corresponding to the wavelet reconstruction data and the number of threads in a single thread block in the row direction of the GPU includes: determining the total number of pixel columns to be restored based on the image width corresponding to the wavelet reconstruction data; determining the number of covered columns of a single thread block based on the number of threads in a single thread block in the row direction of the GPU; and determining the number of thread blocks required for pixel restoration processing based on the calculation result of dividing the total number of pixel columns by the number of covered columns.

[0017] In some example embodiments of this disclosure, based on the foregoing scheme, the pixel restoration process includes one or more processing operations such as color space transformation, DC component translation transformation, and storage format transformation; the method further includes: if the pixel restoration process includes at least one processing operation, then the at least one processing operation is executed by the pixel restoration kernel function during a single kernel function execution.

[0018] In some example embodiments of this disclosure, based on the foregoing scheme, the image decoding method further includes: transmitting the decoded data from the GPU back to the CPU; and encapsulating the decoded data in the CPU based on a target output format.

[0019] According to a second aspect of the present disclosure, an image decoding apparatus is provided, the apparatus comprising: a data parsing module, configured to parse image data to be decoded to obtain multiple code blocks, and transmit the multiple code blocks to an entropy decoding module, wherein the image data to be decoded is obtained by wavelet transform encoding; an entropy decoding module, configured to perform entropy decoding processing on the multiple code blocks in parallel through multiple thread blocks to obtain subband coefficients; a wavelet transform module, configured to perform inverse wavelet transform on the subband coefficients to obtain wavelet restored data; and a pixel restoration module, configured to perform pixel restoration processing on the wavelet restored data to obtain decoded data.

[0020] According to a third aspect of the present disclosure, an image decoding device is provided, comprising: a processor, including a CPU and a GPU; and a memory storing computer-readable instructions that, when executed by the processor, implement the image decoding method as described in the first aspect.

[0021] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the image decoding method as described in the first aspect.

[0022] The technical solutions provided in this disclosure can include the following beneficial effects: The image decoding method in this embodiment first performs code block parsing in the CPU and then performs entropy decoding, inverse wavelet transform, and pixel restoration processing in the GPU. This allows the parsing step to be completed in the CPU, while the computationally intensive steps are completed in the GPU, resulting in a more reasonable workload distribution across different processors. Furthermore, by performing entropy decoding on multiple code blocks in parallel using multiple thread blocks, the entropy parsing of each code block is performed independently. This reduces the number of GPU memory accesses under parallel scheduling, avoiding decoding latency caused by serial parsing. Therefore, it has the advantages of improving decoding efficiency and resource utilization.

[0023] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0024] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure. It is obvious that the drawings described below are merely some embodiments of this disclosure, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0025] Figure 1 The illustration shows a schematic flowchart of an image decoding method according to some embodiments of the present disclosure.

[0026] Figure 2 The illustration shows a schematic diagram of the entropy decoding process for code blocks according to some embodiments of the present disclosure.

[0027] Figure 3 The schematic diagram illustrates a process for obtaining subband coefficients according to some embodiments of the present disclosure.

[0028] Figure 4 The schematic diagram illustrates a flow chart of an image decoding method according to some other embodiments of the present disclosure.

[0029] Figure 5 A block diagram schematically illustrates an image decoding apparatus according to some embodiments of the present disclosure.

[0030] Figure 6 The schematic diagram illustrates the structural schematic of a computer system of an image decoding apparatus according to some embodiments of the present disclosure.

[0031] Figure 7 A schematic diagram of a computer-readable storage medium according to some embodiments of the present disclosure is shown.

[0032] In the accompanying drawings, the same or corresponding reference numerals indicate the same or corresponding parts. Detailed Implementation

[0033] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this specification. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this specification as detailed in the appended claims.

[0034] The terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to be limiting of this specification. The singular forms “a,” “the,” and “the” as used in this specification and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any and all possible combinations of one or more of the associated listed items.

[0035] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be more thorough and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art.

[0036] Furthermore, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. Numerous specific details are provided in the following description to give a thorough understanding of embodiments of this disclosure. However, those skilled in the art will recognize that the technical solutions of this disclosure can be practiced without one or more of the specific details, or other methods, components, apparatuses, steps, etc., can be employed. In other instances, well-known methods, apparatuses, implementations, or operations are not shown or described in detail to avoid obscuring various aspects of this disclosure.

[0037] Furthermore, the accompanying drawings are for illustrative purposes only and are not necessarily drawn to scale. The block diagrams shown in the drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0038] In this example embodiment, an image decoding method is first provided. This image decoding method can be applied to an image decoding system, which includes a central processing unit (CPU) and a graphics processing unit (GPU). Furthermore, the image decoding method in this disclosure can be applied to wavelet transform-based decoding methods, such as those based on JPEG 2000 (Joint Photographic Experts Group 2000, J2K) and High-Throughput JPEG 2000 (HTJ2K). Figure 1 A schematic flowchart illustrating an image decoding method according to some embodiments of the present disclosure is shown. Reference Figure 1 As shown, the image decoding method may include the following steps: In step S110, the CPU parses the image data to be decoded to obtain multiple code blocks and transmits the multiple code blocks to the GPU. The image data to be decoded is obtained through wavelet transform coding. In step S120, the GPU performs entropy decoding on multiple code blocks in parallel using multiple thread blocks to obtain subband coefficients. Step S130: The GPU performs an inverse wavelet transform on the subband coefficients to obtain the wavelet-reconstructed data; In step S140, the GPU performs pixel recovery processing on the wavelet-reconstructed data to obtain the decoded data.

[0039] According to the image decoding method in this example embodiment, firstly, code block parsing is completed in the CPU, while entropy decoding, inverse wavelet transform, and pixel restoration are performed in the GPU. This allows the parsing step to be completed in the CPU, while computationally intensive steps are completed in the GPU, resulting in a more reasonable workload distribution across different processors. Furthermore, by performing entropy decoding on multiple code blocks in parallel using multiple thread blocks, the entropy parsing of each code block is performed independently. This reduces the number of GPU memory accesses under parallel scheduling, avoiding decoding latency caused by serial parsing. Consequently, this method offers advantages in improving decoding efficiency and resource utilization.

[0040] The image decoding method in this example embodiment will be further described below.

[0041] In step S110, the CPU parses the image data to be decoded to obtain multiple code blocks and transmits the multiple code blocks to the GPU. The image data to be decoded is obtained through wavelet transform coding.

[0042] The image data to be decoded can represent image encoded data generated based on a preset wavelet transform coding method and not yet decoded. This image encoded data can include a bitstream structure generated after forward wavelet transform, coefficient quantization, and entropy coding, which needs to be recovered into a pixel-domain image through inverse wavelet transform. A code block can represent an independent decoding unit obtained by dividing the image data to be decoded. Each code block can correspond to a fixed region in the wavelet coefficient matrix and can independently perform the decoding process without relying on other code blocks. Wavelet transform coding can represent a multi-resolution coding method based on wavelet functions, which decomposes the original image into low-frequency and high-frequency components, mapping the original image to multiple sub-bands in the frequency domain, thereby achieving image compression coding. The image data to be decoded in this disclosure is obtained through wavelet transform coding, such as JPEG2000 image data and High-Throughput JPEG 2000 (HTJ2K) image data. JPEG2000 can represent a still image compression standard based on discrete wavelet transform. It uses wavelet decomposition to perform multi-resolution analysis of image data and combines code block partitioning and entropy coding to achieve lossy or lossless image compression. HTJ2K can represent a high-throughput coding standard based on the JPEG 2000 coding system. It reduces the computational complexity of entropy decoding and improves the decoding speed of large-scale image data by redesigning the entropy coding structure to perform fast bit parsing of wavelet coefficients. In this step, by completing the parsing of the image data to be decoded in the CPU, the structure reading, field judgment, and control flow processing involved in the parsing stage are handled by the CPU. This avoids placing the parsing operation on the GPU, which is not good at handling such tasks. As a result, the GPU can directly enter the parallel computing process after receiving multiple parsed code blocks, thus making the task division in the decoding process more reasonable.

[0043] In step S120, the GPU performs entropy decoding on multiple code blocks in parallel using multiple thread blocks to obtain subband coefficients.

[0044] In this context, a thread block represents an execution unit in the GPU composed of multiple threads executing in parallel. Threads within a thread block can share the same shared memory region and collaboratively complete the processing task of the same target code block. Entropy decoding represents the decoding process of retrieving the original coefficient values ​​from the compressed bitstream by performing symbol analysis and context modeling on the bit plane data in the code block according to the entropy coding rules used in wavelet transform coding. Subband coefficients represent the quantized coefficient values ​​obtained from the bit plane of each code block through entropy decoding, used for subsequent inverse wavelet transform. These quantized coefficient values ​​include the low-frequency and high-frequency subband coefficients required for wavelet data reconstruction. Furthermore, in the entropy decoding process, one thread block processes one code block. This step, by having multiple thread blocks undertake the entropy decoding tasks for different code blocks, fully utilizes the GPU's parallel execution structure, enabling the GPU to process multiple independent bitstreams simultaneously, reducing the waiting time in the decoding stage and improving the overall parsing throughput, thereby accelerating the parsing speed of the code block.

[0045] In step S130, the GPU performs an inverse wavelet transform on the subband coefficients to obtain wavelet-reconstructed data.

[0046] The inverse wavelet transform can represent the step-by-step reconstruction process of the subband coefficients obtained through entropy decoding. It achieves the restoration of frequency domain data to spatial domain pixels through prediction and update steps for low-frequency and high-frequency components. In this disclosure, the inverse wavelet transform can be the inverse discrete wavelet transform or other wavelet-based inverse transform methods, such as the inverse integer wavelet transform or the inverse floating-point wavelet transform. The wavelet-reconstructed data can represent the image pixel matrix obtained after completing the inverse wavelet transform of the subband coefficients and spatial domain reconstruction, which can be used as input data for subsequent pixel restoration processing.

[0047] In step S140, the GPU performs pixel recovery processing on the wavelet reconstruction data to obtain decoded data.

[0048] Pixel restoration processing can refer to the pixel domain restoration operation performed on the wavelet-restored data after the inverse wavelet transform, which may include processing steps such as color space transformation, DC component translation transformation, and storage format transformation. Decoded data can represent the image data obtained after converting the wavelet-restored data into the final pixel representation, which can be used for display, storage, or subsequent image processing.

[0049] In addition, in other embodiments of this disclosure, after obtaining the decoded data, the decoded data can also be encapsulated in the CPU. Specifically, the decoded data can be transferred from the GPU back to the CPU; and in the CPU, the decoded data is encapsulated based on the target output format.

[0050] The target output format can represent a predetermined file format used to store or transmit decoded data, which may include format requirements such as bit depth, channel arrangement, pixel arrangement, and data alignment. For example, the decoded data can be encapsulated according to the format specifications of Portable Network Graphics (PNG), Joint Photographic Experts Group (JPEG), Tagged ImageFile Format (TIFF), or Raw Image Format (RAW) to meet the corresponding output requirements. Encapsulation can represent the process of organizing the decoded data into a data structure that conforms to the file format requirements according to the target output format. In this embodiment, encapsulating the decoded data in the CPU allows the formatting operations involving file format organization and output structure construction to be handled by the CPU, which is more suitable for processing control logic instructions, thereby avoiding the occupation of the GPU's parallel computing resources.

[0051] The contents of steps S110 to S140 will be described in detail below.

[0052] In some embodiments, parsing the image data to be decoded to obtain multiple code blocks specifically includes the following technical steps: The CPU reads the image bitstream corresponding to the image data to be decoded and parses the encapsulation header information of the image bitstream to obtain encapsulation header information, wherein the encapsulation header information may include image size, code block division method, level parameters, and entropy coding mode, etc. Then, using the image size, code block division method, and level parameters, the code block data packets in the image bitstream are split to obtain multiple code blocks. Subsequently, the task scheduling module allocates the decoding task of each code block to the corresponding thread block in the GPU, so that the thread block can perform subsequent parallel entropy decoding processing based on the allocated code block. Furthermore, before performing decoding processing, the CPU can also initialize the GPU to ensure that the GPU's thread execution environment, device memory space, and task scheduling channel are in a usable state that accepts subsequent decoding tasks.

[0053] The encapsulation header information represents the structural description data attached to the front end of the image data to be decoded. This structural description data indicates the basic attributes and encoding control parameters of the image. The image size represents the number of pixels in the horizontal and vertical directions of the image data to be decoded. The code block partitioning method represents the data block division rules used to divide the image to be decoded into multiple independent code blocks in the wavelet transform domain. This can include the number of rows and columns of the code blocks, the size of the code blocks, and the positional relationship of the code blocks in the image. The hierarchy parameters represent the configuration data used to describe the wavelet multi-resolution decomposition structure. This can include the number of wavelet decomposition levels, subband structure information, and the resolution ratio between different levels. The entropy coding mode represents the control parameters used to indicate the entropy coding mode adopted by the bitstream of the image data to be decoded during the encoding process. This can include whether to enable high-throughput mode, whether to use the Cleanup Pass, and the combination of the Significance Propagation Pass (SigProp) and Magnitude Refinement Pass (MagRef). In this embodiment, the control / IO-intensive steps of decoding, such as GPU initialization, bitstream reading, encapsulation header parsing, and code block segmentation, are completed by the CPU. This isolates the steps involving format parsing and control decisions in the decoding process from the parallel computing steps of the GPU, thereby avoiding the occupation of GPU's parallel computing resources and improving decoding efficiency.

[0054] In some embodiments, transmitting multiple code blocks to the GPU includes the following technical steps: allocating corresponding video memory in the GPU according to the size of the multiple code blocks; and transmitting the multiple code blocks to the video memory through asynchronous data transmission.

[0055] In this context, video memory (VRAM) refers to a high-speed storage space on the GPU used to store data to be processed and intermediate calculation results. It can be used to store GPU-side processing objects such as code block data, subband coefficients, row and column direction processing data, and pixel reconstruction data to support parallel computing operations on the GPU. Asynchronous data transfer refers to a data transfer method between the CPU and GPU that utilizes a transfer channel independent of the computation path. This transfer method can initiate data transfer without waiting for the current computation process to finish and can be executed in parallel with the GPU's internal computation process. In this embodiment, by allocating VRAM on demand according to the code block size in the GPU and writing code blocks using asynchronous data transfer, data transfer and GPU computation can be carried out in parallel, thereby reducing processing latency caused by transmission waiting.

[0056] Specifically, allocating corresponding video memory in the GPU based on the size of multiple code blocks can include: determining the target storage size based on the bitstream length of each code block in the image bitstream, and rounding the target storage size up according to the GPU's memory alignment rules to obtain the data storage amount for that code block; subsequently, using the GPU's device memory management mechanism, allocating a video memory region matching the data storage amount for each code block, and recording the starting address and byte range of each video memory region in a video memory allocation table, so that subsequent data transmission processes can write the code block data into the corresponding video memory region according to the video memory allocation table.

[0057] Furthermore, multiple code blocks are written to video memory via asynchronous data transfer, specifically including the following steps: writing multiple code blocks to fixed-page memory to obtain fixed-page code block data; transferring the fixed-page code block data to video memory via asynchronous data transfer, and storing the fixed-page code block data in video memory according to a block-based storage method.

[0058] In this context, fixed-page memory refers to host memory with a fixed physical location that can be directly accessed by the GPU. Fixed-page code block data refers to a set of code block data located in fixed-page memory, formed by writing multiple code blocks into fixed-page memory. This data set can serve as the source data for asynchronous data transfer while maintaining physical address continuity. Block storage can refer to a storage layout method in which data is partitioned and stored in video memory according to predetermined two-dimensional block units. This method stores logically adjacent data in the same area to improve memory access locality and reduce video memory bandwidth consumption when threads access cross areas. In this embodiment, fixed-page memory is used to form a contiguous data area that can be directly accessed by the GPU, and code blocks are stored in video memory in a block manner, thus optimizing both data transfer and storage layout, thereby reducing the overall overhead of data transfer and video memory access.

[0059] In the specific implementation process, the CPU can be used to write multiple code blocks sequentially into consecutive physical pages in fixed-page memory, so that the fixed-page code block data forms a continuous data segment. Then, the asynchronous transfer instruction associated with the GPU transfer channel is called, using the fixed-page code block data as the transfer source and the corresponding data segment is written to the GPU-side video memory area according to the target address recorded in the video memory allocation table. During the writing process, the fixed-page code block data can be divided into blocks according to the preset two-dimensional block size of the block storage method, so that the data corresponding to the same two-dimensional block is placed in the same area of ​​video memory, so that subsequent thread blocks can perform centralized memory access based on this block layout when performing parallel decoding.

[0060] In some embodiments, reference Figure 2 As shown, multiple code blocks are entropy-decoded in parallel using multiple thread blocks to obtain subband coefficients. The specific technical steps include the following: Step S210 involves assigning multiple code blocks to multiple thread blocks. Preferably, one thread block processes one code block. For example, at least two thread blocks may be assigned one code block each for parallel entropy decoding, while the remaining thread blocks are assigned at least two code blocks according to available computing resources and entropy decoding is performed sequentially. This improves overall decoding throughput without requiring all code blocks to run in parallel. In the actual decoding process, corresponding code block indices can be generated for each code block; corresponding thread block indices can be generated for each thread block; a one-to-one correspondence can be established between the code block indices and the thread block indices, ensuring that each thread block only obtains its corresponding code block data when the entropy encoding kernel function is executed.

[0061] Step S220: Divide the bit plane data in the code block into multiple threads within the corresponding thread block.

[0062] Here, bit-plane data can represent a set of data obtained by splitting the coefficient values ​​in the code block according to the binary bit order from high bit to low bit, with each bit-plane corresponding to the bit information of the code block coefficients in the binary bits. The bit values ​​of each coefficient in the same binary bit are organized into the same bit-plane, allowing each bit-plane to be allocated as a relatively independent data unit to multiple threads within the thread block. Each thread performs entropy decoding on at least one bit-plane. Preferably, each thread processes at least one bit-plane of data. In this step, by having each thread undertake the entropy decoding task of at least one bit-plane, information from different binary layers within the code block can be parsed in parallel, reducing the time overhead of sequentially traversing bit-plane data and improving the utilization of computing resources in the GPU.

[0063] In step S230, the decoding state data used for context modeling is loaded into the shared memory of the thread block, and multiple threads perform entropy decoding processing on the bit plane data in parallel based on the decoding state data to obtain the subband coefficients.

[0064] Context modeling represents the process of determining the feature category of the current bit position based on the decoding state of its neighborhood position in the code block during entropy decoding, and selecting the processing method for the decoding path used for bit value parsing accordingly. Decoding state data represents a set of states used to characterize the parsing progress of each coefficient in the code block during entropy decoding. This data can include saliency state data, sign state data, and magnitude refinement state data, used to record whether a coefficient has reached saliency, the positive or negative sign of the coefficient, and the bit parsing status of the coefficient magnitude during the refinement process. In this step, by centrally storing the decoding state data in shared memory, multiple threads do not need to frequently access global memory when performing entropy decoding, thereby reducing the number of high-latency memory accesses during the decoding process.

[0065] In some embodiments, loading decoding state data for context modeling into the shared memory of a thread block includes: reading saliency state data, sign state data, and amplitude refinement state data from a code block and loading them into the shared memory of the thread block corresponding to the code block, so that the thread block can perform context modeling during the entropy decoding process. Specifically, the saliency state data represents the marking information indicating whether each coefficient in the code block has reached saliency; the sign state data represents the marking information indicating the positive or negative sign attribute of each coefficient in the code block; and the amplitude refinement state data represents the marking information indicating the bit resolution status of each coefficient's amplitude during the refinement process.

[0066] Specifically, saliency state data can represent a set of states used to record whether each coefficient in the code block has been determined to be saliency before the current bit plane. It is arranged in a manner corresponding to the two-dimensional coefficient matrix of the code block, and stores a corresponding saliency marker at each coefficient position to characterize whether the coefficient has already encountered its first non-zero bit in the previous bit plane resolution, thus providing context for the saliency propagation path and cleanup path resolution in subsequent bit planes. Symbol state data can represent a set of states used to record the symbol attributes of each coefficient in the code block after it becomes saliency. It records the sign (positive or negative) of each saliency coefficient and stores it using an index layout consistent with the two-dimensional structure of the code block, so that it can be read in the amplitude refinement path resolution path during subsequent bit resolution, thereby combining with the symbol context prediction model to resolve symbol bits. Amplitude refinement state data can represent a set of states used to record the bit resolution status of each saliency coefficient in the code block during the amplitude refinement process in different bit planes, used to support the step-by-step resolution of low-order bits in the amplitude refinement path. Furthermore, the refinement process can represent the stage in entropy decoding where the magnitude information of coefficients that have reached saliency is progressively supplemented and updated bit-plane by bit. Bit resolution status can represent the status marker information indicating whether the bit values ​​of the coefficients on each bit plane have been read and used for magnitude updates during the refinement process. In this step, by preloading saliency state data, symbol state data, and magnitude refinement state data into the shared memory of the thread block, the thread can complete context modeling based on high-speed, accessible saliency state data when performing entropy decoding, thereby reducing the bandwidth consumption of off-chip memory access and improving the parallel efficiency of bit-plane resolution.

[0067] Furthermore, in some embodiments, reference is made to... Figure 3 As shown, multiple threads perform entropy decoding on the bit-plane data based on the decoding state data to obtain sub-band coefficients. The specific technical steps include the following: Step S310 involves multiple threads reading decoding state data from shared memory. For example, each thread within a thread block can read saliency state data, symbol state data, and amplitude refinement state data corresponding to its assigned bit plane position from a pre-allocated state region in shared memory, using these data as input for subsequent context modeling and bit value parsing.

[0068] Step S320: Context modeling is performed by each thread based on the decoding state data, and bit value parsing is performed in parallel on each bit position of the bit plane data in the entropy decoding channel based on the results of context modeling.

[0069] The entropy decoding channel represents the set of decoding paths selected based on decoding state data and context modeling results during entropy decoding of bit-plane data. This channel can include a cleanup channel for handling insignificant coefficients, a significance propagation channel for propagating significance, and an amplitude refinement channel for refining the amplitude of significant coefficients. A bit position represents the location in the bit-plane data determined by the row index, column index, and bit-plane index of the coefficient, indicating a single binary bit of the corresponding coefficient on a specific bit-plane. Bit value parsing represents the process of determining the value of a specific bit position in the bit-plane data based on the entropy decoding channel and context modeling results during entropy decoding, and using the parsed bit value to reconstruct the numerical representation of the corresponding coefficient.

[0070] In the specific implementation process, after acquiring the decoding state data, each thread reads the saliency marker, symbol marker, and amplitude refinement marker corresponding to the current bit position to be parsed in the bit plane data from the decoding state data to determine the context category to which the bit position belongs. Subsequently, each thread selects the corresponding entropy decoding path from the cleanup channel, saliency propagation channel, and amplitude refinement channel based on the context category, and uses the entropy decoding path to perform bit value parsing on the current bit position in the bit plane data, so that the parsed bit value can meet the context state requirements of the bit position in the decoding process. Among them, bit value parsing includes performing symbol parsing, amplitude supplementation, or saliency determination on the input bit stream according to the path determined by the context model to obtain the bit value corresponding to the bit position.

[0071] Step S330: Generate subband coefficients based on the results of bit value parsing. Specifically, after completing the bit value parsing of the bit plane data, each thread performs an accumulation and combination operation on multiple bit values ​​belonging to the same coefficient according to a predetermined binary bit order based on the bit value corresponding to each bit position, in order to recover the integer amplitude of the coefficient; subsequently, each thread further determines the sign attribute of the integer amplitude based on the sign marker in the decoded state data, and writes the recovered signed integer value into the corresponding coefficient position, thereby forming subband coefficients arranged in row and column order.

[0072] In this embodiment, by performing context modeling based on shared state and parsing the bit plane bit by bit in the entropy decoding channel, each bit position is restored during the decoding process according to the same saliency, sign and amplitude refinement relationship as in the encoding stage. This ensures the consistency between the subband coefficients and the encoding coefficients, while locally caching the context-related information in the shared storage space, reducing repeated access to the external storage space and reducing memory access bandwidth usage, thereby improving decoding efficiency.

[0073] In some embodiments, performing inverse wavelet transform on the subband coefficients specifically includes the following technical steps: when the wavelet transform encoding is lossy, dequantizing the subband coefficients based on the quantization step size corresponding to the subband coefficients; loading the dequantized subband coefficients from video memory to shared memory; and performing inverse wavelet transform on the dequantized subband coefficients in shared memory in both row and column directions using different thread blocks. Furthermore, when the wavelet transform encoding is lossless, dequantization of the subband coefficients is unnecessary.

[0074] Lossy coding refers to a coding method that uses quantization to process the coefficient amplitudes during wavelet transform coding. This lossy coding maps multiple adjacent amplitudes to the same quantization index, thus losing some high-precision information in the compression stage to achieve a higher compression ratio. The quantization step size represents a preset amplitude interval parameter used in lossy coding to map the wavelet transform-derived coefficient amplitudes to the quantization index. This amplitude interval determines the accuracy of amplitude information retention and serves as the basis for coefficient recovery in the dequantization operation during the decoding stage. Dequantization processing represents the operation of re-expanding the quantization index using the quantization step size during the decoding stage. Specifically, it involves restoring the quantization index to the corresponding integer or floating-point amplitude according to the quantization step size to generate subband coefficients that can be used for inverse wavelet transform. Lossless coding refers to a coding method that does not quantize the coefficient amplitudes or only uses reversible integer wavelet transform during wavelet transform coding. This method can completely restore the original coefficient values ​​before coding in the decoding stage without causing a loss of data precision.

[0075] In this embodiment, by performing dequantization based on the quantization step size under lossy encoding, and then loading the dequantized subband coefficients into shared memory, different thread blocks perform inverse wavelet transforms in the row and column directions respectively. This allows the coefficient amplitudes to be processed in parallel within shared memory while restoring accuracy, thereby reducing external memory access pressure and improving the parallel execution efficiency of the inverse wavelet transform while ensuring the accuracy of coefficient reconstruction. Furthermore, using different thread blocks to perform inverse wavelet transforms on the subband coefficients in the row and column directions respectively allows the inverse wavelet transforms in the row and column directions to be completed within different thread blocks, avoiding repeated memory reads caused by mixed processing, thus effectively reducing memory bandwidth pressure.

[0076] Specifically, the sub-band coefficients are dequantized based on the quantization step size corresponding to the sub-band coefficients. The amplitude of each coefficient in the sub-band coefficients is recovered according to a preset dequantization rule using the quantization step size. The dequantization rule may include multiplying the integer coefficients in the sub-band coefficients with the quantization step size to recover the corresponding amplitude range, or, when there is an offset, compensating for the predetermined offset while performing the multiplication operation, so that the amplitude of the dequantized coefficients can be consistent with the amplitude range of the original wavelet coefficients before encoding.

[0077] In some embodiments, loading the dequantized subband coefficients from video memory to shared memory includes the following technical steps: dividing the dequantized subband coefficients into row direction coefficients and column direction coefficients; loading the row direction coefficients and column direction coefficients in video memory into shared memory respectively.

[0078] In this embodiment, row-direction coefficients represent the set of coefficients extracted from the dequantized subband coefficients arranged in row order. This set of coefficients maintains the adjacency relationship of the original coefficients in the row direction and is used as input data when performing a one-dimensional inverse wavelet transform in the row direction. Column-direction coefficients represent the set of coefficients extracted from the dequantized subband coefficients arranged in column order. This set of coefficients maintains the adjacency relationship of the original coefficients in the column direction and is used as input data when performing a one-dimensional inverse wavelet transform in the column direction. In this embodiment, by dividing the dequantized subband coefficients into rows and columns and loading them into corresponding regions of shared memory, the one-dimensional inverse wavelet transforms in the row and column directions can be performed within their respective continuous data regions. This reduces the memory bandwidth usage caused by cross-row and cross-column memory accesses and improves the memory access efficiency of thread blocks, thereby improving decoding efficiency.

[0079] Furthermore, the subband coefficients corresponding to a code block are processed by two thread blocks, namely a first thread block and a second thread block. The inverse wavelet transforms of the dequantized subband coefficients in the shared memory are performed in the row and column directions using different thread blocks. Specifically, the following technical steps are included: the first thread block is used to perform the inverse wavelet transform of the row direction coefficients in the shared memory in the row direction to obtain the row direction transform result; the second thread block is used to perform the inverse wavelet transform of the row direction transform result and the column direction coefficients in the column direction.

[0080] The first thread block can represent a thread block unit in the GPU scheduled to perform inverse wavelet transform in the row direction. Multiple threads within this thread block process the row direction coefficients in shared memory in parallel according to a predetermined row direction data access order to complete the inverse wavelet transform operation in the row direction. The second thread block can represent a thread block unit in the GPU scheduled to perform inverse wavelet transform in the column direction. This thread block uses the row direction transform result generated by the first thread block and combines it with the column direction coefficients in shared memory to perform inverse wavelet transform operation in the column direction to generate the column direction processing result for image reconstruction. The row direction transform result can represent intermediate reconstructed data obtained by the first thread block performing a one-dimensional inverse wavelet transform in the row direction on the row direction coefficients in shared memory. This intermediate reconstructed data maintains the coefficient arrangement structure after the row direction transform and serves as input data for the second thread block when performing the inverse wavelet transform in the column direction.

[0081] In the specific implementation, multiple threads within the first thread block sequentially read the corresponding coefficient values ​​according to the row order of the row direction coefficients in shared memory, and perform row direction lifting operations based on the preset filtering coefficients of the row direction inverse wavelet transform. After completing the row direction lifting operation, the row direction reconstruction coefficients calculated by each thread are written to the output buffer corresponding to the first thread block to form the row direction transformation result. Subsequently, the second thread block reads the row direction transformation result from the output buffer of the first thread block in column order, and reads the column direction coefficients from shared memory in column order, enabling the second thread block to obtain the corresponding input coefficient column in the column direction. Next, multiple threads within the second thread block perform column direction lifting operations on the input coefficient column based on the preset filtering coefficients of the column direction inverse wavelet transform to obtain wavelet reconstruction data. Here, the wavelet reconstruction data represents a two-dimensional reconstruction matrix in the coefficient domain, where each element corresponds to the intermediate reconstruction coefficient of the image in the spatial domain. Since the wavelet reconstruction data still exists in the form of reconstruction coefficients, it does not yet meet the requirements of the pixel domain's numerical range, channel arrangement, and color space. Therefore, further pixel restoration processing is required to convert the reconstruction coefficients into pixel data that can be used as image output. Furthermore, in this embodiment, the inverse wavelet transform in the row and column directions is separated by thread blocks, enabling the two-stage transform to be executed efficiently and in a pipelined manner within shared memory, thereby reducing repeated memory accesses caused by direction switching and improving the overall computational efficiency of the inverse wavelet transform.

[0082] In some embodiments, pixel restoration processing is performed on wavelet-reconstructed data to obtain decoded data, specifically including the following technical steps: determining the number of thread blocks required for pixel restoration processing based on the image width corresponding to the wavelet-reconstructed data and the number of threads in a single thread block in the row direction in the GPU; determining pixel restoration thread blocks based on the number of thread blocks; calling the pixel restoration kernel function required for pixel restoration processing, and executing the pixel restoration kernel function in parallel on the threads within the pixel restoration thread block to perform pixel restoration processing on the wavelet-reconstructed data; wherein, the processing object of the thread is the pixel column of the wavelet-reconstructed data.

[0083] The number of thread blocks can represent the number of thread blocks used to cover the pixel columns to be restored in the wavelet reconstruction data. A pixel restoration thread block can represent a set of thread blocks used for pixel restoration processing, determined based on the image width of the wavelet reconstruction data and the number of threads in a single thread block in the row direction on the GPU. Each thread in the thread block processes one or more pixel columns in the wavelet reconstruction data. The pixel restoration kernel function can represent a kernel function on the GPU used to perform pixel restoration processing. This kernel function can be scheduled in parallel across multiple target thread blocks and performs at least one operation within each thread: color space transformation, DC component shift transformation, or storage format transformation, on the pixel columns of the wavelet reconstruction data to generate the final decoded data. In this embodiment, by determining the number of target thread blocks based on the image width and executing the pixel restoration kernel function in parallel column-by-column within each thread block, adjacent threads can continuously access the storage address of the wavelet reconstruction data, avoiding the cross-step access problem that occurs during row-by-row processing, thereby improving the memory access efficiency and data throughput in the pixel restoration stage.

[0084] Furthermore, based on the image width corresponding to the wavelet-reconstructed data and the number of threads in a single thread block in the row direction of the GPU, the number of thread blocks required for pixel restoration processing is determined. Specifically, this includes the following technical steps: determining the total number of pixel columns to be restored based on the image width corresponding to the wavelet-reconstructed data; determining the number of covered columns of a single thread block based on the number of threads in a single thread block in the row direction of the GPU; and determining the number of thread blocks required for pixel restoration processing based on the calculation result of dividing the total number of pixel columns by the number of covered columns.

[0085] The total number of pixel columns represents the number of pixel columns contained in the wavelet reconstruction data to be preprocessed along the image width. The number of covered columns represents the number of pixel columns that a single thread block can process simultaneously along the row direction, characterizing the column task range that a single thread block can cover in a single parallel execution. Furthermore, when determining the number of thread blocks required for pixel reconstruction, the total number of pixel columns can be divided by the number of covered columns to obtain an initial calculated value for the number of thread blocks. When the initial calculated value is not an integer, it is rounded up to obtain the number of thread blocks required for pixel reconstruction. When the initial calculated value is an integer, it is used as the number of thread blocks required for pixel reconstruction.

[0086] Furthermore, in the specific implementation, the thread block can be set as a two-dimensional thread block, which includes threads in the row direction and threads in the column direction. The number of threads in the column direction of the two-dimensional thread block is equal to the number of pixel components in the wavelet reconstruction data, so that each column direction thread can process a data channel corresponding to one pixel component. A row direction thread processes one pixel column in the wavelet reconstruction data, and a column direction thread processes one pixel component within that pixel column. During pixel restoration processing, the row direction threads extract the corresponding pixel column data from consecutive memory address ranges according to the arrangement order of pixel columns in the wavelet reconstruction data, and provide this pixel column data to the column direction threads within the same thread block. Simultaneously, after obtaining the multi-channel pixel values ​​corresponding to the pixel column, the column direction threads perform at least one operation—color space transformation, DC component shift, or storage format transformation—on their respective pixel component channels, enabling the processing of different components of the same pixel column to be completed in parallel within the same thread block. The two-dimensional thread structure described above enables the thread to continuously access the storage address of the wavelet reconstruction data in the pixel column direction and synchronously perform processing operations in the pixel component direction, thereby avoiding the problem of step-by-step memory access caused by row-by-row access.

[0087] Furthermore, the pixel restoration process described above includes one or more processing operations such as color space transformation, DC component translation transformation, and storage format transformation; if the pixel restoration process includes at least one processing operation, then at least one processing operation is performed during a single kernel function execution by the pixel restoration kernel function.

[0088] Color space transformation can represent the color domain conversion operation performed on multiple pixel components of the wavelet reconstruction data during the pixel restoration stage. It is used to convert the color representation used in the encoding stage to the color space required by the target output format. DC-level shift transformation can represent the integer shift operation performed on each pixel component in the wavelet reconstruction data according to the discrete wavelet coefficient offset rules of the encoding stage. It is used to restore the numerical range of the decoded coefficients to the normal pixel domain range. Storage format transformation can represent the processing operation of converting the arrangement structure of the wavelet reconstruction data in memory from the format used in the decoding stage to the pixel arrangement format required by the target output format. For example, storage format transformation may include conversion from planar format to packed format. Furthermore, color space transformation, DC-level shift transformation, and storage format transformation are performed sequentially.

[0089] Specifically, the corresponding pixel restoration processing steps can be selected based on the specific format of the wavelet-reconstructed data. If the wavelet-reconstructed data is in YUV format and the decoding output requires RGB format, then color space transformation of the wavelet-reconstructed data is necessary. If the wavelet-reconstructed data is in an independent storage format (planar format) and the decoding output requires an interleaved storage format, then the independent storage format can be transformed into an interleaved storage format through storage format transformation. An independent storage format can represent a pixel data layout where each color component of the image is stored separately in physical memory, in contiguous and non-overlapping independent storage areas. An interleaved storage format can represent a pixel data layout where multiple color components of the same pixel are arranged adjacently in a predetermined order and alternately stored in memory. Furthermore, by performing at least one processing operation during a single kernel function execution, the pixel restoration process can be completed within a single kernel function execution. This eliminates the need for multiple calls to different kernel functions during the pixel restoration stage, thereby reducing thread rescheduling and repeated memory access overhead, and improving the execution efficiency and data processing continuity of the pixel restoration process.

[0090] Figure 4 A schematic flowchart of another image decoding method of this disclosure is shown. (Referring to...) Figure 4 As shown, the image decoding method may include the following steps: Step 401, Data Parsing and Task Allocation. Specifically, the CPU reads the image bitstream corresponding to the image data to be decoded and parses the encapsulation header information of the image bitstream to obtain encapsulation header information containing image size, code block division method, level parameters, and entropy coding mode; based on the image size and code block division method, the image bitstream is split into multiple code blocks; subsequently, the task scheduling module allocates the decoding task corresponding to each code block to the corresponding thread block in the GPU, and initializes the GPU before the allocation process to ensure that the thread execution environment and video memory space are available for subsequent decoding tasks.

[0091] Step 402, Entropy Decoding Process. In actual operation, firstly, the bit plane data in each code block is divided into multiple threads within the corresponding thread block, enabling each thread to execute the entropy decoding process for at least one bit plane in that code block; then, saliency state data, symbol state data, and amplitude refinement state data are read from the code block and loaded into the shared memory of the corresponding thread block, allowing the threads to perform context modeling based on the shared decoding state data; next, each thread performs bit-by-bit parsing of the bit plane data in the entropy decoding channel, which includes a cleanup channel, a saliency propagation channel, and an amplitude refinement channel, based on the context modeling results; finally, the subband coefficients corresponding to the code block are generated based on the bit-value parsing results.

[0092] Step 403, inverse quantization. Specifically, if the wavelet transform coding is lossy, the subband coefficients are inverse quantized based on the quantization step size corresponding to the subband coefficients. If the wavelet transform coding is lossless, the inverse quantization step is skipped and the process directly proceeds to the inverse wavelet transform stage.

[0093] Step 404, Inverse Wavelet Transform. Specifically, the dequantized subband coefficients are divided into row direction coefficients and column direction coefficients; then, the row direction coefficients and column direction coefficients are loaded into the corresponding regions of shared memory; next, the first thread block performs an inverse wavelet transform in the row direction on the row direction coefficients in shared memory to obtain the row direction transform result; then, the second thread block uses the row direction transform result and the column direction coefficients to perform an inverse wavelet transform in the column direction to obtain the wavelet-reconstructed data.

[0094] Step 405, pixel restoration processing. Specifically, the required number of target thread blocks is determined based on the image width of the wavelet-reconstructed data and the number of threads in a single thread block in the row direction of the GPU; then, the pixel restoration kernel function is called and executed in parallel in multiple target thread blocks, so that each thread in the target thread block performs pixel restoration processing on the corresponding pixel column in the wavelet-reconstructed data.

[0095] Step 406: Encapsulation based on the target output format. Specifically, the decoded data is transferred from the GPU back to the CPU, and the CPU encapsulates the decoded data based on the target output format.

[0096] Furthermore, in the implementation process, entropy decoding, inverse quantization, inverse wavelet transform, and pixel restoration can be encapsulated into independent GPU kernel functions. Multiple kernel functions can then be scheduled in parallel using CUDA Stream (Compute Unified Device Architecture Stream) or multi-queue scheduling, enabling a decoding-level pipeline execution relationship between different code blocks and different kernel functions. Under this scheduling mechanism, the GPU can load data for the next code block while performing entropy decoding on the current code block, allowing multiple code blocks to be executed alternately between different GPU kernel functions, thus constructing an approximately pipelined parallel decoding process.

[0097] It should be noted that although the steps of the method in this disclosure are described in a specific order in the accompanying drawings, this does not require or imply that the steps must be performed in that specific order, or that all the steps shown must be performed to achieve the desired result. Additional or alternative steps may be omitted, multiple steps may be combined into one step, and / or a step may be broken down into multiple steps.

[0098] Furthermore, in this exemplary embodiment, an image decoding apparatus is also provided. (Refer to...) Figure 5 As shown, the image decoding device 500 includes a data parsing module 510, an entropy decoding module 520, a wavelet transform module 530, and a pixel recovery module 540. Wherein: The data parsing module 510 can be used to parse the image data to be decoded to obtain multiple code blocks, and transmit the multiple code blocks to the entropy decoding module 520. The image data to be decoded is obtained through wavelet transform encoding. The entropy decoding module 520 can be used to perform entropy decoding processing on multiple code blocks in parallel through multiple thread blocks to obtain subband coefficients. The wavelet transform module 530 can be used to perform inverse wavelet transform on the subband coefficients to obtain wavelet restored data. The pixel restoration module 540 can be used to perform pixel restoration processing on the wavelet restored data to obtain decoded data.

[0099] The specific details of each module of the above image decoding device have been described in detail in the corresponding image decoding methods, so they will not be repeated here.

[0100] It should be noted that although several modules or units of the image decoding apparatus have been mentioned in the detailed description above, this division is not mandatory. In fact, according to embodiments of this disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0101] Furthermore, in an exemplary embodiment of this disclosure, an electronic device capable of implementing the above-described image decoding method is also provided.

[0102] Those skilled in the art will understand that various aspects of this disclosure can be implemented as a system, method, or program product. Therefore, various aspects of this disclosure can be embodied in the following forms: a completely hardware embodiment, a completely software embodiment (including firmware, microcode, etc.), or an embodiment combining hardware and software aspects, collectively referred to herein as a "circuit," "module," or "system."

[0103] The following reference Figure 6 To describe an image decoding device 600 according to such an embodiment of the present disclosure. Figure 6 The image decoding device 600 shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments disclosed herein.

[0104] like Figure 6As shown, the image decoding device 600 is presented in the form of a general-purpose computing device. The components of the image decoding device 600 may include, but are not limited to: at least one processor 610, at least one memory 620, a bus 630 connecting different system components (including memory 620 and processor 610), and a display unit 640. The processor includes a CPU and a GPU.

[0105] The storage unit stores program code that can be executed by the processor 610, causing the processor 610 to perform the steps described in the "Exemplary Methods" section of this specification according to various exemplary embodiments of this disclosure. For example, the processor 610 can perform actions such as... Figure 1 The steps are shown in the figure.

[0106] The memory 620 may include a readable medium in the form of volatile memory cells, such as random access memory (RAM) 621 and / or cache 622, and may further include read-only memory (ROM) 623.

[0107] The memory 620 may also include a program / utility 624 having a set (at least one) of program modules 625, including but not limited to: an operating system, one or more application programs, other program modules, and program data, each or some combination of these examples may include an implementation of a network environment.

[0108] Bus 630 can represent one or more of several types of bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the various bus structures.

[0109] The image decoding device 600 can also communicate with one or more external devices 670 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), one or more devices that enable users to interact with the image decoding device 600, and / or any device that enables the image decoding device 600 to communicate with one or more other computing devices (e.g., routers, modems, etc.). This communication can be performed via input / output (I / O) interface 650. Furthermore, the image decoding device 600 can also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via network adapter 660. As shown, network adapter 660 communicates with other modules of the image decoding device 600 via bus 630. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with the image decoding device 600, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0110] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, terminal device, or network device, etc.) to execute the methods according to the embodiments of this disclosure.

[0111] In exemplary embodiments of this disclosure, a computer-readable storage medium is also provided, on which a program product capable of implementing the methods described above is stored. In some possible embodiments, various aspects of this disclosure may also be implemented as a program product including program code that, when the program product is run on a terminal device, causes the terminal device to perform the steps described in the "Exemplary Methods" section of this specification according to various exemplary embodiments of this disclosure.

[0112] refer to Figure 7 As shown, a program product 700 for implementing the above-described image decoding method according to an embodiment of the present disclosure is described. This product may employ a portable compact disc read-only memory (CD-ROM) and include program code, and may run on a terminal device, such as a personal computer. However, the program product of the present disclosure is not limited thereto. In this document, a readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, apparatus, or device.

[0113] The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0114] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium, capable of sending, propagating, or transmitting programs for use by or in conjunction with an instruction execution system, apparatus, or device.

[0115] The program code contained on the readable medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0116] Program code for performing the operations of this disclosure can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0117] Furthermore, the above figures are merely illustrative of the processes included in the method according to exemplary embodiments of this disclosure and are not intended to be limiting. It is readily understood that the processes shown in the above figures do not indicate or limit the temporal order of these processes. Additionally, it is readily understood that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0118] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, touch terminal, or network device, etc.) to execute the methods according to the embodiments of this disclosure.

[0119] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and embodiments are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the claims.

[0120] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. An image decoding method, characterized in that, Applied to an image decoding system, the image decoding system including a central processing unit (CPU) and a graphics processing unit (GPU), the method includes: The CPU parses the image data to be decoded to obtain multiple code blocks, and transmits the multiple code blocks to the GPU. The image data to be decoded is obtained through wavelet transform coding. The GPU performs entropy decoding on the multiple code blocks in parallel using multiple thread blocks to obtain subband coefficients. The GPU performs an inverse wavelet transform on the subband coefficients to obtain wavelet-reconstructed data; The GPU performs pixel recovery processing on the wavelet-reconstructed data to obtain decoded data.

2. The image decoding method according to claim 1, characterized in that, The process of performing entropy decoding on the multiple code blocks in parallel using multiple thread blocks to obtain subband coefficients includes: Assign the plurality of code blocks to the plurality of thread blocks; The bit plane data in the code block is divided into multiple threads within the corresponding thread block; The decoding state data used for context modeling is loaded into the shared memory of the thread block, and the multiple threads perform entropy decoding processing on the bit plane data in parallel based on the decoding state data to obtain the subband coefficients.

3. The image decoding method according to claim 2, characterized in that, The step of loading the decoded state data used for context modeling into the shared memory of the thread block includes: Significant state data, symbol state data, and amplitude refinement state data are read from the code block and loaded into the shared memory of the thread block corresponding to the code block, so that the thread block can perform context modeling in the entropy decoding process; The significance status data indicates whether each coefficient in the code block has reached significance; the sign status data indicates the sign attribute of each coefficient in the code block; and the amplitude refinement status data indicates the bit resolution status of each coefficient in the code block during the refinement process.

4. The image decoding method according to claim 2, characterized in that, The process of performing entropy decoding on the bit plane data in parallel by the multiple threads based on the decoding state data to obtain the sub-band coefficients includes: The decoding status data is read from the shared memory by the multiple threads respectively; Each thread performs context modeling based on the decoding state data, and in the entropy decoding channel, the bit value is parsed in parallel for each bit position of the bit plane data according to the result of the context modeling. The subband coefficients are generated based on the results of the bit value parsing.

5. The image decoding method according to claim 1, characterized in that, The transmission of the plurality of code blocks to the GPU includes: The GPU allocates corresponding video memory according to the size of the multiple code blocks; The multiple code blocks are transmitted to the video memory using an asynchronous data transmission method.

6. The image decoding method according to claim 5, characterized in that, The step of writing the multiple code blocks into the video memory via asynchronous data transmission includes: Write the multiple code blocks into a fixed-page memory to obtain fixed-page code block data; The fixed page number block data is transmitted to the video memory via asynchronous data transmission, and the fixed page number block data is stored in the video memory in a block-based storage manner.

7. The image decoding method according to claim 1, characterized in that, The inverse wavelet transform of the sub-band coefficients includes: When the wavelet transform coding is lossy coding, the sub-band coefficients are dequantized based on the quantization step size corresponding to the sub-band coefficients. Load the dequantized subband coefficients from video memory to shared memory; Different thread blocks are used to perform inverse wavelet transforms on the dequantized subband coefficients in the shared memory in both the row and column directions.

8. The image decoding method according to claim 7, characterized in that, The step of loading the dequantized subband coefficients from video memory to shared memory includes: The dequantized subband coefficients are divided into row direction coefficients and column direction coefficients; The row direction coefficients and column direction coefficients in the video memory are loaded into the shared memory respectively.

9. The image decoding method according to claim 8, characterized in that, The subband coefficients corresponding to a code block are processed by two thread blocks, which include a first thread block and a second thread block. The step of performing inverse wavelet transforms on the dequantized subband coefficients in the shared memory using different thread blocks in both the row and column directions includes: The first thread block is used to perform an inverse wavelet transform on the row direction coefficients in the shared memory to obtain the row direction transformation result; The second thread block is used to perform an inverse wavelet transform in the column direction on the row direction transformation result and the column direction coefficient.

10. The image decoding method according to claim 1, characterized in that, The step of performing pixel restoration processing on the wavelet-reconstructed data to obtain decoded data includes: Based on the image width corresponding to the wavelet reconstruction data and the number of threads in a single thread block in the row direction of the GPU, the number of thread blocks required for the pixel restoration process is determined. The pixel recovery thread block is determined based on the number of thread blocks; The pixel recovery kernel function required for the pixel recovery processing is invoked, and the pixel recovery kernel function is executed in parallel on a thread within the pixel recovery thread block to perform pixel recovery processing on the wavelet restored data; wherein, the processing object of the thread is the pixel column of the wavelet restored data.

11. The image decoding method according to claim 10, characterized in that, The step of determining the number of thread blocks required for pixel restoration processing based on the image width corresponding to the wavelet-reconstructed data and the number of threads in a single thread block in the row direction of the GPU includes: The total number of pixel columns to be restored is determined based on the image width corresponding to the wavelet restored data. The number of columns covered by a single thread block is determined based on the number of threads in the row direction of a single thread block in the GPU; The number of thread blocks required for the pixel recovery process is determined based on the calculation result of dividing the total number of pixel columns by the number of covered columns.

12. The image decoding method according to claim 10, characterized in that, The pixel restoration process includes one or more of the following processing operations: color space transformation, DC component shift transformation, and storage format transformation; the method further includes: If the pixel recovery process includes at least one processing operation, then the at least one processing operation is executed during a single kernel function execution by the pixel recovery kernel function.

13. The image decoding method according to claim 1, characterized in that, The method further includes: The decoded data is transmitted from the GPU back to the CPU; In the CPU, the decoded data is encapsulated based on the target output format.

14. An image decoding device, characterized in that, The device includes: The data parsing module is used to parse the image data to be decoded to obtain multiple code blocks, and transmit the multiple code blocks to the entropy decoding module, wherein the image data to be decoded is obtained by wavelet transform coding. The entropy decoding module is used to perform entropy decoding processing on the multiple code blocks in parallel through multiple thread blocks to obtain subband coefficients; The wavelet transform module is used to perform inverse wavelet transform on the sub-band coefficients to obtain wavelet-reconstructed data; The pixel restoration module is used to perform pixel restoration processing on the wavelet-restored data to obtain decoded data.

15. An image decoding device, characterized in that, include: Processors, including CPUs and GPUs; as well as Memory for storing the executable instructions of the processor; The processor is configured to execute the image decoding method according to any one of claims 1-13 by executing the executable instructions.

16. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the image decoding method according to any one of claims 1-13.

Images