Image coding method and device, equipment and storage medium

Abstract

The invention provides an image coding method and device, equipment and a storage medium, and relates to the technical field of computers. The method comprises the following steps that: a CPU (Central Processing Unit) analyzes an image to be coded in a wavelet transform coding process to obtain original pixel data, and transmits the original pixel data to a GPU (Graphics Processing Unit); the GPU performs preprocessing operation on the original pixel data in parallel by using a plurality of threads to obtain standard pixel data meeting the wavelet transform requirement; wherein one thread processes one pixel column in the original pixel data; the GPU performs forward wavelet transform on the standard pixel data to obtain a sub-band coefficient; and the GPU performs entropy coding processing on the sub-band coefficients to obtain coded data. According to the scheme, image analysis is completed in the CPU, and preprocessing, wavelet transform and entropy coding processing are executed in parallel in the GPU, so that the coding throughput rate and the resource utilization rate in the image coding process can be improved.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and more specifically, to an image encoding method, apparatus, device, and storage medium. Background Technology

[0002] In wavelet transform-based coding, multiple data processing steps need to be performed sequentially on the image to be coded. Under the existing computing architecture, relevant wavelet transform coding schemes, such as High-Throughput JPEG 2000 (HTJ2K) coding, typically complete the various coding processing steps of the image to be coded serially in the CPU.

[0003] While encoding independently within the CPU is simple to implement and highly versatile, the limited number of CPU cores makes it difficult to perform sufficient parallel computing when processing high-resolution images, thus limiting the overall encoding speed. When encoding ultra-large or batch images, each processing step must be completed serially, resulting in a significant increase in time overhead and making it difficult to meet the requirements of real-time encoding. Summary of the Invention

[0004] The purpose of this disclosure is to provide an image encoding method, image encoding apparatus, image encoding device, and computer-readable storage medium, which can improve the encoding throughput and resource utilization in the image encoding process by performing image parsing in the CPU and performing preprocessing in the GPU in a multi-threaded parallel manner with pixel columns as the processing objects.

[0005] Other features and advantages of this disclosure will become apparent from the following detailed description, or may be learned in part from practice of this disclosure.

[0006] According to a first aspect of the present disclosure, an image encoding method is provided, applied to an image encoding system, the image encoding system including a central processing unit (CPU) and a graphics processing unit (GPU), the method comprising: the CPU parsing an image to be encoded during wavelet transform encoding to obtain raw pixel data, and transmitting the raw pixel data to the GPU; the GPU using multiple threads to perform preprocessing operations on the raw pixel data in parallel to obtain standard pixel data that meets the requirements of wavelet transform; wherein, one thread processes a pixel column in the raw pixel data; the GPU performing a forward wavelet transform on the standard pixel data to obtain subband coefficients; and the GPU performing entropy encoding processing on the subband coefficients to obtain encoded data.

[0007] In some example embodiments of this disclosure, based on the foregoing scheme, the step of using multiple threads to perform preprocessing operations on the raw pixel data in parallel includes: determining the number of thread blocks required for the preprocessing operation based on the image width and the number of threads in a single thread block in the row direction of the GPU; determining multiple thread blocks corresponding to the preprocessing operation based on the number of thread blocks; calling the preprocessing kernel function required for the preprocessing operation, and executing the preprocessing kernel function in parallel on the threads in the multiple thread blocks to perform preprocessing operations on the raw pixel data.

[0008] In some example embodiments of this disclosure, based on the foregoing scheme, determining the number of thread blocks required for the preprocessing operation based on the image width and the number of threads in a single thread block in the row direction of the GPU includes: determining the total number of pixel columns to be preprocessed based on the image width; determining the number of covered columns of a single thread block based on the number of threads in a single thread block in the row direction of the GPU; and determining the number of thread blocks required for the preprocessing operation based on the calculation result of dividing the total number of pixel columns by the number of covered columns.

[0009] In some example embodiments of this disclosure, based on the foregoing scheme, the thread block is a two-dimensional thread block; the two-dimensional thread block includes threads in the row direction and threads in the column direction; the number of threads in the column direction of the two-dimensional thread block is equal to the number of pixel components of the original pixel data; the parallel execution of the preprocessing kernel function on the threads in the plurality of thread blocks includes: executing the preprocessing kernel function in parallel on the threads in the row direction and the threads in the column direction of the plurality of two-dimensional thread blocks; wherein, one thread in the row direction processes one pixel column in the original pixel data, and one thread in the column direction processes one pixel component within the pixel column.

[0010] In some example embodiments of this disclosure, based on the foregoing scheme, the original pixel data is in an interleaved storage format and the component representation of the original pixel data needs to undergo color space transformation; the interleaved storage format indicates that the pixel components in the original pixel data are arranged alternately in pixel order; the parallel execution of the preprocessing kernel function on the threads in the plurality of thread blocks includes: if the preprocessing kernel function is a first kernel function, then the first kernel function is executed in parallel on the threads in the plurality of thread blocks; wherein, the first kernel function is a kernel function that can complete storage format conversion, DC component shift processing and color space transformation in a single kernel function execution.

[0011] In some example embodiments of this disclosure, based on the foregoing scheme, the step of executing the preprocessing kernel function in parallel on threads in the plurality of thread blocks to perform preprocessing operations on the original pixel data further includes: if the preprocessing kernel function includes a second kernel function capable of completing storage format conversion and DC component shift processing in a single kernel function execution, and a third kernel function for color space transformation, then the second kernel function is executed in parallel on threads in the plurality of thread blocks to perform the storage format conversion and DC component shift processing on the original pixel data; after the storage format conversion and DC component shift processing are completed, the third kernel function is executed in parallel on threads in the plurality of thread blocks to perform the color space transformation on the original pixel data.

[0012] In some example embodiments of this disclosure, based on the foregoing scheme, the original pixel data is in an independent storage format and the component representation of the original pixel data needs to undergo color space transformation; the independent storage format means that each pixel component in the original pixel data is stored in partitions; the parallel execution of the preprocessing kernel function on the threads in the plurality of thread blocks includes: if the preprocessing kernel function is a fourth kernel function, then the fourth kernel function is executed in parallel on the threads in the plurality of thread blocks; wherein, the fourth kernel function is a kernel function that can complete DC component translation processing and color space transformation in a single kernel function execution.

[0013] In some example embodiments of this disclosure, based on the foregoing scheme, the step of transmitting the original pixel data to the GPU includes: allocating corresponding video memory in the GPU according to the size of the original pixel data; and transmitting the original pixel data to the video memory via asynchronous data transmission.

[0014] In some example embodiments of this disclosure, based on the foregoing scheme, the step of performing a forward wavelet transform on the standard pixel data to obtain subband coefficients includes: dividing the standard pixel data into multiple sub-blocks; loading the pixel data corresponding to the sub-blocks from the video memory to shared memory; performing the forward wavelet transform on the pixel data corresponding to the sub-blocks in the shared memory to obtain the subband coefficients, and writing the subband coefficients from the shared memory back to the video memory.

[0015] In some example embodiments of this disclosure, based on the foregoing scheme, dividing the standard pixel data into multiple sub-blocks includes: determining the target size and critical support region corresponding to the sub-block based on a preset wavelet kernel, wherein the wavelet kernel is a filtering operator used when performing the forward wavelet transform, and the critical support region represents the region located at the edge of the sub-block and used for boundary calculation in the forward wavelet transform; dividing the standard pixel data based on the target size and the critical support region to obtain the multiple sub-blocks.

[0016] In some example embodiments of this disclosure, based on the foregoing scheme, performing the forward wavelet transform on the pixel data corresponding to the sub-block in the shared memory includes: performing boundary expansion processing on the pixel data corresponding to the sub-block in the shared memory to obtain the corresponding expanded sub-block; if the wavelet transform encoding is lossy encoding, calling the fifth kernel function, and using the fifth kernel function to perform the forward wavelet transform and quantization processing on the expanded sub-block; wherein, the fifth kernel function is a kernel function that combines and executes the forward wavelet transform and quantization processing.

[0017] In some example embodiments of this disclosure, based on the foregoing scheme, the entropy coding process for the sub-band coefficients includes: dividing the sub-band coefficients based on a preset code block size to obtain multiple code blocks; calling the entropy coding kernel function required for the entropy coding process, and executing the entropy coding kernel function in parallel on multiple threads to perform entropy coding process on the multiple code blocks respectively; wherein, the entropy coding kernel function is related to the encoding method of the wavelet transform coding.

[0018] In some example embodiments of this disclosure, based on the foregoing scheme, the image encoding method further includes: transmitting the encoded data from the GPU back to the CPU; and the CPU encapsulating the encoded data based on a target output format.

[0019] According to a second aspect of the present disclosure, an image encoding apparatus is provided for wavelet transform encoding. The apparatus includes an image parsing module for parsing an image to be encoded during the wavelet transform encoding process to obtain raw pixel data and transmitting the raw pixel data to a preprocessing module; a preprocessing module for performing preprocessing operations on the raw pixel data in parallel using multiple threads to obtain standard pixel data that meets the requirements of wavelet transform; wherein, one thread processes one pixel column in the raw pixel data; a wavelet transform module for performing a forward wavelet transform on the standard pixel data to obtain subband coefficients; and an entropy encoding module for performing entropy encoding processing on the subband coefficients to obtain encoded data.

[0020] According to a third aspect of the present disclosure, an image encoding apparatus is provided, comprising: a processor, including a CPU and a GPU; and a memory storing computer-readable instructions that, when executed by the processor, implement the image encoding method as described in the first aspect.

[0021] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the image encoding method as described in the first aspect.

[0022] The technical solutions provided in this disclosure may have the following beneficial effects: The image encoding method in this embodiment first performs image parsing in the CPU and preprocessing, forward wavelet transform, and entropy encoding in the GPU. This allows the parsing step to be completed in the CPU and the computationally intensive steps to be completed in the GPU, resulting in a more reasonable workload distribution across different processors. Furthermore, multiple threads are used to process corresponding pixel columns in the original pixel data in parallel to complete the preprocessing operations. On the one hand, by distributing the original pixel data to different threads along the column dimension, the preprocessing operations that would normally be performed sequentially can be carried out simultaneously in a multi-threaded manner within the same stage. This results in higher thread parallelism and computational resource utilization during the preprocessing stage, improving the overall data processing throughput. On the other hand, the column-based concurrent processing method allows threads to access data that is continuously arranged in the storage space during execution, avoiding the strafing memory access behavior that is prone to occur in the traditional row-based concurrent method, improving the actual memory access throughput, and reducing performance loss caused by memory access serialization.

[0023] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0024] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure. It is obvious that the drawings described below are merely some embodiments of this disclosure, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0025] Figure 1 The illustration shows a schematic flowchart of an image encoding method according to some embodiments of the present disclosure.

[0026] Figure 2The illustration shows a flowchart of parallel preprocessing operations using thread blocks according to some embodiments of the present disclosure.

[0027] Figure 3 The schematic diagram illustrates a process for obtaining subband coefficients according to some embodiments of the present disclosure.

[0028] Figure 4 The schematic diagram illustrates a flow chart of an image encoding method according to some other embodiments of the present disclosure.

[0029] Figure 5 A block diagram schematically illustrates an image encoding apparatus according to some embodiments of the present disclosure.

[0030] Figure 6 The schematic diagram illustrates the structural schematic of a computer system of an image encoding apparatus according to some embodiments of the present disclosure.

[0031] Figure 7 A schematic diagram of a computer-readable storage medium according to some embodiments of the present disclosure is shown.

[0032] In the accompanying drawings, the same or corresponding reference numerals indicate the same or corresponding parts. Detailed Implementation

[0033] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this specification. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this specification as detailed in the appended claims.

[0034] The terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to be limiting of this specification. The singular forms “a,” “the,” and “the” as used in this specification and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any and all possible combinations of one or more of the associated listed items.

[0035] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be more thorough and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art.

[0036] Furthermore, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. Numerous specific details are provided in the following description to give a thorough understanding of embodiments of this disclosure. However, those skilled in the art will recognize that the technical solutions of this disclosure can be practiced without one or more of the specific details, or other methods, components, apparatuses, steps, etc., can be employed. In other instances, well-known methods, apparatuses, implementations, or operations are not shown or described in detail to avoid obscuring various aspects of this disclosure.

[0037] Furthermore, the accompanying drawings are for illustrative purposes only and are not necessarily drawn to scale. The block diagrams shown in the drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0038] In this example embodiment, an image coding method is first provided, which can be applied to an image coding system capable of performing wavelet transform coding. The image coding system can represent a combination of computing devices for performing multi-stage coding processing on input image data, including a central processing unit (CPU) for parsing and scheduling, and a graphics processing unit (GPU) for high-parallel data processing. These two components work together to complete the coding process from image parsing, data preprocessing, forward wavelet transform to entropy coding.

[0039] Furthermore, the wavelet transform coding in this disclosure may include wavelet transform coding methods based on JPEG 2000 (Joint Photographic Experts Group 2000, J2K) and High-Throughput JPEG 2000 (HTJ2K), etc. Figure 1 A schematic flowchart illustrating an image encoding method according to some embodiments of the present disclosure is shown. Reference Figure 1 As shown, the image encoding method may include the following steps: Step S110: The CPU parses the image to be encoded during the wavelet transform encoding process to obtain the original pixel data and transmits the original pixel data to the GPU. In step S120, the GPU uses multiple threads to perform preprocessing operations on the original pixel data in parallel to obtain standard pixel data that meets the requirements of wavelet transform; wherein, one thread processes one column of pixels in the original pixel data. Step S130: The GPU performs a forward wavelet transform on the standard pixel data to obtain the sub-band coefficients; In step S140, the GPU performs entropy encoding on the subband coefficients to obtain encoded data.

[0040] According to the image encoding method in this example embodiment, firstly, image parsing is performed in the CPU, while preprocessing, forward wavelet transform, and entropy encoding are performed in the GPU. This allows the parsing step to be completed in the CPU, and the computationally intensive steps to be completed in the GPU, thus creating a more reasonable workload distribution across different processors in the encoding process. Furthermore, multiple threads are used to process corresponding pixel columns in the original pixel data separately, completing the preprocessing operation on the original pixel data in parallel. On the one hand, by distributing the original pixel data to different threads along the column dimension, the preprocessing operations that originally needed to be completed sequentially can be carried out simultaneously in a multi-threaded manner within the same stage, thereby achieving higher thread parallelism and computational resource utilization in the preprocessing stage and improving the overall data processing throughput. On the other hand, the column-based concurrent processing method allows threads to access data that is continuously arranged in the storage space during execution, thus avoiding the strafing memory access behavior that is prone to occur in the traditional row-based concurrent method, improving the actual memory access throughput, and reducing the performance loss caused by memory access serialization.

[0041] The image encoding method in this example embodiment will be further explained below.

[0042] In step S110, the CPU parses the image to be encoded during the wavelet transform encoding process to obtain the original pixel data and transmits the original pixel data to the GPU.

[0043] Wavelet transform coding can be described as an encoding mechanism based on the principle of multi-scale decomposition, which performs forward wavelet transform, quantization, and entropy coding on the input image to generate compressed data. The image to be encoded can represent the input image data read by the CPU at the beginning of the encoding process and requiring compression processing via wavelet transform coding. The raw pixel data can represent the unprocessed basic pixel data obtained by the central processing unit after parsing the image to be encoded, stored according to image resolution and pixel component arrangement. For example, the raw pixel data can be three-channel RGB image data, or three-channel YUV image data composed of luminance / chrominance components, or grayscale image data containing single-channel grayscale information; furthermore, the raw pixel data can also be multi-component image data composed of several independent pixel components, such as multispectral pixel data output by remote sensing imaging equipment or multi-channel pixel data output by medical imaging equipment.

[0044] In this step, by completing the parsing of the image to be encoded in the CPU, the operations involved in the parsing process, such as format reading, data header information parsing, and data structure decomposition, are executed by the CPU, which is more suitable for handling complex instruction streams. This avoids having tasks with such control logic as the main function executed by the GPU, which is not good at handling serial parsing processes. As a result, the GPU can concentrate its computing resources to handle subsequent large-scale parallel computing steps, thereby improving the overall execution efficiency and processing throughput of the encoding process.

[0045] In step S120, the GPU uses multiple threads to perform preprocessing operations on the original pixel data in parallel to obtain standard pixel data that meets the requirements of wavelet transform; wherein, one thread processes one pixel column of the original pixel data.

[0046] In this context, a thread can represent a basic computational unit in the GPU used to execute data processing instructions. It processes corresponding pixel columns in the original pixel data along the column direction according to a predetermined scheduling method. Preprocessing can be represented as a data transformation process performed on the original pixel data before wavelet transform, used to ensure that the data format and amplitude meet the input requirements of wavelet decomposition. Standard pixel data can represent data generated after preprocessing and directly usable as wavelet transform input, with its data format and pixel component arrangement conforming to the processing requirements of wavelet transform. In actual operation, multiple threads process corresponding pixel columns in the original pixel data along the row direction. The row direction can represent the thread index direction consistent with the image width direction. In this direction, adjacent threads are assigned different pixel column numbers, ensuring each thread corresponds to a fixed pixel column. Subsequently, each thread sequentially accesses pixel data at different row positions within its corresponding pixel column along the column direction and performs preprocessing operations on the accessed pixel data, thus forming a parallel approach where pixel column allocation is completed along the row direction and pixel data processing is completed along the column direction.

[0047] In this step, the raw pixel data is distributed across different threads for processing along the column dimension. This ensures that the physical addresses of the pixel data accessed by each thread during preprocessing are continuously distributed in the video memory. Since raw pixel data is typically stored sequentially along the row direction, using column-based concurrency means that each thread's access to pixel components within a column corresponds to reading continuous linear segments in the video memory. This avoids the straddling memory access pattern that occurs when different threads access data spanning entire row widths in row-based concurrency. In straddling memory access, threads need to access non-contiguous addresses scattered across multiple memory partitions, which can easily lead to bank conflicts and serialize video memory access. The column-based concurrency in this step reduces the probability of such bank conflicts, allowing multiple threads to complete memory access operations with higher parallelism. Therefore, the column-based concurrency preprocessing operation in this step can improve video memory access throughput, reduce performance losses caused by memory access conflicts, and thus improve overall data processing efficiency.

[0048] In step S130, the GPU performs a forward wavelet transform on the standard pixel data to obtain subband coefficients.

[0049] The forward wavelet transform can be described as a process based on multi-scale analysis principles, decomposing input standard pixel data along the row and column directions. This process decomposes the standard pixel data into low-frequency and high-frequency sub-band coefficients through prediction and update steps. The low-frequency sub-band coefficients reflect the overall contour and smoothness information of the image, while the high-frequency sub-band coefficients reflect the detailed changes in the image in the horizontal, vertical, and diagonal directions. For example, the forward wavelet transform can be a forward discrete wavelet transform (FDWT), a reversible wavelet transform based on integer wavelet kernels, or an irreversible wavelet transform based on floating-point wavelet kernels. Furthermore, the forward wavelet transform can employ 5 / 3 or 9 / 7 filter banks to achieve row and column decomposition, generating multi-resolution frequency domain representations.

[0050] Furthermore, subband coefficients can represent multiple sets of coefficient data corresponding to different frequency bands and different directions obtained after forward wavelet transform. Specifically, for one-dimensional wavelet transform, subband coefficients can include low-frequency subband coefficients (L) and high-frequency subband coefficients (H); for two-dimensional wavelet transform, subband coefficients can include low-frequency-low-frequency (LL) subband coefficients, low-frequency-high-frequency (LH) subband coefficients, high-frequency-low-frequency (HL) subband coefficients, and high-frequency-high-frequency (HH) subband coefficients. LL subband coefficients can represent approximate components obtained by low-pass filtering in both row and column directions; LH subband coefficients can represent components obtained by low-pass filtering in the row direction and high-pass filtering in the column direction, reflecting the image's detail features in the horizontal direction; HL subband coefficients can represent components obtained by high-pass filtering in the row direction and low-pass filtering in the column direction, reflecting the image's detail features in the vertical direction; and HH subband coefficients can represent components obtained by high-pass filtering in both row and column directions, reflecting the image's detail features in the diagonal direction.

[0051] In step S140, the GPU performs entropy encoding on the subband coefficients to obtain encoded data.

[0052] Entropy coding can be described as a process of symbol analysis, bit generation, and compression coding of coefficient values ​​based on the numerical distribution of subband coefficients, in order to generate coded data that can express image information with lower redundancy. The coded data can represent the compressed output data obtained through entropy coding to express image content. For example, the coded data can be a compressed bit sequence generated by code blocks, or it can be the image compression result formed by sequentially combining multiple compressed bit sequences.

[0053] In addition, in other embodiments of this disclosure, after obtaining the encoded data, the encoded data can also be encapsulated in the CPU. Specifically, the encoded data can be transferred from the GPU back to the CPU; and in the CPU, the encoded data is encapsulated based on the target output format.

[0054] The target output format can represent a predetermined file format used to store encoded data. It determines the arrangement of encoded data within the file, header information, and necessary descriptive fields based on the target application scenario. For example, the target output format can be a JPH (JPEG 2000 High-Throughput) file format conforming to the High-Throughput JPEG 2000 standard, or a JP2 (JPEG 2000) file format conforming to the JPEG 2000 Part 1 standard, or other image compression file formats selected according to actual application requirements. Encapsulation can represent the process of organizing encoded data into a data structure that conforms to the file format requirements according to the target output format. In this embodiment, encapsulating the encoded data in the CPU leverages the CPU's instruction execution advantages in file format processing, avoiding the encapsulation task being configured on a GPU unsuitable for structured data processing. This avoids consuming GPU parallel computing resources and improves the overall execution efficiency of the encoding process.

[0055] The contents of steps S110 to S140 will be described in detail below.

[0056] In some embodiments, parsing the image to be encoded during wavelet transform coding to obtain raw pixel data specifically includes the following technical steps: In the CPU, the image bitstream corresponding to the image data to be encoded is read and loaded into a CPU-accessible buffer. A format parsing operation is performed on the loaded image bitstream to obtain parsed fields; these parsed fields may include size information, pixel component arrangement, color space mode, and related encoding parameters. A field-by-field interpretation operation is performed on the pixel data regions in the image bitstream according to the parsed fields to extract pixel data segments corresponding to the image to be encoded. According to the parsed size information and pixel component arrangement, the pixel data segments are rearranged into a continuous pixel data sequence to form the raw pixel data without preprocessing. Furthermore, before encoding processing, the CPU can be used to initialize the GPU, ensuring that the GPU's thread execution environment, device memory space, and task scheduling channel are in a usable state acceptable for subsequent encoding tasks.

[0057] During the parsing process, the CPU can sequentially read the image bitstream according to its predetermined structure and perform item-by-item recognition and interpretation operations on each parsing field based on the format specifications recorded in the bitstream header field. For example, the CPU can determine the number of rows and columns based on the size field in the bitstream and determine the storage order of pixels in the bitstream based on the pixel component arrangement field. Furthermore, in this embodiment, the control / IO-intensive steps of the encoding process, such as GPU initialization, bitstream reading, and data parsing, are completed by the CPU. This isolates the steps involving format parsing and control decisions in the encoding process from the parallel computing steps of the GPU, thereby avoiding the occupation of GPU parallel computing resources and improving encoding efficiency.

[0058] In some embodiments, transmitting raw pixel data to the GPU includes the following technical steps: allocating corresponding video memory in the GPU according to the size of the raw pixel data; and transmitting the raw pixel data to the video memory via asynchronous data transmission.

[0059] In this context, video memory (VRAM) refers to the accessible storage area within the GPU, used to store data involved in GPU computation, including raw pixel data, standard pixel data, subband coefficients, encoded data, and other intermediate data generated during thread execution. Asynchronous data transfer refers to a data transfer method between the CPU and GPU that utilizes a transfer channel independent of the computation path. This method allows data transfer to begin without waiting for the current computation process to finish and can be executed in parallel with the GPU's internal computation. In this embodiment, by allocating VRAM on demand according to the size of the raw pixel data and using asynchronous data transfer to write the raw pixel data into VRAM, data transfer and GPU computation can be carried out in parallel, thereby reducing processing latency caused by transmission waiting.

[0060] Specifically, the GPU allocates corresponding video memory based on the size of the original pixel data. The video memory size can be determined by the following formula: Video Memory Size = Image Width × Image Height × Number of Pixel Components × Pixel Bit Depth / 8. Here, image width represents the number of pixels in the horizontal direction of the original pixel data; image height represents the number of pixels in the vertical direction; number of pixel components represents the number of components contained in each pixel, for example, three pixel components for RGB format; and pixel bit depth represents the byte precision of each pixel component, such as 8-bit, 16-bit, or 32-bit.

[0061] In some embodiments, when preprocessing raw pixel data, multiple thread blocks required for the preprocessing operation can be determined first, and then parallel preprocessing operations can be performed based on these multiple thread blocks. (See reference) Figure 2 As shown, multiple threads are used to perform preprocessing operations on the raw pixel data in parallel, specifically including the following technical steps: Step S210: Determine the number of thread blocks required for the preprocessing operation based on the image width and the number of threads in a single thread block in the row direction of the GPU.

[0062] The image width represents the number of pixels in the horizontal direction of the pixel matrix that makes up the image to be encoded, and is used to characterize the size attribute of the original pixel data in the column direction. A thread block represents a set of threads in the GPU that are combined according to a preset size. Each thread can share local storage resources and execute specified data processing instructions under a unified scheduling unit.

[0063] In the specific implementation process, the number of thread blocks required for the preprocessing operation is determined based on the image width and the number of threads in a single thread block in the row direction of the GPU. This can include the following technical steps: determining the total number of pixel columns to be preprocessed based on the image width; determining the number of covered columns of a single thread block based on the number of threads in a single thread block in the row direction of the GPU; and determining the number of thread blocks required for the preprocessing operation based on the calculation result of dividing the total number of pixel columns by the number of covered columns.

[0064] The total number of pixel columns represents the number of pixel columns contained in the original pixel data to be preprocessed along the image width. The number of covered columns represents the number of pixel columns that a single thread block can process simultaneously along the row direction, characterizing the range of column tasks that a single thread block can cover in a single parallel execution. Furthermore, when determining the number of thread blocks required for preprocessing, the total number of pixel columns can be divided by the number of covered columns to obtain an initial calculated value for the number of thread blocks. If the initial calculated value is not an integer, it is rounded up to obtain the number of thread blocks; if the initial calculated value is an integer, it is used as the number of thread blocks to determine the number of thread blocks required for preprocessing. In this step, determining the number of thread blocks required for preprocessing based on the image width and the number of threads in a single thread block along the row direction ensures a correspondence between the number of thread blocks and the image width, thereby avoiding idle pixel column processing tasks in the threads and achieving higher parallel throughput.

[0065] Step S220: Determine multiple thread blocks corresponding to the preprocessing operation based on the number of thread blocks. Specifically, configure multiple thread blocks in the GPU according to the number of thread blocks, and assign a corresponding pixel column processing range to each thread block, so that the multiple thread blocks cover the total number of pixel columns.

[0066] Step S230: Call the preprocessing kernel function required for the preprocessing operation, and execute the preprocessing kernel function in parallel on threads in multiple thread blocks to perform preprocessing operations on the raw pixel data.

[0067] The preprocessing kernel function can represent a program unit used to execute pixel data processing instructions in parallel across multiple threads. It is configured to perform preprocessing operations such as storage format conversion, DC component shifting, or color space transformation on the input data to generate a data structure that meets the input requirements of the subsequent forward wavelet transform.

[0068] Furthermore, in the actual encoding process, thread blocks can be configured as two-dimensional thread blocks; a two-dimensional thread block includes threads in the row direction and threads in the column direction; the number of threads in the column direction of a two-dimensional thread block is equal to the number of pixel components in the original pixel data. The preprocessing operation then includes: executing preprocessing kernel functions in parallel on the threads in the row direction and the threads in the column direction of multiple two-dimensional thread blocks; wherein, one thread in the row direction processes one pixel column in the original pixel data, and one thread in the column direction processes one pixel component within that pixel column.

[0069] In this context, a 2D thread block can represent a collection of threads organized in a 2D coordinate system within the GPU. The thread indices increment independently in both the row and column directions, and are used to process image data in a 2D parallel manner. Row-oriented threads can represent thread units arranged along the row index within the 2D thread block, allocated to process multiple columns of pixels arranged along the column direction in the original pixel data. Column-oriented threads can represent thread units arranged along the column index within the 2D thread block, configured to process multiple pixel components arranged in component order within a single pixel column. Pixel components can represent the basic numerical units constituting a single pixel, used to describe the pixel's values ​​in different color channels or luminance / chrominance components. For example, pixel components can be R (Red), G (Green), and B (Blue), or Y (Luma), Cb (Blue-difference Chroma), and Cr (Red-difference Chroma), or even a single luminance component in a grayscale image. With the above settings, the processing tasks of pixel columns can be executed in parallel on the row-direction threads, and the pixel components within the same pixel column can be executed in parallel on the column-direction threads. This allows the data accessed by each thread to be arranged continuously in memory, avoiding access conflicts caused by cross-step access, thereby improving the parallel efficiency of thread execution and the utilization of video memory bandwidth.

[0070] Furthermore, multiple thread blocks performing preprocessing operations can form a grid for parallel scheduling. During encoding, the number of thread blocks in the grid can be dynamically adjusted based on the image width corresponding to the original pixel data, ensuring that the number of thread blocks in the row direction matches the number of pixel columns to be processed. This allows thread scheduling resources to flexibly expand or shrink as the image width changes, avoiding processing omissions due to insufficient thread blocks or wasted computational resources due to excessive thread blocks.

[0071] In some embodiments, the original pixel data can be in an interleaved storage format, and the component representation of the original pixel data needs to undergo color space transformation; the interleaved storage format means that the pixel components in the original pixel data are arranged alternately according to pixel order. Then, the preprocessing kernel function is executed in parallel on threads within multiple thread blocks, including: if the preprocessing kernel function is a first kernel function, then the first kernel function is executed in parallel on threads within multiple thread blocks; wherein, the first kernel function is a kernel function capable of completing storage format conversion, DC component shift processing, and color space transformation in a single kernel function execution. That is, when performing storage format conversion, DC component shift processing, and color space transformation, the predetermined grid and thread block configuration of the preprocessing stage can be used to maintain the consistency of the thread scheduling method, thereby avoiding additional scheduling overhead caused by switching thread organization structures.

[0072] Storage format conversion can represent a data rearrangement operation performed on the storage arrangement of input pixel data, used to rearrange pixel components arranged in an alternating pixel manner into a continuous data sequence stored in a component-independent manner. DC component shift processing can represent an operation that adjusts the numerical range of the original pixel data, by adding or subtracting a predetermined offset value to the overall value of the pixel components, converting the pixel component values ​​from an unsigned range to a zero-centered signed range. Color space transformation can represent a linear or non-linear combination operation on multiple color components of a pixel based on a predetermined color transformation matrix or component mapping rules, to convert the color representation of the input image from a first color space to a second color space; for example, the RGB color space can be converted to the YCbCr color space. Furthermore, for lossless encoding, color space transformation can be performed using reversible color transformation (RCT); for lossy encoding, color space transformation can be performed using irreversible color transformation (ICT).

[0073] In this embodiment, by executing the first kernel function in parallel on threads in multiple thread blocks, storage format adjustment, numerical range shifting, and color component transformation can be completed continuously within the same thread scheduling cycle. This avoids the round trips to video memory caused by repeatedly loading and writing back pixel data between multiple kernel functions, improves the continuity of the data processing path, and reduces additional scheduling overhead.

[0074] Furthermore, if the original pixel data can be in an interleaved storage format and the component representation of the original pixel data needs to undergo color space transformation, then the preprocessing kernel function is executed in parallel on threads within multiple thread blocks to perform preprocessing operations on the original pixel data. Specifically, if the preprocessing kernel function includes a second kernel function that can complete the storage format conversion and DC component shift processing in a single kernel function execution, and a third kernel function for color space transformation, then the second kernel function is executed in parallel on threads within multiple thread blocks to perform storage format conversion and DC component shift processing on the original pixel data; after the storage format conversion and DC component shift processing are completed, the third kernel function is executed in parallel on threads within multiple thread blocks to perform color space transformation on the original pixel data.

[0075] In this embodiment, by first executing the second kernel function for storage format conversion and DC component shifting, and then executing the third kernel function for color space transformation, the second kernel function can first complete the storage format conversion and DC component shifting of all raw pixel data in parallel under the same thread block configuration. Then, the third kernel function performs color space transformation in parallel while keeping the thread mapping relationship between the column direction and the component direction unchanged. This keeps the data allocation between the two stages of processing continuous and consistent, reduces the overhead of thread remapping and data rearrangement in the intermediate process, and thus improves the overall parallel efficiency and memory access throughput of the preprocessing stage.

[0076] In some embodiments, if the original pixel data is in an independent storage format and the component representation of the original pixel data needs to undergo color space transformation; the independent storage format means that each pixel component in the original pixel data is stored in partitions. The preprocessing kernel function is executed in parallel on threads within multiple thread blocks, specifically including the following steps: if the preprocessing kernel function is a fourth kernel function, then the fourth kernel function is executed in parallel on threads within multiple thread blocks; wherein, the fourth kernel function is a kernel function capable of completing DC component translation and color space transformation in a single kernel function execution.

[0077] Specifically, if the original pixel data is in an independent storage format, there is no need to perform a storage format conversion step in the preprocessing stage. The DC component shifting and color space transformation can be completed in one kernel function execution, improving the continuity of the data processing path and reducing additional scheduling overhead.

[0078] Furthermore, if the original pixel data is stored in an independent format and the component representation of the original pixel data does not require color space transformation—for example, single-channel grayscale images, image data already stored independently using YCbCr components, single-channel data containing only depth or near-infrared channels, or multi-component data for which color space adaptation has been completed by the front-end processing unit—then there is no need to perform color space transformation processing related to color component mapping. In this case, the preprocessing operation can be completed by executing the kernel function corresponding to the DC component translation processing in parallel within multiple threads used for preprocessing operations.

[0079] In some embodiments, shared memory can be used to perform a forward wavelet transform on standard pixel data, referencing... Figure 3 As shown, a forward wavelet transform is performed on the standard pixel data to obtain the sub-band coefficients. The specific technical steps include the following: Step S310: Divide the standard pixel data into multiple sub-blocks.

[0080] Here, a sub-block can represent a smaller rectangular region in the spatial domain obtained by dividing the standard pixel data into smaller rectangular regions during the forward wavelet transform of the standard pixel data in order to reduce on-chip storage usage and improve parallelism. Specifically, the standard pixel data can be divided sequentially using a preset window size.

[0081] Step S320: Load the pixel data corresponding to the sub-block from the video memory to the shared memory.

[0082] In this context, shared memory refers to a high-speed storage resource on the GPU. Its access latency is significantly lower than that of video memory (VRAM), and it can be accessed by multiple threads within the same thread block. It is used for temporary storage and rapid data swapping after data loading. The pixel data corresponding to each sub-block can be represented as a pixel matrix formed by the pixel regions covered by each sub-block after dividing the standard pixel data into multiple sub-blocks. This matrix is ​​used for forward wavelet transform processing in shared memory. The reason for loading the pixel data corresponding to each sub-block from VRAM to shared memory is that while VRAM has a large capacity, it suffers from high access latency and limited bandwidth. Directly performing forward wavelet transform on VRAM would result in frequent data movement and low efficiency. Shared memory, located on-chip, offers fast access and can be efficiently shared by threads within the same thread block. Loading sub-blocks into shared memory first enables the reuse of local data, thereby reducing the number of global memory (VRAM) accesses, lowering latency and bandwidth pressure, and improving thread collaboration efficiency during sub-block processing, ultimately enhancing overall computational efficiency.

[0083] Step S330: Perform a forward wavelet transform on the pixel data corresponding to the sub-block in shared memory to obtain sub-band coefficients, and write the sub-band coefficients back from shared memory to video memory. In this step, completing the forward wavelet transform of the sub-block in shared memory and writing the generated sub-band coefficients back to video memory at once avoids repeated read and write operations on global memory during the phased improvement operations in the horizontal and vertical directions, thereby reducing the number of global memory accesses, improving memory access efficiency, and further improving the overall encoding efficiency.

[0084] In some embodiments, standard pixel data is divided into multiple sub-blocks, specifically including the following technical steps: determining the target size and critical support region corresponding to the sub-block based on a preset wavelet kernel, wherein the wavelet kernel is the filtering operator used when performing forward wavelet transform, and the critical support region represents the region located at the edge of the sub-block and used for boundary calculation in forward wavelet transform; dividing the standard pixel data based on the target size and critical support region to obtain multiple sub-blocks.

[0085] In this embodiment, the wavelet kernel represents the basic computational unit used in the forward wavelet transform. It can consist of a set of low-pass filter coefficients and a set of high-pass filter coefficients. The low-pass filter coefficients are used to extract the low-frequency approximation components of the input signal, while the high-pass filter coefficients are used to extract the high-frequency detail components of the input signal. In this embodiment, the wavelet kernel can be an integer wavelet kernel, such as a 5 / 3 wavelet kernel, or a floating-point wavelet kernel, such as a 9 / 7 wavelet kernel. The target size can represent the size parameter obtained based on the preset wavelet kernel when determining the size of the sub-block. This parameter is used to limit the number of consecutive image rows and columns covered by the sub-block. The filtering operator can represent the operator that performs filtering operations based on the wavelet kernel in the forward wavelet transform processing. It can consist of a low-pass filter operator and a high-pass filter operator. In this embodiment, the target size and critical support region are determined based on the preset wavelet kernel, and sub-blocks are divided accordingly. This ensures that each sub-block has a complete wavelet operation context, thereby reducing boundary processing overhead and improving the local computational efficiency and parallel execution performance of the forward wavelet transform.

[0086] In the specific implementation process, determining the target size and critical support region corresponding to the sub-block based on the wavelet kernel can be done through the following steps: First, determine the number of consecutive image rows and columns required for the sub-block based on the coefficient length of the filter operator. This number of consecutive image rows and columns ensures that the calculation of the center sample can obtain complete contextual support during forward wavelet transform processing. Then, determine the critical support region based on the length of the filter operator. This critical support region can represent the neighboring rows and columns located at the sub-block boundary but necessary for prediction and update operations, providing necessary reference data for samples at the boundary location.

[0087] For example, when the wavelet kernel is a 5 / 3 wavelet kernel, the target size for each sub-block can be set to 32×32, and the corresponding critical support region consists of two rows and two columns of data adjacent to the edge of the sub-block; when the wavelet kernel is a 9 / 7 wavelet kernel, the target size for each sub-block can be set to 64×64, and the corresponding critical support region consists of four rows and four columns of data adjacent to the edge of the sub-block. Of course, in other embodiments of this disclosure, other sub-block sizes and critical support regions can be selected according to the type of wavelet kernel.

[0088] In some embodiments, performing a forward wavelet transform on the pixel data corresponding to the sub-block in the shared memory specifically includes the following technical steps: performing boundary expansion processing on the pixel data corresponding to the sub-block in the shared memory to obtain the corresponding expanded sub-block; if the wavelet transform encoding is lossy encoding, calling the fifth kernel function, and using the fifth kernel function to perform forward wavelet transform and quantization processing on the expanded sub-block; wherein, the fifth kernel function is a kernel function used to combine and perform forward wavelet transform and quantization processing.

[0089] Boundary expansion can be defined as an operation performed on pixel data during forward wavelet transform processing to supplement the data at the boundaries of sub-blocks, ensuring the integrity of the neighborhood samples required for the lifting operation. This operation can generate virtual data outside the boundaries through symmetric copying, mirror copying, or other rules, so that the sub-block edges have an equivalent support range as the internal data during computation. An expanded sub-block can be defined as the data set composed of the original sub-block data and the expanded boundary data after boundary expansion processing. In this embodiment, performing boundary expansion processing on the pixel data corresponding to each sub-block in shared memory can supplement the required neighboring data at the edge positions, ensuring the computational integrity of the forward wavelet transform and reducing global memory dependencies. Furthermore, the forward wavelet transform can include horizontal and vertical lifting operations. In this embodiment, both horizontal and vertical lifting operations are performed in shared memory, confining related read and write operations within shared memory and avoiding repeated access to video memory between two lifting operations, thereby reducing the number of global memory accesses and improving overall computational efficiency.

[0090] Furthermore, lossy coding can represent a coding method that quantizes coefficients after a forward wavelet transform and allows for the introduction of distortion, achieving higher compression efficiency by reducing coefficient precision. Quantization processing can represent a type of numerical transformation process that reduces the magnitude of coefficients based on the coefficients obtained from the wavelet transform according to a preset quantization step size. In this embodiment, the forward wavelet transform and quantization processing are combined in a single kernel function execution, avoiding the round-trip memory read / write caused by two independent kernel function schedulings, thereby reducing global memory access and improving the overall throughput and processing efficiency of the encoding stage.

[0091] In some embodiments, entropy coding processing of subband coefficients specifically includes the following technical steps: dividing the subband coefficients based on a preset code block size to obtain multiple code blocks; calling the entropy coding kernel function required for entropy coding processing, and executing the entropy coding kernel function in parallel on multiple threads to perform entropy coding processing on multiple code blocks respectively; wherein, the entropy coding kernel function is related to the wavelet transform coding method.

[0092] In this context, the code block size can represent a pre-defined two-dimensional segmentation scale parameter within the sub-band region after wavelet decomposition, used to limit the number of elements in each code block in both the row and column directions. A code block can represent a local coefficient region divided from the sub-band coefficients based on the code block size; it serves as the basic input unit for entropy coding and can be processed independently across different threads. The entropy coding kernel function can represent a schedulable execution unit on the GPU used to perform entropy coding. It contains coding logic such as symbol scanning, state updating, and bit stream generation, matching the target coding method, and can run concurrently on multiple threads. Entropy coding processing can represent the encoding process of symbol modeling and bit output for the coefficient data in the code block. It converts the input coefficients into a bit sequence that meets the target output format requirements according to the coding method used in wavelet transform coding. In this embodiment, by executing the entropy coding kernel function in parallel on multiple threads, the coding tasks of each code block are performed synchronously, avoiding the waiting time caused by serial processing. This improves the processing throughput and real-time performance of the entropy coding stage, especially significantly improving coding efficiency for large-size or high-resolution images.

[0093] Furthermore, in the entropy coding process, the number of thread blocks in the grid used for entropy coding can be adjusted according to the number of code blocks. Specifically, the number of thread blocks in the grid can be dynamically determined based on the ratio between the total number of code blocks to be processed and the number of threads contained in a single thread block, so that each thread can process a code block or a specific data segment within a code block, thus forming a parallel scheduling structure that can scale with the number of code blocks. In addition, the entropy coding kernel function is related to the wavelet transform coding method. If the HTJ2K standard coding method is used, the entropy coding kernel function can be a kernel function used to perform high-throughput block coding processing based on the fast block coding path, which can complete symbol extraction, state update, and bit-by-bit output in a thread-level parallel manner. If the J2K standard coding method is used, the entropy coding kernel function can be a kernel function used to perform the traditional EBCOT (Embedded Block Coding with OptimalTruncation) coding process, which can complete symbol modeling, context selection, and arithmetic coding.

[0094] Figure 4A schematic diagram illustrating another image encoding method of this disclosure is shown. (Referring to...) Figure 4 As shown, the image encoding method may include the following steps: Step 401, GPU initialization. Specifically, the CPU is used to initialize the GPU, ensuring that the GPU's thread execution environment, device memory space, and task scheduling channel are in a usable state that can accept subsequent coding tasks.

[0095] Step 402, bitstream reading and data parsing. Specifically, in the CPU, the image bitstream corresponding to the image data to be encoded is read and loaded into a CPU-accessible buffer. A parsing operation is then performed on the loaded image bitstream to obtain the raw pixel data without preprocessing.

[0096] Step 403, Memory Allocation and Data Transfer. Specifically, based on the size of the original pixel data, corresponding video memory is allocated in the GPU, and the original pixel data is written to the video memory using asynchronous data transfer.

[0097] Step 404, preprocessing operation. Specifically, the GPU uses multiple threads to process the corresponding pixel columns in the original pixel data along the row direction, and performs parallel execution based on a grid composed of thread blocks to complete at least one of storage format conversion, DC component shifting, and color space transformation, thereby generating standard pixel data that can meet the requirements of wavelet transform.

[0098] Step 405, Forward Wavelet Transform. Specifically, in the GPU, the standard pixel data is divided into multiple sub-blocks based on a preset wavelet kernel, and the corresponding sub-blocks are loaded from video memory to shared memory; a forward wavelet transform is performed on each sub-block in shared memory to obtain sub-band coefficients.

[0099] Step 406, quantization processing. Specifically, when lossy coding is used, the subband coefficients obtained by the forward wavelet transform need to be quantized. If lossless coding is used, this processing step is skipped.

[0100] Step 407, entropy encoding processing. Specifically, the sub-band coefficients are divided based on a preset code block size to obtain multiple code blocks; the GPU calls the entropy encoding kernel function corresponding to the encoding method, and executes it in parallel on multiple threads to complete the entropy encoding processing of each code block, thereby obtaining encoded data.

[0101] Step 408: Encapsulation based on the target output format. Specifically, the encoded data is transferred from the GPU back to the CPU, and the CPU encapsulates the encoded data according to the file structure requirements of the target output format to generate an encoded image file that conforms to the target output format.

[0102] It should be noted that although the steps of the method in this disclosure are described in a specific order in the accompanying drawings, this does not require or imply that the steps must be performed in that specific order, or that all the steps shown must be performed to achieve the desired result. Additional or alternative steps may be omitted, multiple steps may be combined into one step, and / or a step may be broken down into multiple steps.

[0103] Furthermore, in this exemplary embodiment, an image encoding apparatus is also provided. (Refer to...) Figure 5 As shown, the image encoding device 500 includes an image parsing module 510, a preprocessing module 520, a wavelet transform module 530, and an entropy encoding module 540. Specifically: the image parsing module 510 parses the image to be encoded during the wavelet transform encoding process to obtain raw pixel data and transmits the raw pixel data to the preprocessing module; the preprocessing module 520 uses multiple threads to perform parallel preprocessing operations on the raw pixel data to obtain standard pixel data that meets the wavelet transform requirements; wherein one thread processes one pixel column from the raw pixel data; the wavelet transform module 530 performs a forward wavelet transform on the standard pixel data to obtain subband coefficients; and the entropy encoding module 540 performs entropy encoding processing on the subband coefficients to obtain encoded data.

[0104] The specific details of each module of the above image encoding device have been described in detail in the corresponding image encoding methods, so they will not be repeated here.

[0105] It should be noted that although several modules or units of the image encoding apparatus have been mentioned in the detailed description above, this division is not mandatory. In fact, according to embodiments of this disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0106] Furthermore, in an exemplary embodiment of this disclosure, an electronic device capable of implementing the above-described image encoding method is also provided.

[0107] Those skilled in the art will understand that various aspects of this disclosure can be implemented as a system, method, or program product. Therefore, various aspects of this disclosure can be embodied in the following forms: a completely hardware embodiment, a completely software embodiment (including firmware, microcode, etc.), or an embodiment combining hardware and software aspects, collectively referred to herein as a "circuit," "module," or "system."

[0108] The following reference Figure 6 To describe an image encoding device 600 according to such an embodiment of the present disclosure. Figure 6 The image encoding device 600 shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments disclosed herein.

[0109] like Figure 6 As shown, the image encoding device 600 is presented in the form of a general-purpose computing device. The components of the image encoding device 600 may include, but are not limited to: at least one processor 610, at least one memory 620, a bus 630 connecting different system components (including memory 620 and processor 610), and a display unit 640, wherein the processor includes a CPU and a GPU.

[0110] The storage unit stores program code that can be executed by the processor 610, causing the processor 610 to perform the steps described in the "Exemplary Methods" section of this specification according to various exemplary embodiments of this disclosure. For example, the processor 610 can perform actions such as... Figure 1 The steps are shown in the figure.

[0111] The memory 620 may include a readable medium in the form of volatile memory cells, such as random access memory (RAM) 621 and / or cache 622, and may further include read-only memory (ROM) 623.

[0112] The memory 620 may also include a program / utility 624 having a set (at least one) of program modules 625, including but not limited to: an operating system, one or more application programs, other program modules, and program data, each or some combination of these examples may include an implementation of a network environment.

[0113] Bus 630 can represent one or more of several types of bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the various bus structures.

[0114] The image encoding device 600 can also communicate with one or more external devices 670 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), one or more devices that enable users to interact with the image encoding device 600, and / or any device that enables the image encoding device 600 to communicate with one or more other computing devices (e.g., routers, modems, etc.). This communication can be performed via input / output (I / O) interface 650. Furthermore, the image encoding device 600 can also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via network adapter 660. As shown, network adapter 660 communicates with other modules of the image encoding device 600 via bus 630. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with the image encoding device 600, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0115] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, terminal device, or network device, etc.) to execute the methods according to the embodiments of this disclosure.

[0116] In exemplary embodiments of this disclosure, a computer-readable storage medium is also provided, on which a program product capable of implementing the methods described above is stored. In some possible embodiments, various aspects of this disclosure may also be implemented as a program product including program code that, when the program product is run on a terminal device, causes the terminal device to perform the steps described in the "Exemplary Methods" section of this specification according to various exemplary embodiments of this disclosure.

[0117] refer to Figure 7 As shown, a program product 700 for implementing the above-described image encoding method according to an embodiment of the present disclosure is described. This product may employ a portable compact disc read-only memory (CD-ROM) and include program code, and may run on a terminal device, such as a personal computer. However, the program product of the present disclosure is not limited thereto. In this document, a readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, apparatus, or device.

[0118] The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0119] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium, capable of sending, propagating, or transmitting programs for use by or in conjunction with an instruction execution system, apparatus, or device.

[0120] The program code contained on the readable medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0121] Program code for performing the operations of this disclosure can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0122] Furthermore, the above figures are merely illustrative of the processes included in the method according to exemplary embodiments of this disclosure and are not intended to be limiting. It is readily understood that the processes shown in the above figures do not indicate or limit the temporal order of these processes. Additionally, it is readily understood that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0123] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, touch terminal, or network device, etc.) to execute the methods according to the embodiments of this disclosure.

[0124] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and embodiments are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the claims.

[0125] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. An image encoding method, characterized in that, Applied to an image coding system, the image coding system including a central processing unit (CPU) and a graphics processing unit (GPU), the method includes: The CPU parses the image to be encoded during the wavelet transform encoding process to obtain the original pixel data, and transmits the original pixel data to the GPU; The GPU uses multiple threads to perform preprocessing operations on the original pixel data in parallel to obtain standard pixel data that meets the requirements of wavelet transform; wherein, one thread processes one column of pixels in the original pixel data. The GPU performs a forward wavelet transform on the standard pixel data to obtain sub-band coefficients; The GPU performs entropy encoding on the subband coefficients to obtain encoded data.

2. The image encoding method according to claim 1, characterized in that, The preprocessing operation on the raw pixel data using multiple threads in parallel includes: The number of thread blocks required for the preprocessing operation is determined based on the image width and the number of threads in a single thread block in the row direction of the GPU. The number of thread blocks is used to determine the multiple thread blocks corresponding to the preprocessing operation; The preprocessing kernel function required for the preprocessing operation is invoked, and the preprocessing kernel function is executed in parallel on the threads of the plurality of thread blocks to perform preprocessing operations on the raw pixel data.

3. The image encoding method according to claim 2, characterized in that, Determining the number of thread blocks required for the preprocessing operation based on the image width and the number of threads in a single thread block in the row direction of the GPU includes: The total number of pixel columns to be preprocessed is determined based on the image width; The number of columns covered by a single thread block is determined based on the number of threads in the row direction of a single thread block in the GPU; The number of thread blocks required for the preprocessing operation is determined based on the result of dividing the total number of pixel columns by the number of covered columns.

4. The image encoding method according to claim 2, characterized in that, The thread block is a two-dimensional thread block; the two-dimensional thread block includes threads in the row direction and threads in the column direction; the number of threads in the column direction of the two-dimensional thread block is equal to the number of pixel components of the original pixel data; The parallel execution of the preprocessing kernel function on threads within the plurality of thread blocks includes: The preprocessing kernel function is executed in parallel on the row-direction threads and column-direction threads in the multiple two-dimensional thread blocks; In this process, one thread in the row direction processes a column of pixels in the original pixel data, and one thread in the column direction processes a pixel component within the column of pixels.

5. The image encoding method according to claim 2, characterized in that, The original pixel data is in an interleaved storage format and the component representation of the original pixel data needs to be transformed in color space; the interleaved storage format means that the pixel components in the original pixel data are arranged alternately in pixel order. The parallel execution of the preprocessing kernel function on threads within the multiple thread blocks includes: If the preprocessing kernel function is a first kernel function, then the first kernel function is executed in parallel on the threads in the plurality of thread blocks; wherein, the first kernel function is a kernel function that can complete storage format conversion, DC component shift processing and color space transformation in a single kernel function execution.

6. The image encoding method according to claim 5, characterized in that, The method of executing the preprocessing kernel function in parallel on threads within the plurality of thread blocks to perform preprocessing operations on the raw pixel data further includes: If the preprocessing kernel function includes a second kernel function that can complete storage format conversion and DC component shift processing in a single kernel function execution, and a third kernel function for color space transformation, then the second kernel function is executed in parallel on the threads in the plurality of thread blocks to perform the storage format conversion and DC component shift processing on the original pixel data; After the storage format conversion and DC component shifting are completed, the third kernel function is executed in parallel on the threads of the multiple thread blocks to perform the color space transformation on the original pixel data.

7. The image encoding method according to claim 2, characterized in that, The original pixel data is stored in an independent format and the component representation of the original pixel data needs to be transformed by color space. The independent storage format indicates that each pixel component in the original pixel data is stored in partitions; The parallel execution of the preprocessing kernel function on threads within the multiple thread blocks includes: If the preprocessing kernel function is a fourth kernel function, then the fourth kernel function is executed in parallel on the threads in the plurality of thread blocks; wherein, the fourth kernel function is a kernel function that can complete DC component translation and color space transformation in a single kernel function execution.

8. The image encoding method according to claim 1, characterized in that, The transmission of the raw pixel data to the GPU includes: The corresponding video memory is allocated in the GPU according to the size of the original pixel data; The original pixel data is transferred to the video memory using an asynchronous data transfer method.

9. The image encoding method according to claim 8, characterized in that, The step of performing a forward wavelet transform on the standard pixel data to obtain sub-band coefficients includes: The standard pixel data is divided into multiple sub-blocks; Load the pixel data corresponding to the sub-block from the video memory to the shared memory; The forward wavelet transform is performed on the pixel data corresponding to the sub-block in the shared memory to obtain the sub-band coefficients, and the sub-band coefficients are written back from the shared memory to the video memory.

10. The image encoding method according to claim 9, characterized in that, The step of dividing the standard pixel data into multiple sub-blocks includes: The target size and critical support region corresponding to the sub-block are determined based on a preset wavelet kernel, wherein the wavelet kernel is the filtering operator used when performing the forward wavelet transform, and the critical support region represents the region located at the edge of the sub-block and used for boundary calculation in the forward wavelet transform; The standard pixel data is divided based on the target size and the critical support region to obtain the multiple sub-blocks.

11. The image encoding method according to claim 9, characterized in that, The step of performing the forward wavelet transform on the pixel data corresponding to the sub-block in the shared memory includes: Perform boundary expansion processing on the pixel data corresponding to the sub-block in the shared memory to obtain the corresponding expanded sub-block; If the wavelet transform encoding is lossy, the fifth kernel function is called, and the forward wavelet transform and quantization processing are performed on the extended sub-block using the fifth kernel function; wherein, the fifth kernel function is the kernel function that combines and executes the forward wavelet transform and quantization processing.

12. The image encoding method according to claim 1, characterized in that, The entropy encoding process for the sub-band coefficients includes: The sub-band coefficients are divided based on a preset code block size to obtain multiple code blocks; The entropy coding kernel function required for the entropy coding process is invoked and executed in parallel on multiple threads to perform entropy coding processing on the multiple code blocks respectively; wherein, the entropy coding kernel function is related to the encoding method of the wavelet transform coding.

13. The image encoding method according to claim 1, characterized in that, The method further includes: The encoded data is transmitted from the GPU back to the CPU; The CPU encapsulates the encoded data based on the target output format.

14. An image coding apparatus, applied to wavelet transform coding, characterized in that, The device includes: The image parsing module is used to parse the image to be encoded during the wavelet transform coding process to obtain the original pixel data, and transmit the original pixel data to the preprocessing module; The preprocessing module is used to perform preprocessing operations on the original pixel data in parallel using multiple threads to obtain standard pixel data that meets the requirements of wavelet transform; wherein, one thread processes one column of pixels in the original pixel data; The wavelet transform module is used to perform a forward wavelet transform on the standard pixel data to obtain sub-band coefficients; The entropy coding module is used to perform entropy coding processing on the sub-band coefficients to obtain coded data.

15. An image encoding device, characterized in that, include: Processors, including CPUs and GPUs; as well as Memory for storing the executable instructions of the processor; The processor is configured to execute the image encoding method according to any one of claims 1-13 by executing the executable instructions.

16. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the image encoding method according to any one of claims 1-13.

Images