Picture decoding method and device, equipment, storage medium and product

Abstract

The invention provides a picture decoding method and device, equipment, a storage medium and a product, and belongs to the technical field of picture processing. The method comprises: when a picture is decoded, determining a decoding table of the picture, the decoding table being used for storing a corresponding relationship between a picture data stream of the picture and a coefficient value, the picture data stream being a data stream obtained by encoding the picture, and the coefficient value being quantization of an occurrence probability of image data; reading a picture data stream of the picture; searching a first coefficient value corresponding to the picture data stream from a decoding table of the picture based on the picture data stream; and based on the first coefficient value, decoding to obtain the picture. In the embodiment of the invention, the process of decoding the picture data stream to obtain the coefficient value is realized in a table look-up manner, so that the picture decoding efficiency can be improved, and the proportion of CPU resources in the picture decoding process can be reduced.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an image decoding method, apparatus, device, storage medium, and product. Background Technology

[0002] The Joint Photographic Experts Group (JPEG) is one of the most widely used image storage formats. When storing images, electronic devices encode the images into JPEG format image data (also known as JPEG images) and then store the JPEG images, thereby reducing the image storage space. When users need to view or preview the images, the electronic devices decode the JPEG images and then display or preview the decoded images. Summary of the Invention

[0003] This application provides an image decoding method, apparatus, device, storage medium, and product, which not only improves image decoding efficiency but also reduces the CPU resource consumption during the image decoding process. The technical solution is as follows:

[0004] On the one hand, an image decoding method is provided, the method comprising:

[0005] When decoding an image, a decoding table for the image is determined. The decoding table is used to store the correspondence between the image data stream and the coefficient values. The image data stream is the data stream obtained by encoding the image, and the coefficient values ​​are the quantification of the probability of the image data appearing.

[0006] Read the image data stream of the image;

[0007] Based on the image data stream, the first coefficient value corresponding to the image data stream is found from the image decoding table;

[0008] The image is obtained by decoding based on the first coefficient value.

[0009] On the other hand, an image decoding apparatus is provided, the apparatus comprising:

[0010] The first determining module is used to determine the decoding table of the image when decoding the image. The decoding table is used to store the correspondence between the image data stream and the coefficient values ​​of the image. The image data stream is the data stream obtained by encoding the image. The coefficient values ​​are the quantification of the probability of the image data appearing.

[0011] A reading module is used to read the image data stream of the image;

[0012] The lookup module is used to look up a first coefficient value corresponding to the image data stream from the image decoding table based on the image data stream;

[0013] The first decoding module is used to decode the image based on the first coefficient value.

[0014] On the other hand, an electronic device is provided, comprising a processor and a memory, wherein the memory stores at least one line of program code, which is loaded and executed by the processor to implement the image decoding method described above.

[0015] On the other hand, a computer-readable storage medium is provided, wherein at least one piece of program code is stored in the storage medium, the at least one piece of program code being loaded and executed by a processor to implement the image decoding method described above.

[0016] On the other hand, a computer program product is provided, which stores at least one piece of program code for execution by a processor to implement the image decoding method described above.

[0017] In this embodiment, a decoding table is generated in advance to store the correspondence between the image data stream and the coefficient values. When decoding a JPEG image, the first coefficient value corresponding to the image data stream is obtained directly from the decoding table based on the read image data stream. This transforms the step of obtaining the first coefficient value based on the image data stream into a table lookup method, which not only improves the image decoding efficiency but also reduces the proportion of CPU resources consumed by the image decoding process. Attached Figure Description

[0018] Figure 1 A schematic diagram illustrating the implementation environment of an image decoding method according to an exemplary embodiment of this application is shown;

[0019] Figure 2 A flowchart illustrating an exemplary embodiment of the image decoding method of this application is shown;

[0020] Figure 3 A flowchart illustrating an exemplary embodiment of the image decoding method of this application is shown;

[0021] Figure 4 A block diagram illustrating an image decoding apparatus according to an exemplary embodiment of this application is shown;

[0022] Figure 5 A block diagram of a terminal illustrated in an exemplary embodiment of this application is shown;

[0023] Figure 6A block diagram of a server illustrated in an exemplary embodiment of this application is shown. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0025] In this article, "multiple" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0026] It should be noted that all information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.), and signals involved in this application have been authorized by the user or fully authorized by all parties, and the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the relevant countries and regions. For example, the decoding table and image data stream involved in this application were obtained with full authorization.

[0027] Please refer to Figure 1 This diagram illustrates an implementation environment for an image decoding method according to an exemplary embodiment of this application. (Reference) Figure 1 The implementation environment includes an electronic device, which is a device with image encoding and decoding capabilities. In some embodiments, the electronic device can be either a terminal or a server. When the electronic device is a terminal, it may include, but is not limited to, mobile phones, tablets, laptops, and in-vehicle terminals. When the electronic device is a server, it may be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms, but is not limited to these.

[0028] In some embodiments, after an electronic device takes a picture, it needs to store the photo in a photo album. To reduce the storage space required, the electronic device encodes the image and stores the encoded image data. For example, the electronic device encodes the image into JPEG format image data (also called a JPEG image) and then stores the JPEG image. When a user views or previews images in the photo album, the electronic device decodes the image data to obtain the image. For example, the electronic device decodes the JPEG image back into the original image and then displays or previews the decoded image.

[0029] In some embodiments, the process of an electronic device encoding an image into a JPEG image includes color space conversion (converting a color image to a grayscale image), discrete cosine transform (converting the grayscale image from the pixel domain to the frequency domain, achieving compression by removing unimportant high-frequency components), quantization (representing the grayscale image in the frequency domain as integers proportional to its amplitude, obtaining coefficient values), and entropy encoding (converting the quantized coefficient values ​​into binary data). Correspondingly, the process of an electronic device decoding a JPEG image into the original image may include: entropy decoding (converting binary data into coefficient values), dequantization (converting the coefficient values ​​into a grayscale image in the frequency domain), inverse discrete cosine transform (converting the compressed grayscale image in the frequency domain back to the pixel domain), and color space conversion (converting the grayscale image into a color image). Entropy decoding includes Huffman decoding and arithmetic decoding; currently, Huffman decoding is the most commonly used entropy decoding method. In the aforementioned photo album scenario, analysis of the load during the process of decoding JPEG images into the original images by electronic devices revealed that the CPU load of the thread decoding JPEG images (in the photo album scenario, JPEG images are decoded into JPEG images using software, i.e., JPEG images are decoded by software) has the highest proportion. The CPU load proportion of each thread is shown in Table 1.

[0030] Table 1

[0031] Thread Classification CPU load percentage JPEG image software decoding thread 75.8％ RenderThread 9.5％ Photo album main thread 6.7％ HeapTaskDaemon 5.4％ Other threads 2.6％

[0032] In some embodiments, the CPU microarchitecture metrics of the electronic device in the photo album scenario are analyzed, showing computationally intensive characteristics, that is, the proportion of computation-related retirement instructions reaches 52%. The analysis of various CPU microarchitecture metrics is shown in Table 2:

[0033] Table 2

[0034] CPU microarchitecture metrics percentage Front-end pause 3％ Backend pause 29％ Branch prediction 16％ retiring 52％

[0035] In some embodiments, the electronic device further breaks down computation-related retiring, revealing that the bottleneck mainly lies in a large number of arithmetic and logical operations. The further breakdown of computation-related retiring is shown in Table 3.

[0036] Table 3

[0037] Arithmetic Logic percentage branch_ratio 9.05％ crypto_ratio 0.00％ integer_dp_ratio 50.06％ load_ratio 12.25％ scalar_tp_ratio 0.05％ simd_ratio 23.32％ store_ratio 3.83％

[0038] In some embodiments, based on the proportion of arithmetic and logical operations in the relevant functions during JPEG image software decoding, it can be found that the arithmetic and logical operations are concentrated in the decode_mcu function, which performs Huffman decoding. decode_mcu occupies more than 50% of the CPU in most JPEG image software decoding. For example, please refer to Table 4 for the proportion of arithmetic and logical operations in the relevant functions during JPEG image software decoding.

[0039] Table 4

[0040] Function Classification percentage decode_mcu 56.97％ jsimd_idct_2x2_neon 2.91％ encode_mcu_gather 2.30％ decompress_onepass 2.19％ jsimd_huff_encode_one_bloc_neon 1.11％ concurrentCopying MarkRoot 0.63％ jsimd_ycc_extrgbx_convert_neon 0.59％ other 33.30％

[0041] Based on the above analysis, it is evident that in a photo album scenario, when an electronic device performs software decoding of JPEG images, the `decode_mcu` function consumes more than 50% of the CPU resources. This means that over 50% of the CPU resources are occupied by the `decode_mcu` function called by the JPEG image software decoding thread, preventing other threads from effectively allocating and utilizing CPU resources. This can easily lead to increased system performance and power consumption in electronic devices. Therefore, in this embodiment, from a CPU microarchitecture perspective, the distribution of the algorithmic logic operation instructions of the `decode_mcu` function is further broken down to identify optimization opportunities and reduce the overhead of arithmetic logic operation instructions.

[0042] In this embodiment, the electronic device generates a decoding table in advance. The decoding table is used to store the correspondence between the image data stream and the coefficient values ​​of the image. When decoding a JPEG image, the first coefficient value corresponding to the image data stream is obtained directly from the decoding table based on the read image data stream. This transforms the step of obtaining the first coefficient value based on the image data stream into a table lookup method, which not only improves the image decoding efficiency but also reduces the proportion of CPU resources used in the image decoding process.

[0043] Please refer to Figure 2 The diagram illustrates a flowchart of an image decoding method according to an exemplary embodiment of this application. (Reference) Figure 2 The method includes:

[0044] Step 201: When decoding an image, determine the image decoding table. The decoding table is used to store the correspondence between the image data stream and the coefficient values. The image data stream is the data stream obtained by encoding the image, and the coefficient values ​​are the quantification of the probability of the image data appearing.

[0045] In some embodiments, one image corresponds to one decoding table; in this step, when decoding an image, the decoding table corresponding to the image is obtained based on the image identifier. In some embodiments, the embodiments of this application can be applied in album scenarios or image transmission scenarios; for example, in an album scenario, when a user views or browses images, it is determined that the image needs to be decoded. Similarly, in an image transmission scenario; for example, when a user transmits an image to a friend via instant messaging, it is determined that the image needs to be decoded when the user selects the image to be sent. Furthermore, when a user transmits an image to another user via a short-range communication module such as Bluetooth, it is determined that the image needs to be decoded when the user selects the image to be sent.

[0046] In some embodiments, the decoding table is generated before decoding the image; that is, the decoding table is generated first, and then the image decoding process is performed (e.g., the Huffman decoding function decode_mcu is executed). The specific process of generating the decoding table in this embodiment is described in detail in the following embodiments.

[0047] Step 202: Read the image data stream.

[0048] The image data stream includes a first preset bit of image data, which is encoded binary data. Image encoding is performed in blocks; that is, the image is divided into multiple data blocks, each with a size of 8*8, and each block is encoded separately to obtain image data. Similarly, image decoding is also performed in blocks; that is, the data stream is read from the image data stream of one data block, and step 203 is executed until the coefficient values ​​of that data block are decoded.

[0049] The first preset bit is greater than the default second preset bit; correspondingly, this step can be to read the image data stream of the first preset bit, which is greater than the default second preset bit. Both the first and second preset bits can be set and changed as needed. In this embodiment, neither the first nor the second preset bit is specifically limited, as long as the first preset bit is greater than the second preset bit; for example, the second preset bit is 8 bits and the first preset bit is 10 bits, meaning the length of the image data stream read each time increases from 8 bits to 10 bits. Another example is the second preset bit being 8 bits and the first preset bit being 12 bits, etc.

[0050] In this embodiment, on the one hand, because the number of bits read from the image data stream each time is increased, image decoding can be achieved with fewer read and decode operations, thereby improving image decoding efficiency. On the other hand, because the number of bits read from the image data stream is increased, the read image data stream can cover more encoded values ​​(if Huffman coding is used to encode the image, the image data stream can cover more Huffman encoded values), enabling most image decoding to be achieved by looking up the decoding table, thereby increasing the table lookup coverage from 90% to over 99%.

[0051] The electronic device includes registers to store a 64-bit length of decoded image data stream, which is the binary data used during decoding. When the remaining length of the image data stream stored in the register is less than or equal to 16 bits, another 48 bits of image data are read from the image data stream and stored in the register. This reading of 48 bits of image data is done serially six times, reading 8 bits of image data each time. After each 8-bit image data stream is read, it is determined whether that 8 bits is an escape byte (FF00) or an image marker bit (FFXX, where X represents non-zero). If the read 8 bits is an escape byte, the FF bits are stored in the register. If the read 8 bits are an image marker bit, they are not stored in the register. Because the proportion of escape bytes and image marker bits in the encoded image data stream is relatively low, often less than 10%, this process is necessary. Repeatedly checking these two cases every time an 8-bit image data stream is read is a huge waste of CPU instructions. It should be noted that most modern electronic devices are 64-bit systems, so their registers have a storage length of 64 bits. If the electronic device is a 32-bit system, the register storage length is 32 bits. In this case, when the remaining length of the image data stream stored in the register is less than or equal to 16 bits, the image data stream will be read serially twice, reducing the number of reads from 6 to 2.

[0052] In this embodiment, the above scheme is optimized using the NEON instruction. The specific process is as follows: First, 64 bits of data are read from the image data stream at once using the ARM NEON instruction. Since 1 byte equals 8 bits, this is equivalent to reading 8 bytes of data at once. Then, these 8 bytes of data are processed in parallel, simultaneously determining whether they contain escape bytes or image marker bits. Finally, 48 bits of the 64-bit data stream are stored in a register. For a 32-bit system, 32 bits of data can be read from the image data stream at once using the ARM NEON instruction, equivalent to reading 4 bytes of data at once. These 4 bytes of data are then processed in parallel, and finally, 16 bits of the 32-bit data stream are stored in a register. Because the ARM NEON instruction performs mathematical operations faster than the memory access and branch judgments in the original process, this embodiment can significantly reduce CPU instruction overhead. Correspondingly, the method also includes:

[0053] Based on the advanced NEON instruction set, multiple image data streams are read simultaneously, and these streams are filtered in parallel. The filtered image data streams are then stored in registers. The filtering process includes determining whether the image data stream contains escape bytes or image marker bits, and filtering out 00s from image marker bits and escape bytes. Therefore, the steps for reading the image data stream can be: reading the image data stream from the register. In this embodiment, the method of reading the data stream is optimized based on ARM NEON instructions, improving parallel processing capabilities and thus increasing the efficiency of image decoding.

[0054] Step 203: Based on the image data stream, find the first coefficient value corresponding to the image data stream from the image decoding table.

[0055] This image data stream is read from a register, and based on the filtering process described above, it is the image data stream after filtering out the image marker bits and escape bytes containing 00. For example, if the first preset bit is 10, then the 10 bits of image data are used as an index to look up the first coefficient value corresponding to that index in the image decoding table.

[0056] In this embodiment, Huffman coding is used to encode the image, and a Huffman tree is generated during the Huffman-based image encoding process. Once the Huffman tree is determined, the decoded value is only related to the combination of the input image data stream. Therefore, under this premise, in the entire decoding chain from the image data stream to the coefficient value, the decoding step can be replaced by a table lookup method, that is, by querying the decoding table to obtain the first coefficient value corresponding to the image data stream. The coefficient values ​​in the decoding table are pre-calculated, which reduces the amount of computation and also solves the problem of dependency between the first preset bits of data in the image data stream during decoding, thereby saving the time of Huffman-based image decoding and reducing the proportion of CPU resources used for decoding the image. Therefore, this embodiment can reduce the CPU load, improve CPU utilization efficiency, and thus improve the user performance experience.

[0057] In this embodiment, obtaining the first coefficient value corresponding to the image data stream through a table lookup method is a CPU microarchitecture optimization for the JPEG image software decoding load. This CPU microarchitecture optimization effectively reduces the load and speeds up JPEG image software decoding. For example, taking the aforementioned photo album scenario, this embodiment can reduce the CPU load of JPEG image software decoding by more than 15% and increase the JPEG image software decoding speed by more than 10%. Of course, this data is based on the photo album scenario; the CPU microarchitecture optimization in this application benefits JPEG image software decoding in other scenarios, not just the photo album scenario.

[0058] Step 204: Decode the image based on the first coefficient value.

[0059] In some embodiments, this step can be implemented by the following steps (1) to (4):

[0060] (1) Perform dequantization on the first coefficient value to obtain the third coefficient value before quantization.

[0061] The first coefficient value includes a luminance coefficient value and a chrominance coefficient value. The luminance coefficient value is a quantized representation of the probability of the luminance of the image occurring; the chrominance coefficient value is a quantized representation of the probability of the chrominance of the image occurring. The electronic device stores a first quantization table corresponding to luminance and a second quantization table corresponding to chrominance. Based on the luminance coefficient value and the first quantization table included in the first coefficient value, the luminance coefficient value is dequantized. Based on the chrominance coefficient value and the second quantization table included in the first coefficient value, the chrominance coefficient value is dequantized. The dequantized luminance coefficient value and the dequantized chrominance coefficient value are combined to form the third coefficient value before quantization.

[0062] The first quantization table and the second quantization table are generated during image encoding. In some embodiments, the step of dequantizing the luminance coefficient values ​​based on the luminance coefficient values ​​included in the first coefficient values ​​and the first quantization table can be as follows: for each luminance coefficient value, find the corresponding quantization value from the first quantization table, multiply the luminance coefficient value by the found quantization value to obtain the dequantized luminance coefficient value. By performing dequantization on each luminance coefficient value in this way, the dequantized luminance coefficient value is obtained.

[0063] In other embodiments, based on the chromaticity coefficient values ​​included in the first coefficient value and the second quantization table, the step of dequantizing the chromaticity coefficient values ​​can be as follows: for each chromaticity coefficient value in the chromaticity coefficient value, find the quantization value corresponding to the chromaticity coefficient value from the second quantization table, multiply the chromaticity coefficient value by the found quantization value to obtain the dequantized chromaticity coefficient value. By performing dequantization on each chromaticity coefficient value in the chromaticity coefficient value in this way, the dequantized chromaticity coefficient value is obtained.

[0064] In some embodiments, the chromaticity coefficient values ​​include chromaticity coefficient values ​​corresponding to the blue chromaticity component and chromaticity coefficient values ​​corresponding to the red chromaticity component. The step of inverse quantization of the chromaticity coefficient values ​​based on the chromaticity coefficient values ​​included in the first coefficient value and the second quantization table can be as follows: inverse quantization is performed on the chromaticity coefficient values ​​corresponding to the blue chromaticity component based on the chromaticity coefficient values ​​included in the first coefficient value and the second quantization table; inverse quantization is also performed on the chromaticity coefficient values ​​corresponding to the red chromaticity component based on the chromaticity coefficient values ​​included in the first coefficient value and the second quantization table. Accordingly, the chromaticity coefficients corresponding to the blue chromaticity component after inverse quantization and the chromaticity coefficients corresponding to the red chromaticity component after inverse quantization are combined to form the inverse quantized chromaticity coefficient value.

[0065] (2) Perform inverse discrete cosine transform on the third coefficient value to obtain grayscale image data.

[0066] The inverse discrete cosine transform (ICT) is the inverse operation of the discrete cosine transform (DCT). The DCT transforms an image from the pixel domain to the frequency domain, compressing it by removing unimportant high-frequency components. The ICT, on the other hand, transforms the compressed frequency-domain data back to the pixel domain, thus restoring the original image.

[0067] (3) Perform color space conversion on the grayscale image data to obtain image data of multiple pixels.

[0068] The color space format of an image can be RGB, BGR, RGBX, BGRX, XBGR, XRGB, GRAY, RGBA, BGRA, ABGR, or ARGB, etc.; while JPEG encoding and decoding of images are usually performed on YUV format images. Therefore, when encoding an image for JPEG, it is necessary to convert the image from RGB, BGR, RGBX, BGRX, XBGR, XRGB, GRAY, RGBA, BGRA, ABGR, or ARGB color space format to YUV format before encoding the YUV format image. When decoding an image using JPEG, the process involves Huffman decoding of the YUV format image, followed by inverse quantization, inverse discrete cosine transform, and color space conversion. This converts the YUV format image back to RGB, BGR, RGBX, BGRX, XBGR, XRGB, GRAY, RGBA, BGRA, ABGR, or ARGB color space formats. This step describes the process of converting a YUV format image back to RGB, BGR, RGBX, BGRX, XBGR, XRGB, GRAY, RGBA, BGRA, ABGR, or ARGB color space formats. In other words, the method of this application is adaptable to images in multiple color space formats, thus enabling decoding of images in various color space formats and improving decoding efficiency.

[0069] In this embodiment, the image color space format is RGB as an example. Accordingly, this step can be: converting the grayscale image to an RGB image by performing a color space conversion. An RGB image includes image data for multiple pixels. For example, a grayscale image includes grayscale values ​​for multiple pixels. In this step, for any pixel, the R channel value of that pixel is increased by 255 to obtain the R channel color space converted value. The G channel value of that pixel is also increased by 255 to obtain the G channel color space converted value. The B channel value of that pixel is also increased by 255 to obtain the B channel color space converted value. The R channel color space converted value, G channel color space converted value, and B channel color space converted value of that pixel constitute the image data for that pixel.

[0070] In some embodiments, since the YUV format image can be sampled during JPEG encoding and then quantized, the grayscale image data can also be inverted during JPEG decoding, before color space conversion, and then the inverted grayscale image data can be converted back to RGB format. The inversion process is responsible for restoring the compressed and decoded data to its original resolution. This step involves resampling and interpolating the image data to compensate for details that may have been lost during encoding.

[0071] Inverse sampling is the reverse process of sampling. Sampling includes multiple sampling formats, such as 4:4:4, 4:2:2, 4:1:1, and 4:2:0. Among them, 4:4:4 represents complete sampling; 4:2:2 represents 2:1 horizontal sampling with complete vertical sampling; 4:1:1 represents 4:1 horizontal sampling with complete vertical sampling; and 4:2:0 represents 2:1 horizontal sampling with 2:1 vertical sampling.

[0072] (4) Generate an image based on image data of multiple pixels.

[0073] The image data of multiple pixels is combined into an image. In some embodiments, since the decoding of the image is also performed in units of data blocks, in this step, the image data of multiple pixels is combined into an image block, and then multiple image blocks are obtained according to the above steps, and the multiple image blocks are combined into an image.

[0074] In this embodiment, the electronic device generates a decoding table in advance. The decoding table is used to store the correspondence between the image data stream and the coefficient values ​​of the image. When decoding a JPEG image, the first coefficient value corresponding to the image data stream is obtained directly from the decoding table based on the read image data stream. This transforms the step of obtaining the first coefficient value based on the image data stream into a table lookup method, which not only improves the image decoding efficiency but also reduces the proportion of CPU resources used in the image decoding process.

[0075] Please refer to Figure 3 The diagram illustrates a flowchart of an image decoding method according to an exemplary embodiment of this application. (Reference) Figure 3 The method includes:

[0076] Step 301: When decoding an image, the electronic device determines the image's decoding table. The decoding table is used to store the correspondence between the image data stream and the coefficient values. The image data stream is the data stream obtained by encoding the image, and the coefficient values ​​are the quantification of the probability of the image data appearing.

[0077] In some embodiments, this step is the same as step 201, and will not be described again here.

[0078] In some embodiments, the process of an electronic device generating a decoding table can be as follows: before decoding the image, the electronic device encodes the image to obtain multiple coefficient values; for any coefficient value, the image data stream corresponding to the coefficient value is determined; and the coefficient value and its corresponding image data stream are stored in the decoding table.

[0079] An electronic device encodes an image using Huffman coding to obtain multiple coefficient values ​​for the image. Furthermore, when encoding an image, the electronic device can switch the image into multiple image blocks, and encode each image block using Huffman coding to obtain multiple coefficient values ​​for the image. In some embodiments, the step of the electronic device determining the image data stream corresponding to the coefficient values ​​can be as follows: the electronic device encodes the coefficient values ​​to obtain codewords for the coefficient values; based on the codewords, it determines the length information of the run-length and additional bits; it adds the length information of the run-length and additional bits to the codewords; based on the frequency of occurrence of each codeword in the codewords, it generates a coding tree; and based on the codewords and coding tree, it generates an image data stream.

[0080] In some embodiments, when generating an image data stream, the electronic device generates an image data stream with a first preset bit length. This facilitates subsequent reading of the image data stream with the first preset bit length and improves the success rate of table lookup when performing a lookup based on the image data stream with the first preset bit length.

[0081] Step 302: The electronic device reads the image data stream of the image.

[0082] In some embodiments, this step is the same as step 202, and will not be described again here.

[0083] Step 303: The electronic device looks up the image decoding table based on the image data stream.

[0084] The electronic device searches the image decoding table based on the image data stream to determine whether a first coefficient value corresponding to the image data stream exists in the image decoding table; if the first coefficient value is found in the image decoding table, step 304 is executed; if the first coefficient value is not found in the image decoding table, steps 305 and 306 are executed.

[0085] Step 304: If the first coefficient value corresponding to the image data stream is found in the image decoding table based on the image data stream, the electronic device decodes the image based on the first coefficient value.

[0086] In some embodiments, the electronic device decodes the image based on the first coefficient value in the same way as step 204, and will not be described again here.

[0087] Step 305: If the first coefficient value is not found in the image decoding table based on the image data stream, the electronic device decodes the image data stream to obtain the second coefficient value.

[0088] If the first coefficient value is not found, the image data stream needs to be decoded through a decoding step; correspondingly, the electronic device can decode the image data stream to obtain the second coefficient value through the following steps (1) to (4), including:

[0089] (1) The electronic device obtains the codewords corresponding to the image data stream by querying the encoding tree based on the image data stream. The encoding tree includes the correspondence between the image data and the codewords included in the image data stream.

[0090] In some embodiments, the electronic device uses Huffman decoding to decode the image, and the encoding tree is a Huffman tree. The image data stream includes binary data of a first preset number of bits, and the Huffman tree is a binary tree. Each node in the binary tree represents a codeword, and the connection between nodes represents the number 0 or 1. Accordingly, in this step, the electronic device queries the Huffman tree bit by bit based on the binary data of the first preset number of bits included in the image data stream to obtain codewords. Each codeword includes at least one codeword, and each codeword corresponds to the encoding of one image data.

[0091] (2) The electronic device parses the encoding to obtain the length information of the run and the additional bits. The run refers to the pixels with the same gray value, and the additional bits are the extra information added when encoding the image.

[0092] When electronic devices encode images, they use a combination of Huffman coding and run-length coding. This means that during image generation, run length and extra bit length information are added to the encoding. By using this run length and extra bit length information, pixels with the same grayscale value can be represented with fewer characters, thus achieving a greater degree of compression. Therefore, image decoding also uses a combination of Huffman decoding and run-length decoding, extracting the run length and extra bit length information from the encoding.

[0093] (3) The electronic device obtains the encoding of the coefficient value from the image data stream based on the length information of the stroke and the additional bit.

[0094] The electronic device determines the starting position of the encoding of coefficient values ​​in the image data stream based on the length information of the stroke and the additional bits, and obtains the encoding of coefficient values ​​from the image data stream based on the starting position.

[0095] (4) The electronic device decodes the encoding of the coefficient value to obtain the second coefficient value.

[0096] The electronic device uses the `decode_mcu` function to decode the encoded coefficient value to obtain the second coefficient value. In this embodiment, when the first coefficient value corresponding to the image data stream cannot be found in the decoding table, the second coefficient value corresponding to the image data stream can be obtained through decoding, thereby improving the success rate of image decoding.

[0097] Step 306: The electronic device decodes the image based on the second coefficient value.

[0098] In some embodiments, this step can be implemented by the following steps (1) to (4):

[0099] (1) The electronic device performs dequantization on the second coefficient value to obtain the fourth coefficient value before quantization.

[0100] The second coefficient value includes a luminance coefficient value and a chrominance coefficient value. The luminance coefficient value is a quantized representation of the probability of the luminance of the image occurring; the chrominance coefficient value is a quantized representation of the probability of the chrominance of the image occurring. The electronic device stores a first quantization table corresponding to luminance and a second quantization table corresponding to chrominance. Based on the luminance coefficient value and the first quantization table included in the second coefficient value, the luminance coefficient value is dequantized. Based on the chrominance coefficient value and the second quantization table included in the second coefficient value, the chrominance coefficient value is dequantized. The dequantized luminance coefficient value and the dequantized chrominance coefficient value are combined to form the fourth coefficient value before quantization.

[0101] (2) Perform inverse discrete cosine transform on the fourth coefficient value to obtain grayscale image data.

[0102] The inverse discrete cosine transform (ICT) is the inverse operation of the discrete cosine transform (DCT). The DCT transforms an image from the pixel domain to the frequency domain, compressing it by removing unimportant high-frequency components. The ICT, on the other hand, transforms the compressed frequency-domain data back to the pixel domain, thus restoring the original image.

[0103] (3) Perform color space conversion on the grayscale image data to obtain image data of multiple pixels.

[0104] The color space format of an image can be RGB, BGR, RGBX, BGRX, XBGR, XRGB, GRAY, RGBA, BGRA, ABGR, or ARGB, etc.; while JPEG encoding and decoding of images are usually performed on YUV format images. Therefore, when encoding an image for JPEG, it is necessary to convert the image from RGB, BGR, RGBX, BGRX, XBGR, XRGB, GRAY, RGBA, BGRA, ABGR, or ARGB color space format to YUV format before encoding the YUV format image. When decoding an image using JPEG, the process involves Huffman decoding of the YUV format image, followed by inverse quantization, inverse discrete cosine transform, and color space conversion. This converts the YUV format image back to RGB, BGR, RGBX, BGRX, XBGR, XRGB, GRAY, RGBA, BGRA, ABGR, or ARGB color space formats. This step describes the process of converting a YUV format image back to RGB, BGR, RGBX, BGRX, XBGR, XRGB, GRAY, RGBA, BGRA, ABGR, or ARGB color space formats. In other words, the method of this application is adaptable to images in multiple color space formats, thus enabling decoding of images in various color space formats and improving decoding efficiency.

[0105] In this embodiment, the image color space format is RGB as an example. Accordingly, this step can be: converting the grayscale image to an RGB image by performing a color space conversion. An RGB image includes image data for multiple pixels. For example, a grayscale image includes grayscale values ​​for multiple pixels. In this step, for any pixel, the R channel value of that pixel is increased by 255 to obtain the R channel color space converted value. The G channel value of that pixel is also increased by 255 to obtain the G channel color space converted value. The B channel value of that pixel is also increased by 255 to obtain the B channel color space converted value. The R channel color space converted value, G channel color space converted value, and B channel color space converted value of that pixel constitute the image data for that pixel.

[0106] In some embodiments, since the YUV format image can be sampled during JPEG encoding and then quantized, the grayscale image data can also be inverted during JPEG decoding, before color space conversion, and then the inverted grayscale image data can be converted back to RGB format. The inversion process is responsible for restoring the compressed and decoded data to its original resolution. This step involves resampling and interpolating the image data to compensate for details that may have been lost during encoding.

[0107] Inverse sampling is the reverse process of sampling. Sampling includes multiple sampling formats, such as 4:4:4, 4:2:2, 4:1:1, and 4:2:0. Among them, 4:4:4 represents complete sampling; 4:2:2 represents 2:1 horizontal sampling with complete vertical sampling; 4:1:1 represents 4:1 horizontal sampling with complete vertical sampling; and 4:2:0 represents 2:1 horizontal sampling with 2:1 vertical sampling.

[0108] (4) Generate an image based on image data of multiple pixels.

[0109] The image data of multiple pixels is combined into an image. In some embodiments, since the decoding of the image is also performed in units of data blocks, in this step, the image data of multiple pixels is combined into an image block, and then multiple image blocks are obtained according to the above steps, and the multiple image blocks are combined into an image.

[0110] In this embodiment, the electronic device generates a decoding table in advance. The decoding table is used to store the correspondence between the image data stream and the coefficient values ​​of the image. When decoding a JPEG image, the first coefficient value corresponding to the image data stream is obtained directly from the decoding table based on the read image data stream. This transforms the step of obtaining the first coefficient value based on the image data stream into a table lookup method, which not only improves the image decoding efficiency but also reduces the proportion of CPU resources used in the image decoding process.

[0111] Please refer to Figure 4 This application illustrates an exemplary embodiment of an image decoding apparatus, the apparatus comprising:

[0112] The first determining module 401 is used to determine the decoding table of the image when decoding the image. The decoding table is used to store the correspondence between the image data stream and the coefficient values ​​of the image. The image data stream is the data stream obtained by encoding the image. The coefficient values ​​are the quantification of the probability of the image data appearing.

[0113] The reading module 402 is used to read the image data stream of the image;

[0114] The lookup module 403 is used to look up a first coefficient value corresponding to the image data stream from the image decoding table based on the image data stream;

[0115] The first decoding module 404 is used to decode the image based on the first coefficient value.

[0116] In some embodiments, the apparatus further includes:

[0117] The filtering and storage module is used to read multiple image data streams of the image at one time based on the advanced single instruction multiple data instruction set NEON instruction, perform filtering processing on the multiple image data streams in parallel, and store the filtered image data streams into a register.

[0118] The reading module 402 is used to read the image data stream of the image from the register.

[0119] In some embodiments, the reading module 402 is used to read a data stream of a first preset bit, wherein the first preset bit is greater than a default second preset bit.

[0120] In some embodiments, the apparatus further includes:

[0121] The second decoding module is used to decode the image data stream to obtain a second coefficient value when the first coefficient value is not found in the decoding table of the image based on the image data stream; and to decode the image based on the second coefficient value.

[0122] In some embodiments, the second decoding module is configured to: query an encoding tree based on the image data stream to obtain codewords corresponding to the image data stream, wherein the encoding tree includes the correspondence between image data included in the image data stream and codewords; parse the encoding to obtain length information of the run and additional bits, wherein the run refers to pixels with the same grayscale value, and the additional bits are extra information added when encoding the image; obtain the encoding of coefficient values ​​from the image data stream based on the length information of the run and the additional bits; and decode the encoding of the coefficient values ​​to obtain the second coefficient value.

[0123] In some embodiments, the first decoding module 404 is configured to perform inverse quantization on the first coefficient value to obtain a third coefficient value before quantization; perform inverse discrete cosine transform on the third coefficient value to obtain grayscale image data; perform color space conversion on the grayscale image data to obtain image data of multiple pixels; and generate the image based on the image data of the multiple pixels.

[0124] In some embodiments, the apparatus further includes:

[0125] An encoding module is used to encode the image to obtain multiple coefficient values ​​before decoding the image;

[0126] The second determining module is used to determine the image data stream corresponding to any coefficient value;

[0127] A storage module is used to store the coefficient values ​​and their corresponding image data streams into the decoding table.

[0128] In this embodiment, the electronic device generates a decoding table in advance. The decoding table is used to store the correspondence between the image data stream and the coefficient values ​​of the image. When decoding a JPEG image, the first coefficient value corresponding to the image data stream is obtained directly from the decoding table based on the read image data stream. This transforms the step of obtaining the first coefficient value based on the image data stream into a table lookup method, which not only improves the image decoding efficiency but also reduces the proportion of CPU resources used in the image decoding process.

[0129] It should be noted that the image decoding device provided in the above embodiments is only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the electronic device can be divided into different functional modules to complete all or part of the functions described above. In addition, the image decoding device and the image decoding method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0130] Electronic devices can be either terminals or servers; when an electronic device is a terminal, please refer to [the relevant documentation / reference]. Figure 5 The diagram illustrates a block diagram of a terminal 500 according to an exemplary embodiment of this application. The terminal 500 in this application may include one or more of the following components: a processor 510, a memory 520, and a display screen 530.

[0131] Processor 510 includes at least two processors, namely a Sensor Hub and a main processor. Processor 510 may include one or more processing cores. Processor 510 connects to various parts within the terminal 500 using various interfaces and lines, and performs various functions of the terminal 500 and processes data by running or executing instructions, programs, code sets, or instruction sets stored in memory 520, and by calling data stored in memory 520. Optionally, processor 510 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). Processor 510 may integrate one or a combination of several of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), Neural-network Processing Unit (NPU), and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content displayed on the screen 530; the NPU is used to implement artificial intelligence (AI) functions; and the modem is used for wireless communication. It is understood that the modem may not be integrated into the processor 510 and can be implemented separately as a computer program product.

[0132] The memory 520 may include random access memory (RAM) or read-only memory (ROM). Optionally, the memory 520 may include a non-transitory computer-readable storage medium. The memory 520 may be used to store instructions, programs, code, code sets, or instruction sets. The memory 520 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch function, sound playback function, image playback function, etc.), instructions for implementing the various method embodiments described below, etc.; the data storage area may store data created based on the use of the terminal 500 (such as audio data, phone book, etc.).

[0133] Display screen 530 is a display component used to display a user interface. Optionally, display screen 530 is a touch-enabled display screen, through which users can use their fingers, styluses, or any suitable object to perform touch operations on display screen 530.

[0134] The display screen 530 is typically located on the front panel of the terminal 500. The display screen 530 can be designed as a full-screen, curved screen, irregularly shaped screen, dual-sided screen, or foldable screen. The display screen 530 can also be designed as a combination of a full-screen and a curved screen, or a combination of an irregularly shaped screen and a curved screen, etc., but this embodiment does not limit it in this way.

[0135] In addition, those skilled in the art will understand that the structure of the terminal 500 shown in the above figures does not constitute a limitation on the terminal 500. The terminal 500 may include more or fewer components than shown, or combine certain components, or have different component arrangements. For example, the terminal 500 may also include a Wi-Fi module, an audio acquisition device, a speaker, a radio frequency circuit, an input unit, a sensor, an audio circuit, a Bluetooth module, a power supply, and other components, which will not be described in detail here.

[0136] When the electronic device is a server, the server's block diagram can be found here. Figure 6 The server 600 can vary considerably depending on its configuration or performance. It may include a Central Processing Unit (CPU) 601 and a memory 602. The memory 602 stores at least one line of program code, which is loaded and executed by the processor 601 to perform the operations performed by the server in the aforementioned vehicle-mounted display method. Of course, the server 600 may also have wired or wireless network interfaces, a keyboard, and input / output interfaces for input and output. The server 600 may also include other components for implementing device functions, which will not be elaborated upon here.

[0137] This application also provides a computer-readable medium storing at least one piece of program code, which is loaded and executed by a processor to implement the image decoding method shown in the above embodiments.

[0138] This application also provides a computer program product that stores at least one piece of program code, which is loaded and executed by a processor to implement the image decoding method shown in the above embodiments.

[0139] In some embodiments, the computer program product involved in the present application can be deployed on an electronic device for execution, or executed on multiple electronic devices located in one location, or executed on multiple electronic devices distributed in multiple locations and interconnected through a communication network. Multiple electronic devices distributed in multiple locations and interconnected through a communication network can form a blockchain system.

[0140] Those skilled in the art will recognize that the functions described in the embodiments of this application in one or more of the above examples can be implemented using hardware, software, firmware, or any combination thereof. When implemented using software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or code on a computer-readable medium. Computer-readable media include computer storage media and communication media, wherein communication media include any medium that facilitates the transfer of a computer program from one place to another. Storage media can be any available medium that can be accessed by a general-purpose or special-purpose computer.

[0141] The above description is merely an optional embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. An image decoding method, characterized in that, The method includes: When decoding an image, a decoding table for the image is determined. The decoding table is used to store the correspondence between the image data stream and the coefficient values. The image data stream is the data stream obtained by encoding the image, and the coefficient values ​​are the quantification of the probability of the image data appearing. Read the image data stream of the image; Based on the image data stream, the first coefficient value corresponding to the image data stream is found from the image decoding table; The image is obtained by decoding based on the first coefficient value.

2. The method according to claim 1, characterized in that, The method further includes: Based on the advanced Single Instruction Multiple Data (NEON) instruction set, multiple image data streams of the image are read at once, the multiple image data streams are filtered in parallel, and the filtered image data streams are stored in registers. The process of reading the image data stream includes: Read the image data stream of the image from the register.

3. The method according to claim 1 or 2, characterized in that, The process of reading the image data stream includes: Read the image data stream with a first preset bit, where the first preset bit is greater than the default second preset bit.

4. The method according to claim 1, characterized in that, The method further includes: If the first coefficient value is not found in the decoding table of the image based on the image data stream, the image data stream is decoded to obtain the second coefficient value; The image is obtained by decoding based on the second coefficient value.

5. The method according to claim 4, characterized in that, Decoding the image data stream to obtain the second coefficient value includes: Based on the image data stream, the codewords corresponding to the image data stream are obtained by querying the encoding tree. The encoding tree includes the correspondence between the image data included in the image data stream and the codewords. The length information of the run and the additional bits is obtained by parsing the encoding. The run refers to the pixels with the same gray value, and the additional bits are extra information added when encoding the image. Based on the length information of the travel and the additional bits, the encoding of the coefficient value is obtained from the image data stream; The second coefficient value is obtained by decoding the encoding of the coefficient value.

6. The method according to claim 1, characterized in that, The process of decoding the image based on the first coefficient value includes: The first coefficient value is dequantized to obtain the third coefficient value before quantization. The third coefficient value is subjected to inverse discrete cosine transform to obtain grayscale image data; The grayscale image data is converted to a color space to obtain image data of multiple pixels; The image is generated based on the image data of the multiple pixels.

7. The method according to claim 1, characterized in that, The method further includes: Before decoding the image, the image is encoded to obtain multiple coefficient values ​​for the image; For any coefficient value, determine the image data stream corresponding to that coefficient value; The coefficient values ​​and their corresponding image data streams are stored in the decoding table.

8. An image decoding device, characterized in that, The device includes: The first determining module is used to determine the decoding table of the image when decoding the image. The decoding table is used to store the correspondence between the image data stream and the coefficient values ​​of the image. The image data stream is the data stream obtained by encoding the image. The coefficient values ​​are the quantification of the probability of the image data appearing. A reading module is used to read the image data stream of the image; The lookup module is used to look up a first coefficient value corresponding to the image data stream from the image decoding table based on the image data stream; The first decoding module is used to decode the image based on the first coefficient value.

9. An electronic device, characterized in that, The electronic device includes a processor and a memory, wherein the memory stores at least one piece of program code, which is loaded and executed by the processor to implement the image decoding method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The storage medium stores at least one piece of program code, which is loaded and executed by a processor to implement the image decoding method as described in any one of claims 1 to 7.

11. A computer program product, characterized in that, The computer program product stores at least one line of program code, which is executed by a processor to implement the image decoding method as described in any one of claims 1 to 7.

Images