Image processing method, device and system, streaming media server and computer terminal

Abstract

The invention discloses an image processing method, device and system, a streaming media server and a computer terminal. Relates to the field of image compression, and the method comprises the steps: traversing an image, determining a prediction mode of a pixel value of an image block, and obtaining a first target prediction mode; determining a token probability table based on the first target prediction mode; updating the prediction mode according to the token probability table to obtain a second target prediction mode; coding the image block based on the second target prediction mode and the token probability table to obtain a coding result; and determining a compressed image according to the coding results of all the image blocks. According to the invention, the problem that a hardware encoder determines a prediction mode of a pixel value based on a predefined token probability table and carries out entropy coding on a coding transformation coefficient in a corresponding prediction mode by adopting the predefined token probability table due to the difference between the predefined token probability table and a token probability table required by actual coding is solved; and the compression performance of the hardware encoder is low.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image compression, and in particular, to an image processing method, device, system, streaming media server and computer terminal. BACKGROUND

[0002] WebP is an image format supporting multiple encoding modes such as lossy and lossless compression, transparency (alpha channel) and animation. The encoding process of a WebP format image is complex and computationally intensive. In the lossy encoding mode, the encoding process of a WebP format image involves a large amount of data processing, including pixel value prediction, spatial and frequency domain conversion, quantization, entropy encoding and other steps, and has a high requirement for computing resources. Therefore, in the processing of high-resolution and large image data, a software-based encoder cannot meet the performance or energy efficiency requirements.

[0003] When encoding a multimedia file using the WebP encoding standard, hardware devices can be used for encoding. In order to improve the encoding efficiency of hardware encoding without sacrificing the data throughput of hardware devices, the hardware encoding device usually encodes a plurality of image blocks corresponding to the multimedia file according to a predefined probability model (probability table of tokens of encoding transform coefficients of encoding blocks), and transmits the obtained quantity encoding file after encoding, thereby reducing the resources required for file transmission.

[0004] It should be noted that the probability model (probability table of tokens of encoding transform coefficients of encoding blocks) used in the entropy encoding process of hardware encoding can only consider the current encoding block and the encoding blocks before the current encoding block, and cannot obtain the information of the subsequent blocks of the current encoding block. The WebP encoding standard uses the same probability model for coefficient encoding of the entire frame of an image, which makes it necessary to wait until the entire frame encoding is completed to obtain the appropriate probability model. When encoding the current block, the rate-distortion cost estimation of the prediction mode of the pixel value is performed by using the probability model to determine the appropriate prediction mode, and the probability model used in the rate-distortion cost estimation and the probability model updated after the entire frame encoding are cross-entropy, which reduces the WebP compression efficiency.

[0005] At present, no effective solution has been proposed to solve the above problems. SUMMARY

[0006] The embodiment of the present application provides an image processing method, device, system, streaming media server and computer terminal, so as to at least solve the technical problem of low compression performance of a hardware encoder due to the difference between a predefined token probability table and an actual token probability table required for encoding, the hardware encoder determines a prediction mode of pixel values based on the predefined token probability table, and the hardware encoder adopts the predefined token probability table to perform entropy encoding on the encoded transform coefficients in the corresponding prediction mode.

[0007] According to an aspect of the embodiment of the present application, an image processing method is provided, comprising: traversing an image, determining a prediction mode of pixel values of an image block in each case of traversing the image block, to obtain a first target prediction mode; determining a token probability table used for encoding the image block based on the first target prediction mode, wherein the token probability table comprises probabilities of tokens of encoded transform coefficients; updating the prediction mode of the image block according to the token probability table, to obtain a second target prediction mode; encoding the image block based on the second target prediction mode and the token probability table, to obtain an encoding result; and determining a compressed image according to the encoding results of all image blocks of the image.

[0008] According to another aspect of the embodiment of the present application, another image processing method is further provided, comprising: an encoder receiving an image sent by a data source, wherein the image comprises a plurality of image blocks; the encoder traversing the image, determining a prediction mode of pixel values of an image block in each case of traversing the image block, to obtain a first target prediction mode, determining a token probability table used for encoding the image block based on the first target prediction mode, updating the prediction mode of the image block according to the token probability table, to obtain a second target prediction mode, encoding the image block based on the second target prediction mode and the token probability table, to obtain an encoding result, and determining a compressed image according to the encoding results of all image blocks of the image; and the encoder transmitting the compressed image to an image receiving end, wherein the image receiving end decodes the compressed image to obtain the image.

[0009] According to another aspect of the embodiment of the present application, a streaming media server is further provided, comprising: a hardware encoder, configured to receive a video file, and perform the image processing method in any of the above aspects on a frame image in the video file to obtain a code stream; a storage system, configured to store the video file and the code stream in the compressed image; and a streaming media processing engine, configured to transmit the code stream to a user end through a network, wherein the user end is configured to decode the code stream and display the decoded frame image.

[0010] According to another aspect of the embodiments of the present application, an image processing apparatus is also provided, comprising: a first determining unit configured to traverse an image, and determine a prediction mode of pixel values of an image block in each case of traversing the image block, to obtain a first target prediction mode; a second determining unit configured to determine a token probability table used for encoding the image block based on the first target prediction mode, wherein the token probability table comprises probabilities of tokens of encoded transform coefficients; a third determining unit configured to update the prediction mode of the image block according to the token probability table, to obtain a second target prediction mode; an encoding unit configured to encode the image block based on the second target prediction mode and the token probability table, to obtain an encoding result; and a compression unit configured to determine a compressed image according to the encoding result of all image blocks of the image.

[0011] According to another aspect of the embodiments of the present application, a computer terminal is also provided, comprising: a memory storing an executable program; and a processor configured to run the program, wherein the program, when running, performs the data acquisition method of any one of the above.

[0012] According to another aspect of the embodiments of the present application, a computer program product is also provided, comprising a non-volatile computer readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the data acquisition method of any one of the above.

[0013] According to another aspect of the embodiments of the present application, an image processing system is also provided, comprising: a processor; and a memory connected to the processor, configured to provide the processor with instructions for processing the following processing steps: traversing an image, and determining a prediction mode of pixel values of an image block in each case of traversing the image block, to obtain a first target prediction mode; determining a token probability table used for encoding the image block based on the first target prediction mode, wherein the token probability table comprises probabilities of tokens of encoded transform coefficients; updating the prediction mode of the image block according to the token probability table, to obtain a second target prediction mode; encoding the image block based on the second target prediction mode and the token probability table, to obtain an encoding result; and determining a compressed image according to the encoding result of all image blocks of the image.

[0014] In the embodiment of the present application, the image is encoded by means of double-pass encoding. In the first pass, the encoding setting parameters are determined, including the prediction mode of the pixel value of the image block and the token probability table used for encoding the image block. In the second pass, the prediction mode of the pixel value of the image block is updated according to the token probability table to obtain a second target prediction mode with higher compression efficiency, and the image block is encoded according to the second target prediction mode and the token probability table to obtain the compressed image, so as to improve the compression efficiency of the image, thereby achieving the technical effect of compensating the cross entropy between the pre-defined token probability table and the token probability table actually required for encoding, and further solving the technical problem of low compression performance of the hardware encoder caused by the difference between the pre-defined token probability table and the token probability table actually required for encoding, in which the hardware encoder determines the prediction mode of the pixel value based on the pre-defined token probability table, and uses the pre-defined token probability table to perform entropy encoding on the encoding transform coefficient in the corresponding prediction mode.

[0015] It is easy to note that the general description above and the detailed description below are merely for exemplifying and explaining the present application, and do not constitute a limitation on the present application. BRIEF DESCRIPTION OF DRAWINGS

[0016] The drawings described herein are used to provide further understanding of the present application, constitute a part of the present application, and the illustrative embodiments of the present application and the description thereof are used to explain the present application, and do not constitute an improper limitation on the present application. In the drawings:

[0017] Figure 1 is a schematic diagram of a hardware structure block diagram of a computer terminal for implementing an image processing method according to an embodiment of the present application;

[0018] Figure 2 is a flowchart of an image processing method according to an embodiment of the present application;

[0019] Figure 3 is a structure diagram of a binary tree according to an embodiment of the present application;

[0020] Figure 4 is a flowchart of another image processing method according to an embodiment of the present application;

[0021] Figure 5 is a flowchart of an image processing method based on double-pass traversal according to an embodiment of the present application;

[0022] Figure 6 is a structure diagram of an encoder according to an embodiment of the present application;

[0023] Figure 7 is a structure diagram of a streaming media server according to an embodiment of the present application;

[0024] Figure 8 This is a structural diagram of an image processing apparatus according to an embodiment of this application;

[0025] Figure 9 This is a structural block diagram of a computer terminal according to an embodiment of this application. Detailed Implementation

[0026] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0027] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0028] First, some nouns or terms that appear in the description of the embodiments of this application shall be interpreted as follows:

[0029] WebP encoding: WebP is an image format that supports lossless and lossy compression, as well as transparency and animation features. WebP encoding refers to the process of compressing images using the WebP image compression standard. WebP encoding has efficient compression characteristics, which can produce smaller files than other common image encoding formats while maintaining high image quality.

[0030] Hardware encoding: Hardware encoding involves using dedicated hardware devices to perform the encoding process, rather than using a processor or graphics processor at the software level. Hardware encoders are designed to perform specific tasks and are optimized for those tasks. For example, in video processing, hardware encoders can achieve highly efficient video transcoding because they are dedicated devices, offering faster performance than software encoders and reducing the load on the main processor.

[0031] 2-pass coding, also known as two-pass encoding, is a technique used in video and audio compression for applications requiring high quality. It involves two independent passes in the encoding process. In the first pass, the encoder analyzes the entire video or audio content, collecting important statistical information about the data, such as frame complexity, motion, and color distribution. This information is then used in the second pass to optimize the encoding process, achieving more precise control over target bitrate allocation, image quality, or compression level.

[0032] A token probability table is a statistical data structure that contains a set of tokens and the probabilities of those tokens appearing. In encoding and data compression scenarios, a "token" can represent a symbol or item in the input data, such as a character or a word. Token probability tables are used to guide encoder selection, optimize compression performance, or serve as a prediction mechanism when generating the output of a language model. By configuring an appropriate token probability table, the encoder can achieve higher file compression performance.

[0033] Example 1

[0034] According to an embodiment of this application, an image processing method is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0035] The method embodiment provided in Embodiment 1 of this application can be executed in a mobile terminal, computer terminal or similar computing device. Figure 1 This is a schematic diagram of the hardware structure of a computer terminal for implementing an image processing method according to an embodiment of this application. Figure 1 As shown, computer terminal 10 (or mobile device) may include one or more ( Figure 1 The processor 102 (which may include, but is not limited to, a microprocessor MCU (Microcontroller Unit) or a programmable logic device FPGA, etc.), a memory 104 for storing data, and a transmission module 106 for communication functions may also be included. In addition, it may include: a display, an input / output interface (I / O interface), a Universal Serial Bus (USB) port (which may be included as one of the ports of a BUS bus), a network interface, a power supply, and / or a camera. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the aforementioned electronic device. For example, computer terminal 10 may also include... Figure 1The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0036] It should be noted that the aforementioned one or more processors 102 and / or other data processing circuits are generally referred to herein as "data processing circuits". These data processing circuits may be embodied, in whole or in part, in software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuits may be a single, independent processing module, or may be integrated, in whole or in part, into any other element within the computer terminal 10 (or mobile device). As involved in the embodiments of this application, the data processing circuits serve as a processor control mechanism (e.g., selection of a variable resistor termination path connected to an interface).

[0037] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the image processing method in this embodiment. The processor 102 executes different functional applications and data processing by running the software programs and modules stored in the memory 104, thereby realizing the above-mentioned image processing method. The memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the computer terminal 10 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0038] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of the computer terminal 10. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module, used for wireless communication with the Internet.

[0039] The display can be, for example, a touchscreen liquid crystal display (LCD), which allows the user to interact with the user interface of the computer terminal 10 (or mobile device).

[0040] In the field of image compression, hardware encoders can be implemented based on FPGAs (Field Programmable Gate Arrays) and encode and compress images in the following ways:

[0041] First, the image is preprocessed using an FPGA, including color space conversion and block segmentation (dividing the image into blocks of a specified size, typically 4x4, 8x8, or 16x16 pixel blocks). Then, the FPGA performs intra-frame prediction, predicting the pixel values ​​of different image blocks by referencing adjacent reconstructed pixels, and calculating the difference between the predicted and original pixel values ​​to obtain residual values. Further, the FPGA performs Discrete Cosine Transform (DCT) to transform the residual values ​​of different image blocks from the spatial domain to the frequency domain, identifying the high-frequency and low-frequency components of the image and obtaining DCT coefficients. Next, the FPGA quantizes the DCT coefficients, removing high-frequency details imperceptible to the human eye, thus reducing the storage space required for the DCT coefficients. The quantized DCT coefficients are then scanned in a zigzag pattern, rearranging them into a one-dimensional array. Furthermore, based on the subsequent entropy coding process, the bit rate cost is estimated, and the inverse quantization and inverse transformation process is executed to calculate the reconstructed pixel values ​​and estimate the distortion cost.

[0042] Repeat the above process to perform image segmentation, prediction, transformation, quantization, scanning, entropy coding and other calculations, thereby inferring the prediction method with the lowest rate distortion pixel values. Compared with other prediction methods, it has the lowest reconstruction quality performance at the same bit rate.

[0043] Having obtained a prediction method with the lowest rate-distortion pixel values, the FPGA performs pixel value prediction, spatial and frequency domain conversion, quantization, and entropy encoding based on this method. The packaging module is responsible for packaging the entropy-encoded data and necessary metadata (such as image size and color configuration) into RIFF WebP format (Resource Interchange File Format). The encoded data is then sent out through the FPGA's output interface, either by direct storage to external memory or by direct transmission through the interface.

[0044] Under the aforementioned image encoding method and operating environment, this application provides the following: Figure 2 The image processing method shown. Figure 2 This is a flowchart of an image processing method according to an embodiment of this application. It includes the following steps:

[0045] S202, Traverse the image. In each traversal of an image block, determine the prediction method for the pixel values ​​of the image block to obtain the first target prediction method.

[0046] Specifically, an image refers to image data that needs to be compressed. An image block, also known as a macroblock in the field of image processing, refers to a region composed of multiple pixels in an image. The pixel value prediction method refers to the way the pixel value is estimated using the internal data of the image in the WebP compression algorithm. In the WebP compression algorithm, different image blocks can choose different prediction methods, such as DC, TM, vertical, horizontal, and non-vertical horizontal angle prediction methods.

[0047] During the first pass through the image, the image to be compressed is divided into multiple image blocks. For each image block, the available pixel value prediction methods are counted. Different prediction methods result in different bitrate and distortion costs, leading to different rate-distortion costs. To select the prediction method with the highest compression efficiency (i.e., the lowest rate-distortion cost), the first target prediction method with the lowest rate-distortion cost is selected from multiple candidate prediction methods for each image block. For example, for 16×16 and 8×8 image blocks, the available pixel value prediction methods include DC, TM, vertical, and horizontal, a total of four prediction methods. For a 4×4 image block, the available pixel value prediction methods include DC, TM, vertical, horizontal, and six non-vertical / horizontal angle prediction methods, a total of ten prediction methods. It should be noted that 4×4, 16×16, and 8×8 refer to components at different scales of the image block. 4×4 and 16×16 image blocks represent the luminance component, while 8×8 image blocks represent the chrominance component.

[0048] Among the prediction methods, DC prediction refers to predicting pixel values ​​based on the average brightness or grayscale value of an image. In some cases, the DC component can represent the overall brightness of the image while ignoring detailed information. TM prediction refers to predicting pixel values ​​in image blocks through the "TM domain," which is the domain obtained after the image has undergone some transformation, such as Fourier transform or wavelet transform. In these transformed domains, image information is redistributed, making it easier to analyze or process. Vertical prediction considers vertical continuity when predicting pixel values ​​in image processing. For example, in image compression, it can be assumed that adjacent vertical pixels have similar values, thus reducing data volume. Horizontal prediction considers horizontal continuity when predicting pixel values. In image compression or prediction, adjacent horizontal pixels may have similar values. Non-vertical / horizontal angular prediction refers to predicting pixel values ​​based on the angular position of a pixel in the image. For example, the brightness or color value of a pixel might be predicted based on its angle relative to the image center.

[0049] S204, determine a token probability table for encoding image blocks based on the first target prediction method, wherein the token probability table includes the probability of tokens encoding transform coefficients.

[0050] It should be noted that image coding can be based on Huffman coding to achieve entropy coding. In the entropy coding stage, a probabilistic model (i.e., a token probability table) is needed to construct the coding table. In order to obtain more accurate probabilistic information, the rate-distortion cost of different prediction methods is evaluated, and then the prediction method with the lowest rate-distortion cost and related information are selected to obtain the first target prediction method. Then, the frequency of different tokens in the image block is statistically analyzed when image coding is performed under the first target prediction method, and a token probability table for encoding the image block is constructed.

[0051] Specifically, the token probability table refers to the probability table of tokens used to guide the entropy coding of image blocks. Pixel values ​​in the image block are predicted using a first target prediction method to obtain predicted pixel values. The difference between the predicted pixel value and the reference pixel value is calculated and processed to obtain the residual. The reference pixel value refers to the pixel value of the pixel preceding the predicted pixel. For example, if the first target prediction method is a vertical prediction method, and the predicted pixel is 'a', the pixel 'b' directly above 'a' in the vertical direction is the reference pixel, and the pixel value of 'b' is the reference pixel value. The residual is transformed and quantized to obtain coded transform coefficients. Then, the coded transform coefficients are encoded as tokens. The token is used to represent a symbol or item of a coded transform coefficient. For example, for residual transform coefficients, the token can be the level of the residual transform coefficient. Then, the probability of tokens appearing in the image block is statistically analyzed to generate the token probability table.

[0052] S206, Update the prediction method of the image block according to the token probability table to obtain the second target prediction method.

[0053] It should be noted that the first target prediction method is based on a predefined token probability table to calculate the rate-distortion cost of candidate prediction methods, thereby selecting the method from multiple candidate prediction methods. However, there is cross-entropy between the predefined token probability table and the token probability table required for actual encoding, which affects the compression efficiency. Therefore, in order to further optimize the compression efficiency, after statistically analyzing the frequency of different tokens in the image block when encoding the image under the first target prediction method and constructing a more accurate token probability table, it is necessary to re-select the prediction method with the lowest rate-distortion cost.

[0054] Therefore, during the second traversal of the image, the token probability table obtained during the first traversal guides the selection of the second target prediction method. That is, the rate-distortion cost of the non-prediction method is calculated based on the token probability table obtained during the first traversal, and then the prediction method with the lowest rate-distortion cost is selected, which is the second target prediction method.

[0055] S208, the image block is encoded based on the second target prediction method and the token probability table to obtain the encoding result.

[0056] After determining the second target prediction method, the pixel values ​​of the image blocks are predicted according to the second target prediction method. Then, the residuals are calculated using the predicted pixel values ​​obtained by the second target prediction method, and transformed and quantized to obtain the coding transformation coefficients. The coding transformation coefficients are entropy encoded using a token probability table, and the entropy encoding results are packaged to obtain the coding result of each image block.

[0057] S210, determine the compressed image based on the encoding results of all image blocks of the image.

[0058] After obtaining the encoding results of all image blocks, the encoding results of all image blocks are combined to obtain the compressed image.

[0059] In this embodiment of the invention, image encoding is performed using a two-pass encoding method. In the first pass, encoding settings parameters are determined, including the prediction method for pixel values ​​of image blocks and a token probability table for encoding image blocks. In the second pass, the prediction method for pixel values ​​of image blocks is updated according to the token probability table to obtain a second target prediction method with higher compression efficiency. The image blocks are then encoded according to the second target prediction method and the token probability table to obtain the compressed image, thereby improving the image compression efficiency. This achieves the technical effect of compensating for the cross-entropy between the predefined token probability table and the token probability table required for actual encoding. Furthermore, it solves the technical problem that the hardware encoder has low compression performance due to the difference between the predefined token probability table and the token probability table required for actual encoding, which causes the hardware encoder to determine the prediction method for pixel values ​​based on the predefined token probability table and perform entropy encoding on the encoding transform coefficients under the corresponding prediction mode using the predefined token probability table.

[0060] After determining the first target prediction method, the pixel values ​​of the image are predicted according to the first target prediction method, and a token probability table is compiled based on the prediction results. Optionally, in the image processing method of this embodiment, determining the token probability table for encoding image blocks based on the first target prediction method includes: predicting the pixel values ​​of the image blocks based on the first target prediction method to obtain predicted pixel values; obtaining reference pixel values ​​of the image blocks, and processing the difference between the predicted pixel values ​​and the reference pixel values ​​into encoding transform coefficients; determining an initial token probability table based on the occurrence frequency of tokens of encoding transform coefficients; calculating the ratio of the probability of tokens of encoding transform coefficients to the total number of tokens in the initial token probability table; quantizing the ratio of the probability of tokens of encoding transform coefficients to the total number of tokens to obtain a quantized value; and updating the probability of tokens with different encoding transform coefficients in the initial token probability table to the quantized value to obtain the token probability table.

[0061] The difference between the predicted pixel value and the reference pixel value can be measured using SATD (Sum of Absolute Transformed Differences, used in image and video compression to measure the difference between image patches). This difference value can be processed by a discrete cosine transform (DCT) to convert it from the spatial domain to the frequency domain, thereby compressing high-frequency details in the image and converting the difference value into coded transform coefficients. To facilitate entropy coding of the coded transform coefficients corresponding to the image patches, the tokens corresponding to the coded transform coefficients need to be determined. Then, the frequency of token occurrences in the image patches is statistically analyzed to obtain an initial token probability table. Since the values ​​in the initial token probability table have a large dynamic range, quantization is used to further reduce the amount of data in the image. For example, the ratio of the probability of different tokens to the total number of tokens is calculated, and this ratio replaces the original token probabilities. The ratio can be further quantized according to a preset quantization step size, for example, an 8-bit depth. The quantized value of the ratio is used as the probability of the tokens appearing in the token probability table.

[0062] It's important to note that the token probability table contains four distinct indices: the plane, the frequency band, the token value of the previous pixel's encoded transform coefficient, and the current token value. The token probability table is updated by recording bits, encoding records along the WebP standard's token encoding tree according to the index. When a node (i.e., the token) is encoded, the total number of records corresponding to that node is incremented by one; when a node's encoding value is 1, the 1 count for that node's record is incremented by one. Here, the 1 count refers to a counter representing the number of times a specific event, such as a particular token or coefficient value, occurs within the entropy coding context model. The total number of records refers to the sum of the 1 counts for all possible events within the entropy coding context. The ratio of the 1 count to the total number of records can be calculated for different items in the token probability table, and this ratio can be quantized to 8 bits. The 8-bit quantized ratio is then used as the probability to update the token probability table.

[0063] This application embodiment uses an initial prediction method for searching images and a first target prediction method based on minimum rate distortion cost screening. Under the first target prediction method, the ratio of the occurrence frequency of different tokens to the total occurrence frequency of tokens is calculated to determine the probability value of the tokens. The probability value of the tokens is then quantized to a preset bit depth, and the probability values ​​of the tokens in the initial token probability table are updated. The updated token probability table can more accurately reflect the statistical characteristics of the current image content, thereby improving compression efficiency.

[0064] The first target prediction method is selected from the initial prediction method set based on rate-distortion cost. Optionally, in the image processing method of this embodiment, determining the prediction method for the pixel value of an image block to obtain the first target prediction method includes: determining the initial prediction method set for the image block and calculating the predicted pixel value of the image block under different initial prediction methods in the initial prediction method set; obtaining the reference pixel value of the image block and calculating the difference between the predicted pixel value and the reference pixel value under different initial prediction methods; converting the initial prediction method set into an initial prediction method sequence according to the order of the difference value of the image block under different initial prediction methods from smallest to largest; selecting a first candidate prediction method from the initial prediction method sequence with a preset number of first candidate prediction methods to obtain a first candidate prediction method sequence; and selecting the prediction method with the minimum rate-distortion cost from the first candidate prediction method sequence to obtain the first target prediction method.

[0065] The difference value can be the SATD evaluation value under the current prediction method, used to measure the magnitude of the residual between the predicted pixel value and the reference pixel value. Different prediction methods are used to predict image patches, and the SATD evaluation values ​​under different prediction methods are calculated. The prediction methods for image patches are sorted in ascending order of SATD evaluation values ​​to obtain an initial test method sequence. Then, a preset number of test methods are selected from the initial test method sequence as the first candidate prediction methods for the image patches, resulting in a first candidate prediction method sequence. The rate-distortion cost of the first candidate prediction methods is then calculated, and the first candidate prediction method corresponding to the minimum rate-distortion cost is selected as the first target prediction method.

[0066] For example, the initial prediction methods for a 4x4 image block include DC, TM, vertical, horizontal, and six non-vertical / horizontal angle prediction methods, totaling 10 prediction methods. Nine prediction methods, excluding the DC prediction method, are applied to the 4x4 image block. The original pixel is used as the reference pixel to remove the dependency of the rate distortion optimization loop. Then, the SATD evaluation values ​​between different predicted pixels and the original pixels are calculated and recorded. The initial prediction method set is converted into an initial prediction method sequence based on the SATD evaluation values ​​in ascending order. Since a 16x16 image block consists of four 4x4 image blocks, when calculating the SATD evaluation values ​​of the prediction methods for the 16x16 image block, for the four prediction methods (DC, TM, vertical, and horizontal), the SATD evaluation values ​​of the four 4x4 image blocks under the corresponding prediction method can be accumulated to deduce the SATD evaluation values ​​of different prediction methods for the 16x16 image block to which the four 4x4 image blocks belong.

[0067] It should be noted that different numbers of candidate prediction methods can be selected for different image blocks, i.e., preset numbers. For example, for a 4x4 image block, the three prediction methods with the lowest SATD evaluation value are retained along with the DC prediction method to construct the first candidate prediction method sequence for the 4x4 image block. For a 16x16 image block, the two prediction methods with the lowest SATD evaluation value are retained along with the DC prediction method to construct the first candidate prediction method sequence for the 16x16 image block. The first candidate prediction method sequence for an 8x8 image block can be reused from the first candidate prediction method sequence for the 16x16 image block; that is, to save computational resources, the first candidate prediction method sequence for the 8x8 image block does not need to be calculated separately. The above calculation process is repeated for different image blocks until the first candidate prediction method sequences for all components of all image blocks are constructed.

[0068] This application embodiment calculates the SATD evaluation value of different prediction methods for image blocks to filter the first candidate prediction method sequence from the initial prediction method set of image blocks, and then determines the first target prediction method with the minimum rate distortion cost from the first candidate prediction method sequence, thereby achieving the purpose of efficiently determining the prediction method with appropriate pixel values.

[0069] Since the token probability table has been updated, in order to further improve the image compression efficiency, a second target prediction method with the minimum rate-distortion cost is re-selected based on the first candidate prediction method sequence and the token probability table. Optionally, in the image processing method of this embodiment of the invention, updating the prediction method of the image block according to the token probability table to obtain the second target prediction method includes: selecting N second candidate prediction methods from the first candidate prediction method sequence to obtain a set of second candidate prediction methods, wherein the set of second candidate prediction methods includes the first target prediction method, N is a positive integer, and N is less than the number of all prediction methods in the first candidate prediction method sequence; determining the rate-distortion cost of the image block under the second candidate prediction method according to the token probability table, and selecting the second candidate prediction method with the minimum rate-distortion cost from the set of second candidate prediction methods to obtain the second target prediction method.

[0070] It should be noted that although rate-distortion cost optimization was performed during the first image traversal, i.e., the first target prediction method with the lowest rate-distortion cost was selected, the first target prediction method is the prediction method with the lowest rate-distortion cost selected by evaluating the rate-distortion cost of the first candidate prediction method according to a predefined token probability table. The predefined token probability table cannot accurately represent the features of the image. After predicting the pixel values ​​in the image through the first target prediction method, the resulting token probability table can more accurately represent the features of the image compared to the predefined token probability table.

[0071] Therefore, during the second traversal, rate-distortion optimization can be performed again based on the token probability table generated by the first target prediction method. That is, the rate-distortion cost of different first candidate prediction methods is re-evaluated, thereby selecting the more accurate second target prediction method with the minimum rate-distortion cost. This embodiment achieves the goal of accurately determining the prediction method with the minimum rate-distortion cost through a more precise rate-distortion cost evaluation scheme by re-evaluating the rate-distortion cost of the first candidate prediction methods using the token probability table obtained under the first target prediction method.

[0072] It should be noted that in the first rate-distortion cost optimization, the first target prediction method is directly selected from the initial prediction method sequence. However, the rate-distortion optimization in the second traversal process includes the rate-distortion optimization process in the first traversal process. In the second rate-distortion cost optimization process, the selection range of prediction methods can be composed of the first target prediction method and the first candidate prediction method sequence. The difference between the two rate-distortion cost optimizations is that the second rate-distortion cost optimization is based on a dynamic candidate prediction method sequence, thereby improving the rate-distortion optimization efficiency while ensuring the rate-distortion optimization effect.

[0073] For image patches with different components, a corresponding filtering method for the second candidate prediction method set can be set. Optionally, in the image processing method of this embodiment, the image patch includes a first luminance component, a chrominance component, and a second luminance component in descending order of scale. The first candidate prediction method set includes a first candidate prediction method set under the first luminance component, a first candidate prediction method set under the chrominance component, and a first candidate prediction method set under the second luminance component. The second candidate prediction method includes a second candidate prediction method set under the first luminance component, a second candidate prediction method set under the chrominance component, and a second candidate prediction method set under the second luminance component. Filtering N second candidate prediction methods from the first candidate prediction method sequence to obtain the second candidate prediction method set includes: for the first luminance component, if the first target prediction method under the first luminance component is the same as the Nth prediction method in the first candidate prediction method sequence under the first luminance component, obtaining the first N first candidate prediction methods in the first candidate prediction method sequence under the first luminance component to obtain the second candidate prediction method set under the first luminance component. The first target prediction method under the first luminance component is the same as the Nth prediction method in the first candidate prediction method sequence under the first luminance component. When the prediction methods are different, N first candidate prediction methods other than the Nth prediction method in the first candidate prediction method sequence under the first luminance component are obtained to obtain the second candidate prediction method set under the first luminance component. For the chrominance component, if the first target prediction method under the chrominance component is the same as the Nth prediction method in the first candidate prediction method sequence under the chrominance component, the first N first candidate prediction methods in the first candidate prediction method sequence under the chrominance component are obtained to obtain the second candidate prediction method set under the chrominance component. If the first target prediction method under the chrominance component is different from the Nth prediction method in the first candidate prediction method sequence under the chrominance component, N first candidate prediction methods other than the Nth prediction method in the first candidate prediction method sequence under the chrominance component are obtained to obtain the second candidate prediction method set under the chrominance component. For the second luminance component, the second candidate prediction method set under the second luminance component is determined according to the prediction methods in the first candidate prediction method sequence under the second luminance component. The second candidate prediction method set for the image block is determined according to the second candidate prediction method set under the first luminance component, the second candidate prediction method set under the chrominance component, and the second candidate prediction method set under the second luminance component.

[0074] The first luminance component can be a 16×16 image block, the chrominance component can be an 8×8 image block, and the second luminance component can be a 4×4 image block. For 16×16 and 8×8 image blocks, the Nth method can be the third prediction method. For example, if the first candidate prediction method sequence for the image block is {DC, TM, Vertical, Horizontal}, and the first target prediction method is the vertical prediction method (i.e., the first target prediction method is the third prediction method in the first candidate prediction method sequence), then the first three prediction methods in the first candidate prediction method sequence, i.e., {DC, TM, Vertical}, are used as the second candidate prediction method set. If the first target prediction method is DC, TM, or Horizontal (i.e., the first target prediction method is not the third prediction method in the first candidate prediction method sequence), then {DC, TM, Horizontal} are used as the second candidate prediction method set. For 4×4 image blocks, since the effect of re-determining the second candidate prediction method set is not significant, to save computational power, the prediction methods in the first candidate prediction method sequence can be reused as the second candidate prediction method set.

[0075] According to the embodiment of this application, the second candidate prediction method set is dynamically selected from the first candidate prediction method sequence based on the position of the first target prediction method in the first candidate prediction method sequence. This ensures that the second target prediction method selected in the second traversal is based on the first target prediction method, thus ensuring that the best prediction method is selected from the best, making the second target prediction method more accurate in predicting the pixel values ​​in the image, thereby improving the compression efficiency.

[0076] To select the first target prediction method, it is necessary to calculate the rate-distortion cost of different first candidate prediction methods. Optionally, in the image processing method of this embodiment, the rate-distortion cost of an image block under the first candidate prediction method is determined by the following method: determining the first code rate value of the image block under the first candidate prediction method based on a preset token probability table; calculating the reconstructed pixel value and the distorted pixel value of the image block under the first candidate prediction method; calculating the first distortion value based on the reconstructed pixel value and the distorted pixel value of the image block under the first candidate prediction method; and determining the rate-distortion cost of the image block under the first candidate prediction method based on the first code rate value and the first distortion value.

[0077] The preset token probability table refers to the token probability table predefined during the first traversal. During rate-distortion optimization of the image, the encoder evaluates the distortion cost and bitrate cost of different encoding decisions, determining the rate-distortion cost based on both. By selecting the first target prediction method and quantization parameters that minimize the rate-distortion cost, the desired compression ratio and image quality are achieved. The rate-distortion cost is calculated as: Rate-distortion cost = R * λ + D. Here, R is the bitrate cost, which refers to the number of bits required for encoding the compressed image (data rate); λ is a preset weighting parameter used to balance the relationship between the bitrate cost and the distortion cost; and D is the distortion cost, used to measure the difference between the original image and the compressed image.

[0078] The distortion cost can be evaluated using the SSE (Sum of Squared Error) method. The formula for calculating D is D = ∑ i，j (P i，j -O i，j ) 2 Among them, P i，j This represents the compressed pixel value, also known as the reconstructed pixel value, while O i，j This represents the original pixel value, also known as the distorted pixel value. After calculating the bitrate cost and distortion cost, the rate-distortion cost of the current image patch under different candidate prediction methods can be calculated using the rate-distortion cost calculation formula.

[0079] The bitrate cost can be evaluated using a theoretical entropy-based method, and it comprises two parts: the head bits and the residual bits. For example, for the head bits, the head bit cost can be calculated for different candidate prediction schemes. For the four first candidate prediction schemes for a 16x16 image block, the bitrate cost can be calculated using the binary tree specified in the standard and the corresponding token probability table {145, 156, 163, 128}.

[0080] Figure 3 This is a structural diagram of a binary tree according to an embodiment of this application, such as... Figure 3As shown, B_PRED refers to the prediction method where a 16x16 image block is divided into 4x4 image blocks for prediction; DC_PRED indicates the use of DC prediction; V_PRED indicates the use of horizontal prediction; H_PRED indicates the use of vertical prediction; TM_PRED indicates the use of TM prediction; 145 refers to the probability of selecting node B_PRED in the binary tree for prediction; 255-145 correspond to the probability of selecting node 1 in the binary tree but not selecting node B_PRED for prediction; 156 indicates the probability of selecting node 2... The probability of predicting node 10 in the binary tree is represented by 255-156, which corresponds to the probability of choosing node 11 in the binary tree instead of choosing node 10 for prediction. 163 represents the probability of choosing node DC_PRED in the binary tree for prediction. 255-163 corresponds to the probability of choosing node V_PRED instead of choosing node DC_PRED in the binary tree for prediction. 128 represents the probability of choosing node H_PRED in the binary tree for prediction. 255-128 corresponds to the probability of choosing node TM_PRED in the binary tree instead of choosing node H_PRED for prediction.

[0081] Therefore, the head bit cost for different prediction methods can be calculated as follows:

[0082] The header bit cost of DC_PRED = -(log2((255-145) / 255)+log2(156 / 255)+log2(163 / 255)) = 2.57;

[0083] The header bit cost of V_PRED = -(log2((255-145) / 255)+log2(156 / 255)+log2((255-163) / 255)) = 3.39;

[0084] The header bit cost of H_PRED = -(log2((255-145) / 255)+log2((255-156) / 255)+log2(128 / 255)) = 3.57;

[0085] Header bit cost of TM_PRED =

[0086] -(log2((255-145) / 255)+log2((255-156) / 255)+log2((255-128) / 255))=3.58.

[0087] After shifting the above data by 8 bits, the bit rate cost of the head bits for the four prediction methods can be calculated as {657, 915, 869, 915}. Similarly, the bit overhead required for the 10 prediction methods for 4x4 image blocks and for the residual bits of image blocks of different sizes and components can be calculated in the same way.

[0088] This application embodiment calculates the rate-distortion cost under different first candidate prediction methods, and then selects the first target prediction method with the minimum rate-distortion cost from the first candidate prediction methods.

[0089] To select a second target prediction method, it is necessary to calculate the rate-distortion cost of different second candidate prediction methods. Optionally, in the image processing method of this embodiment of the invention, the rate-distortion cost of an image block under the second candidate prediction method is determined by the following method: determining the second code rate value of the image block under the second candidate prediction method based on a token probability table; calculating the reconstructed pixel value and the distorted pixel value of the image block under the second candidate prediction method; calculating the second distortion value based on the reconstructed pixel value and the distorted pixel value of the image block under the second candidate prediction method; and determining the rate-distortion cost of the image block under the second candidate prediction method based on the second code rate value and the second distortion value.

[0090] Specifically, the rate-distortion cost is calculated using the formula: Rate-distortion cost = R*λ + D. It should be noted that when selecting the second target prediction method, the rate-distortion cost is no longer calculated using a predefined token probability table, but rather using a token probability table derived from the pixel values ​​of the image blocks predicted by the first target prediction method. The updated token probability table is then used to calculate the second rate-distortion cost of different second candidate prediction methods, thus yielding the rate-distortion cost evaluated by the updated token probability table.

[0091] In the second traversal process of this application embodiment, the token probability table obtained in the first traversal process is used to calculate the rate-distortion cost for different second candidate prediction methods. That is, the second code rate value of different second candidate prediction methods is calculated based on the token probability table, and then the rate-distortion cost is calculated based on the second code rate value and the second distortion value. This makes the calculation result of the rate-distortion cost for the second candidate prediction method more accurate, ensures that the result of the predicted pixel value of the selected second target prediction method is more accurate, and improves the compression efficiency in the image compression process.

[0092] After determining the second target prediction method, the image block is encoded according to the second target prediction method and the probability table. Optionally, in the image processing method of this embodiment of the invention, encoding the image block based on the second target prediction method and the token probability table to obtain the encoding result includes: predicting the pixel value of the image block based on the second target prediction method, transforming and quantizing the difference between the predicted pixel value and the reference pixel value to obtain the encoding transformation coefficient; and entropy encoding the token of the encoding transformation coefficient based on the token probability table to obtain the encoding result of the image block.

[0093] Specifically, in the second traversal, the predicted pixel values ​​of the image blocks are obtained based on the second target prediction method. Then, the coding transformation coefficients are obtained through steps such as difference value transformation and quantization. The coding transformation coefficients are entropy encoded according to the token probability table to obtain the bitstream of the image blocks, that is, the coding result of the image blocks. The compressed image is obtained based on the bitstream of all image blocks of the compressed image.

[0094] It should be noted that in the WebP encoding process, the image data is ultimately converted into a compressed bitstream, or bitstream, through entropy coding. For example, the original pixel values ​​in the image data are transformed into a more efficient representation. In the WebP lossy compression process, the difference between the predicted pixel values ​​and the reference pixel values ​​is quantized through discrete cosine transform to obtain the coding transformation system corresponding to different image blocks.

[0095] Then, a token probability table is used to calculate the Huffman code or other forms of entropy coding for the transformation system. The WebP coding standard includes three coding methods. The first is direct coding, one of the most direct forms, used to encode data that cannot be efficiently compressed by other methods. Direct coding is used to represent specific coefficient values, which are directly encoded into the final bitstream in binary form. The second is Boolean coding, a special entropy coding method in WebP. The Boolean encoder uses a dynamic probability model to adjust the coding process to optimize compression efficiency. It can be used to handle flag bits, mode selection, and encoding small range values. The key is to maintain a current probability state and dynamically adjust the coding based on the matching of the input token with the current probability. The third is tree coding, an entropy coding method based on a predefined coding tree, used to encode token sets with complex probability distributions. It has some similarities to Huffman coding. In WebP, tree coding is used to encode transform coefficients, where the construction of the coding tree reflects the statistical characteristics of the coefficients, allowing common patterns to be represented with fewer bits.

[0096] The entropy-encoded data is combined with any necessary auxiliary information and synchronization signals and encapsulated into a formatted data stream. In the FPGA hardware implementation, updating the token probability table, performing rate-distortion optimization, and calculating entropy encoding are all done in parallel and pipelined to ensure fast and efficient data processing.

[0097] Finally, bitstream packaging integrates all data and metadata generated during the encoding process into a unified, structured bitstream for easy storage and transmission. For example, WebP format bitstream packaging includes the following parts: frame header, token probability table, segment pattern index, and residual coefficient group, etc. Combining all these components involves placing them in a specific format to ensure the decoder can decompress and reconstruct the original image in the correct order and logic. The final product of bitstream packaging is a file or data stream containing all the compressed image data and the information necessary for image reconstruction. This application embodiment uses a second target prediction method and an updated token probability table to entropy encode image blocks, reducing cross-entropy loss, improving the accuracy of entropy coding, and thus improving image compression efficiency.

[0098] The frame header contains descriptive information about the current frame of the image from the encoder, such as image size (defining the image's width and height), color configuration (describing the color space and subsampling mode used by the image), frame type (e.g., lossy, lossless, or extended), and other control information (including the presence of an alpha channel, timestamps, etc.). The token probability table contains the probability information of token occurrences; in this embodiment, "token" specifically refers to the token of the residual coefficients. The encoding process uses segmentation, and the segment mode index indicates the encoding and quantization parameters of the segment. Segmentation allows the encoder to process different image regions with different quantization parameters and filter intensities, thereby improving compression efficiency and image quality. The residual coefficient set refers to the remaining encoded transform coefficients after quantization and entropy coding. In WebP's lossy compression process, the encoded transform coefficients are quantized to reduce precision and the amount of data required; entropy coding further compresses these quantized coefficients to reduce the actual number of bits.

[0099] It should be noted that the embodiments of this application employ a double-pass traversal method. During the first traversal, the token characteristics of the image (i.e., the frequency of token occurrence in the encoded transform coefficients) are statistically analyzed, and the token probability table is updated. This token probability table from the first traversal is used in the encoding process of the second traversal, thereby improving image compression efficiency. The second traversal of the image further optimizes the compression efficiency based on the selected first target prediction method and quantization parameters. The second traversal includes updating the token probability table, second rate-distortion optimization, and final entropy coding calculation. The second traversal aims to improve the coding efficiency during image compression, ensuring that the output bitstream is minimized while maintaining image quality. Experimental results show that the double-pass image compression method provided in this application embodiment achieves a 4.8%-12.4% improvement in compression efficiency compared to the single-pass compression method in related technologies. When the single-pass traversal implementation uses a predefined token probability table, the compression efficiency is improved by approximately 12.4%; when the single-pass traversal implementation uses a token probability table trained on the dataset, the compression efficiency is improved by approximately 4.8%. In this embodiment, compression efficiency refers to the relative difference in bitrate after image compression, under the background of compressing images of the same quality.

[0100] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0101] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, they can also be implemented by hardware. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the embodiments of this application.

[0102] Example 2

[0103] According to an embodiment of this application, another image processing method is also provided. Figure 4 This is a flowchart of another image processing method according to an embodiment of this application, such as... Figure 4 As shown, the method includes:

[0104] S402, the encoder receives an image sent by the data source, wherein the image includes multiple image blocks;

[0105] Specifically, the encoder can be a hardware encoder for WebP encoding. Hardware encoders utilize dedicated hardware circuitry to implement the standard WebP encoding process, freeing up the CPU / GPU for other tasks and reducing overall power consumption. Hardware encoders demonstrate performance advantages when processing large amounts of image data. Typical applications include cloud computing environments, large websites, or high-traffic content delivery networks, where hardware encoders receive images requiring compression from data sources.

[0106] S404, the encoder is used to traverse the image. In the case of traversing one image block of the image each time, it determines the prediction method of the pixel value of the image block to obtain the first target prediction method. Based on the first target prediction method, it determines the token probability table for encoding the image block. According to the token probability table, it updates the prediction method of the image block to obtain the second target prediction method. Based on the second target prediction method and the token probability table, it encodes the image block to obtain the encoding result. Based on the encoding results of all image blocks of the image, it determines the compressed image.

[0107] Specifically, during the first pass through the image, the image to be compressed is divided into multiple image blocks. For each image block, the prediction methods for the available pixel values ​​are statistically analyzed. Different prediction methods result in different bitrate and distortion costs, leading to different rate-distortion costs. To select the prediction method with the highest compression efficiency (i.e., the lowest rate-distortion cost), the first target prediction method with the lowest rate-distortion cost is selected from multiple candidate prediction methods for each image block. The first target prediction method is used to predict the pixel values ​​in the image block, and the difference between the predicted pixel value and the reference pixel value is calculated and processed to obtain the residual. The residual is then transformed and quantized to obtain coded transform coefficients. These coded transform coefficients are then encoded as tokens, which represent the symbol or item of a coded transform coefficient. For example, for residual transform coefficients, the token can be the level of the residual transform coefficient. Finally, the probability of tokens appearing in the image block is statistically analyzed to generate a token probability table.

[0108] During the second traversal of the image, the token probability table obtained during the first traversal guides the selection of the second target prediction method. The pixel values ​​of the image blocks are predicted according to the second target prediction method. Then, the residuals are calculated using the predicted pixel values ​​obtained by the second target prediction method, and transformed and quantized to obtain the coding transformation coefficients. The coding transformation coefficients are entropy encoded using the token probability table, and the entropy encoding results are packaged to obtain the compressed image.

[0109] S406, the encoder transmits the compressed image to the image receiver, where the image receiver decodes the compressed image to obtain the final image.

[0110] Specifically, after acquiring the compressed image, the encoder transmits the compressed image to the image receiver. The image compression process ends, and the image receiver can decode the compressed image to obtain the final image.

[0111] In this embodiment of the invention, the image processing method described above is used to encode the image through a double-pass encoding method. In the first pass, the encoding setting parameters are determined, including the prediction method of the pixel values ​​of the image block and the token probability table used to encode the image block. In the second pass, the prediction method of the pixel values ​​of the image block is updated according to the token probability table to obtain a second target prediction method with higher compression efficiency. The image block is then encoded according to the second target prediction method and the token probability table to obtain the compressed image, thereby achieving the purpose of improving the compression efficiency of the image. This achieves the technical effect of compensating for the cross-entropy between the predefined token probability table and the token probability table required for actual encoding. In turn, it solves the technical problem that the hardware encoder has low compression performance due to the difference between the predefined token probability table and the token probability table required for actual encoding, which causes the hardware encoder to determine the prediction method of the pixel values ​​based on the predefined token probability table and use the predefined token probability table to entropy encode the encoding transform coefficients in the corresponding prediction mode.

[0112] Example 3

[0113] According to an embodiment of this application, an image processing method based on double traversal is also provided. Figure 5 This is a flowchart of an image processing method based on double traversal according to an embodiment of this application, such as... Figure 5 As shown, the method includes:

[0114] S502 performs the first-level encoding on the image, including intra-frame mode coarse search, rate-distortion optimization, and token probability table statistical calculation.

[0115] Specifically, during the first traversal of the image, first-level encoding is performed. The prediction method for the pixel values ​​of the image blocks in the image is determined by coarse search of the intra-frame mode. Based on rate-distortion optimization, the first target prediction method is selected from the prediction methods. The predicted pixel values ​​of the image blocks are predicted by the first target prediction method. The difference between the predicted pixel values ​​and the parameter pixel values ​​is calculated. The difference values ​​are transformed and quantized into encoding transformation coefficients. The tokens corresponding to the encoding transformation coefficients are determined. Then, the frequency of token occurrence is counted to obtain a token probability table used to encode the encoding transformation coefficients of the image blocks.

[0116] S504 performs second-level encoding on the image, including updating the token probability table, rate-distortion optimization, and entropy encoding calculation.

[0117] Specifically, the second level of encoding is performed during the second traversal. Token probability table updating refers to further quantizing the token probability table obtained during the first traversal to more accurately reflect the statistical characteristics of the image. Through rate-distortion optimization, a second target prediction method is re-selected from the prediction methods based on the token probability table. Then, entropy encoding calculation is performed based on the second target prediction method and the token probability table.

[0118] S506, stream packaging, includes packaging the frame header, token probability table, segment mode index, and residual coefficient group.

[0119] Specifically, the entropy coding calculation results are packaged into a bitstream. For example, the frame header, token probability table, segment pattern index, and residual coefficient group are packaged to obtain the compressed image.

[0120] According to an embodiment of this application, an encoder for implementing the above-described image processing method based on double traversal is also provided. Figure 6 This is a structural diagram of an encoder according to an embodiment of this application, such as... Figure 6 As shown, the encoder implements first-level and second-level coding, where IMD refers to intra-frame mode coarse search, RDO refers to rate-distortion optimization, and E_C refers to entropy coding. The encoder output is a bitstream, including a frame header, token probability table, segment mode index, and residual coefficient set (i.e., ...). Figure 7 (Residuals 0 and 1, etc.).

[0121] In this embodiment of the invention, a first-level encoding is performed on the image, including intra-frame pattern coarse search, rate-distortion optimization, and token probability table statistical calculation. A second-level encoding is performed on the image, including token probability table update, rate-distortion optimization, and entropy coding calculation. The bitstream packaging method includes packaging the frame header, token probability table, segment mode index, and residual coefficient group. It employs a two-pass encoding approach. In the first pass, encoding parameters such as the prediction method for pixel values ​​of image blocks and the token probability table used to encode the image blocks are determined. In the second pass, a second target prediction method with higher compression efficiency is selected from the determined first target prediction methods. The image blocks are then encoded according to the second target prediction method and the token probability table to obtain the compressed image, thus improving image compression efficiency. This compensates for the cross-entropy between the predefined token probability table and the token probability table required for actual encoding. Furthermore, it solves the technical problem of low compression performance caused by the hardware encoder determining the pixel value prediction method based on the predefined token probability table and using the predefined token probability table for entropy encoding of the encoding transform coefficients under the corresponding prediction mode due to the difference between the predefined and actual token probability tables.

[0122] Example 4

[0123] According to an embodiment of this application, a streaming media server for implementing the above-described image processing method is also provided. Figure 7 This is a structural diagram of a streaming media server according to an embodiment of this application, such as... Figure 7 As shown, the streaming media server includes:

[0124] The hardware encoder 701 is used to receive video files and perform any of the above image processing methods on the frame images in the video files to obtain a bitstream.

[0125] Specifically, the hardware encoder 701 can be a hardware encoder for WebP encoding. The hardware encoder uses dedicated hardware circuitry to implement the standard WebP encoding process. The hardware encoder 701 receives the video file to be compressed and performs the image processing method described above on the frame images in the video file. Through a double-pass encoding method, in the first pass, the encoding setting parameters, such as the prediction method of the pixel values ​​of the image block and the token probability table used to encode the image block, are determined. In the second pass, a second target prediction method with higher compression efficiency is selected from the determined first target prediction method, and the image block is encoded according to the second target prediction method and the token probability expression to obtain the compressed image, i.e., the bitstream.

[0126] Storage system 702 is used to store video files and bitstreams in compressed images.

[0127] Specifically, the storage system 702 can be a database or server set up in the streaming media server. After the hardware encoder 701 compresses the video file, it stores the compression result, that is, the video file and the bitstream in the compressed image.

[0128] The streaming media processing engine 703 is used to transmit the bitstream over the network to the user terminal, where the user terminal is used to decode the bitstream and display the decoded frame images.

[0129] Specifically, the streaming media processing engine 703 can be a module that establishes a network communication connection with the user terminal. It is responsible for transmitting the compressed result of the video file, i.e., the bitstream, to the user terminal. After receiving the bitstream, the user terminal decodes it to obtain the video file.

[0130] In this embodiment of the invention, a hardware encoder 701 receives a video file, performs any of the above-mentioned image processing methods on the frame images in the video file to obtain a bitstream, a storage system 702 stores the video file and the bitstream in the compressed image, and a streaming media processing engine 703 transmits the bitstream to the user terminal via a network. The user terminal is used to decode the bitstream and display the decoded frame images. The hardware encoder 701 compresses the video file, the storage system 702 stores the compressed bitstream, and the streaming media processing engine 703 sends the compression result of the video file to the user terminal. This achieves the goal of improving the compression efficiency of the video file, thereby compensating for the cross-entropy between the predefined token probability table and the token probability table required for actual encoding. This solves the technical problem that due to the difference between the predefined token probability table and the token probability table required for actual encoding, the hardware encoder determines the prediction method of pixel values ​​based on the predefined token probability table and performs entropy encoding on the encoding transform coefficients in the corresponding prediction mode using the predefined token probability table, resulting in low compression performance of the hardware encoder.

[0131] Example 5

[0132] According to an embodiment of this application, an image processing system for implementing the above-described image processing method is also provided, comprising: a processor; and a memory connected to the processor, configured to provide the processor with instructions for processing the following processing steps: traversing an image; determining a prediction method for the pixel values ​​of an image block each time an image block is traversed, to obtain a first target prediction method; determining a token probability table for encoding the image block based on the first target prediction method, wherein the token probability table includes the probability of tokens encoding transform coefficients; updating the prediction method of the image block according to the token probability table, to obtain a second target prediction method; encoding the image block based on the second target prediction method and the token probability table, to obtain an encoding result; and determining a compressed image based on the encoding results of all image blocks of the image.

[0133] The memory can be used to store software programs and modules, such as the program instructions / modules corresponding to the data acquisition method and device in the embodiments of this application. The processor executes different functional applications and data processing by running the software programs and modules stored in the memory, thereby realizing the above-mentioned data acquisition method.

[0134] Example 6

[0135] According to embodiments of this application, an image processing apparatus for implementing the above-described image processing method is also provided. Figure 8 This is a structural diagram of an image processing apparatus according to an embodiment of this application, such as... Figure 8 As shown, the device includes:

[0136] The first determining unit 801 is used to traverse the image and, in the case of traversing an image block at each time, determine the prediction method of the pixel value of the image block to obtain the first target prediction method.

[0137] The second determining unit 802 is used to determine a token probability table for encoding the image block based on the first target prediction method, wherein the token probability table includes the probability of tokens encoding transform coefficients;

[0138] The third determining unit 803 is used to update the prediction method of the image block according to the token probability table to obtain the second target prediction method;

[0139] The encoding unit 804 is used to encode the image block based on the second target prediction method and the token probability table to obtain the encoding result;

[0140] Compression unit 805 is used to determine the compressed image based on the encoding results of all image blocks of the image.

[0141] Optionally, in the image processing apparatus of this embodiment, the second determining unit 802 includes: a first prediction module, configured to predict the pixel values ​​of an image block based on a first target prediction method to obtain predicted pixel values; an acquisition module, configured to acquire reference pixel values ​​of the image block and process the difference between the predicted pixel values ​​and the reference pixel values ​​into encoded transform coefficients; a probability table determination module, configured to determine an initial token probability table based on the occurrence frequency of tokens in the encoded transform coefficients; a first calculation module, configured to calculate the ratio of the probability of tokens in the encoded transform coefficients to the total number of tokens in the initial token probability table; a quantization processing module, configured to quantize the ratio of the probability of tokens in the encoded transform coefficients to the total number of tokens to obtain a quantized value; and an update module, configured to update the probabilities of tokens with different encoded transform coefficients in the initial token probability table to the quantized values ​​to obtain a token probability table.

[0142] Optionally, in the image processing apparatus of this embodiment of the invention, the first determining unit 801 includes: a second calculation module, configured to determine an initial prediction method set for an image patch and calculate the predicted pixel value of the image patch under different initial prediction methods in the initial prediction method set; a third calculation module, configured to obtain a reference pixel value of the image patch and calculate the difference between the predicted pixel value and the reference pixel value of the image patch under different initial prediction methods; a conversion module, configured to convert the initial prediction method set into an initial prediction method sequence according to the order of the difference values ​​of the image patch under different initial prediction methods from smallest to largest; a first filtering module, configured to filter a first preset number of first candidate prediction methods from the initial prediction method sequence to obtain a first candidate prediction method sequence; and a selection module, configured to select the prediction method with the lowest rate-distortion cost from the first candidate prediction method sequence to obtain a first target prediction method.

[0143] Optionally, in the image processing apparatus of this embodiment of the invention, the third determining unit 803 includes: a second filtering module, configured to filter N second candidate prediction methods from the first candidate prediction method sequence to obtain a set of second candidate prediction methods, wherein the set of second candidate prediction methods includes a first target prediction method, N is a positive integer, and N is less than the number of all prediction methods in the first candidate prediction method sequence; and a first determining module, configured to determine the rate-distortion cost of the image block under the second candidate prediction method according to the token probability table, and select the second candidate prediction method with the minimum rate-distortion cost from the set of second candidate prediction methods to obtain a second target prediction method.

[0144] Optionally, in the image processing apparatus of this embodiment, the second filtering module includes: a first acquisition submodule, configured to, for a first luminance component, if the first target prediction method under the first luminance component is the same as the Nth prediction method in the sequence of first candidate prediction methods under the first luminance component, acquire the first N first candidate prediction methods in the sequence of first candidate prediction methods under the first luminance component to obtain a set of second candidate prediction methods under the first luminance component; and if the first target prediction method under the first luminance component is different from the Nth prediction method in the sequence of first candidate prediction methods under the first luminance component, acquire the N first candidate prediction methods other than the Nth prediction method in the sequence of first candidate prediction methods under the first luminance component to obtain a set of second candidate prediction methods under the first luminance component; and a second acquisition submodule, configured to, for a chrominance component, if the first target prediction method under the chrominance component is different from the Nth prediction method in the sequence of first candidate prediction methods under the first luminance component, acquire the first N first candidate prediction methods .... When the Nth prediction method is the same, the first N first candidate prediction methods in the first candidate prediction method sequence under the chroma component are obtained to obtain the second candidate prediction method set under the chroma component. When the first target prediction method under the chroma component is different from the Nth prediction method in the first candidate prediction method sequence under the chroma component, the N first candidate prediction methods other than the Nth prediction method in the first candidate prediction method sequence under the chroma component are obtained to obtain the second candidate prediction method set under the chroma component. The first determining submodule is used to determine the second candidate prediction method set under the second luminance component based on the prediction methods in the first candidate prediction method sequence under the second luminance component. The second determining submodule is used to determine the second candidate prediction method set of the image block based on the second candidate prediction method set under the first luminance component, the second candidate prediction method set under the chroma component, and the second candidate prediction method set under the second luminance component.

[0145] Optionally, in the image processing apparatus of this embodiment of the invention, the rate-distortion cost of an image block under the first candidate prediction method is determined by: determining the first code rate value of the image block under the first candidate prediction method based on a preset token probability table; calculating the first distortion value based on the reconstructed pixel value and the distorted pixel value of the image block under the first candidate prediction method; calculating the reconstructed pixel value and the distorted pixel value of the image block under the first candidate prediction method; and determining the rate-distortion cost of the image block under the first candidate prediction method based on the first code rate value and the first distortion value.

[0146] Optionally, in the image processing apparatus of this embodiment of the invention, the rate-distortion cost of an image block under the second candidate prediction method is determined by: determining the second code rate value of the image block under the second candidate prediction method based on a token probability table; calculating the reconstructed pixel value and the distorted pixel value of the image block under the second candidate prediction method; calculating the second distortion value based on the reconstructed pixel value and the distorted pixel value of the image block under the second candidate prediction method; and determining the rate-distortion cost of the image block under the second candidate prediction method based on the second code rate value and the second distortion value.

[0147] Optionally, in the image processing apparatus of this embodiment of the invention, the compression unit 804 includes: a second prediction module, configured to predict the pixel values ​​of the image block based on a second target prediction method, and transform and quantize the difference between the predicted pixel values ​​and the reference pixel values ​​to obtain coded transformation coefficients; and an encoding module, configured to entropy encode the tokens of the coded transformation coefficients based on a token probability table to obtain the encoding result of the image block.

[0148] It should be noted that the first determining unit 801, the second determining unit 802, the third determining unit 803, the encoding unit 804 and the compression unit 805 mentioned above correspond to steps S402 to S408 in Embodiment 1. The two modules and the corresponding steps implement the same instances and application scenarios, but are not limited to the content disclosed in Embodiment 1.

[0149] Example 7

[0150] Embodiments of this application may provide a computer terminal, which may be any computer terminal device in a group of computer terminals. Optionally, in this embodiment, the aforementioned computer terminal may also be replaced by a mobile terminal or other terminal device.

[0151] Optionally, in this embodiment, the computer terminal may be located in at least one of multiple network devices in a computer network. In this embodiment, the computer terminal may execute the following method steps: traversing the image, and in each traversal of an image block, determining a prediction method for the pixel values ​​of the image block to obtain a first target prediction method; determining a token probability table for encoding the image block based on the first target prediction method, wherein the token probability table includes the probability of tokens encoding transform coefficients; updating the prediction method of the image block according to the token probability table to obtain a second target prediction method; encoding the image block based on the second target prediction method and the token probability table to obtain an encoding result; and determining the compressed image based on the encoding results of all image blocks of the image.

[0152] Optionally, Figure 9 This is a structural block diagram of a computer terminal according to an embodiment of this application. Figure 9As shown, the computer terminal A may include: one or more ( Figure 9 (Only one is shown) processor 102, memory 104, memory controller, and peripheral interface, wherein the peripheral interface is connected to the radio frequency module, audio module and display.

[0153] The memory can be used to store software programs and modules, such as the program instructions / modules corresponding to the image processing method and apparatus in this application embodiment. The processor executes different functional applications and data processing by running the software programs and modules stored in the memory, thereby realizing the aforementioned image processing method. The memory may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include memory remotely located relative to the processor, and these remote memories can be connected to terminal A via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0154] The processor can invoke information and application programs stored in memory via a transmission device to execute a program with the following steps: traversing the image, determining the prediction method for the pixel values ​​of the image block in each traversal of the image block, and obtaining a first target prediction method; determining a token probability table for encoding the image block based on the first target prediction method, wherein the token probability table includes the probability of tokens encoding transform coefficients; updating the prediction method of the image block according to the token probability table, and obtaining a second target prediction method; encoding the image block based on the second target prediction method and the token probability table, and obtaining an encoding result; and determining the compressed image based on the encoding results of all image blocks of the image.

[0155] Those skilled in the art will understand that Figure 9 The structure shown is for illustrative purposes only. The computer terminal can also be a smartphone, tablet, handheld computer, mobile internet device (MID), PAD, or other terminal device.

[0156] Example 8

[0157] This application also provides a computer program product, including a non-volatile computer-readable storage medium storing a computer program. When executed by a processor, the computer program has the following method steps: traversing an image; determining a prediction method for the pixel values ​​of an image block at each traversal to obtain a first target prediction method; determining a token probability table for encoding the image block based on the first target prediction method, wherein the token probability table includes the probability of tokens encoding transform coefficients; updating the prediction method of the image block according to the token probability table to obtain a second target prediction method; encoding the image block based on the second target prediction method and the token probability table to obtain an encoding result; and determining a compressed image based on the encoding results of all image blocks of the image.

[0158] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0159] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that the flowchart illustrations and / or blocks, and combinations thereof, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0160] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0161] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0162] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0163] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0164] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0165] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0166] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. An image processing method, characterized in that, include: Traverse the image, and in each instance of traversing an image block, determine the prediction method for the pixel values ​​of that image block to obtain the first target prediction method; A token probability table for encoding the image patch is determined based on the first target prediction method, wherein the token probability table includes the probability of tokens encoding transform coefficients; The prediction method for the image patch is updated according to the token probability table to obtain the second target prediction method; The image block is encoded based on the second target prediction method and the token probability table to obtain the encoding result; The compressed image is determined based on the encoding results of all image blocks of the image.

2. The method according to claim 1, characterized in that, The token probability table for encoding the image patch is determined based on the first target prediction method, including: The pixel values ​​of the image block are predicted based on the first target prediction method to obtain the predicted pixel values; Obtain the reference pixel value of the image block, and process the difference between the predicted pixel value and the reference pixel value into coded transform coefficients; The initial token probability table is determined based on the frequency of token occurrence according to the encoding transformation coefficients; Calculate the ratio of the probability of the token with the encoded transformation coefficient in the initial token probability table to the total number of tokens; The ratio of the probability of a token in the encoded transformation coefficient to the total number of tokens is quantized to obtain a quantized value. The token probability table is obtained by updating the probability of tokens with different encoding transformation coefficients in the initial token probability table to the quantized values.

3. The method according to claim 1, characterized in that, The prediction method for determining the pixel values ​​of the image block, and the method for obtaining the first target prediction method, includes: Determine the initial prediction scheme set for the image block, and calculate the predicted pixel value of the image block under different initial prediction schemes in the initial prediction scheme set; Obtain the reference pixel value of the image block, and calculate the difference between the predicted pixel value and the reference pixel value of the image block under different initial prediction methods; Based on the order of the difference values ​​of the image patches under different initial prediction methods from smallest to largest, the set of initial prediction methods is converted into a sequence of initial prediction methods; A first candidate prediction method sequence is obtained by filtering a preset number of first candidate prediction methods from the initial prediction method sequence; The prediction method with the lowest rate-distortion cost is selected from the first candidate prediction method sequence to obtain the first target prediction method.

4. The method according to claim 3, characterized in that, The prediction method for the image patch is updated according to the token probability table to obtain the second target prediction method, which includes: N second candidate prediction methods are selected from the first candidate prediction method sequence to obtain a second candidate prediction method set, wherein the second candidate prediction method set includes the first target prediction method, N is a positive integer, and N is less than the number of prediction methods in the first candidate prediction method sequence; The rate-distortion cost of the image block under the second candidate prediction method is determined according to the token probability table, and the second candidate prediction method with the minimum rate-distortion cost is selected from the set of the second candidate prediction methods to obtain the second target prediction method.

5. The method according to claim 4, characterized in that, The image patch includes a first luminance component, a chrominance component, and a second luminance component in descending order of scale. The first candidate prediction method set includes a first candidate prediction method set under the first luminance component, a first candidate prediction method set under the chrominance component, and a first candidate prediction method set under the second luminance component. The second candidate prediction method set includes a second candidate prediction method set under the first luminance component, a second candidate prediction method set under the chrominance component, and a second candidate prediction method set under the second luminance component. N second candidate prediction methods are selected from the first candidate prediction method sequence to obtain the second candidate prediction method set, which includes: For the first luminance component, if the first target prediction method under the first luminance component is the same as the Nth prediction method in the first candidate prediction method sequence under the first luminance component, the first N first candidate prediction methods in the first candidate prediction method sequence under the first luminance component are obtained to obtain the second candidate prediction method set under the first luminance component. If the first target prediction method under the first luminance component is different from the Nth prediction method in the first candidate prediction method sequence under the first luminance component, the N first candidate prediction methods other than the Nth prediction method in the first candidate prediction method sequence under the first luminance component are obtained to obtain the second candidate prediction method set under the first luminance component. For the chromaticity component, if the first target prediction method under the chromaticity component is the same as the Nth prediction method in the first candidate prediction method sequence under the chromaticity component, the first N first candidate prediction methods in the first candidate prediction method sequence under the chromaticity component are obtained to obtain the second candidate prediction method set under the chromaticity component. If the first target prediction method under the chromaticity component is different from the Nth prediction method in the first candidate prediction method sequence under the chromaticity component, the N first candidate prediction methods other than the Nth prediction method in the first candidate prediction method sequence under the chromaticity component are obtained to obtain the second candidate prediction method set under the chromaticity component. For the second brightness component, the set of second candidate prediction methods for the second brightness component is determined according to the prediction methods in the first candidate prediction method sequence for the second brightness component. The second candidate prediction method set for the image block is determined based on the second candidate prediction method set under the first luminance component, the second candidate prediction method set under the chrominance component, and the second candidate prediction method set under the second luminance component.

6. The method according to claim 3, characterized in that, The rate-distortion cost of the image patch under the first candidate prediction method is determined in the following way: The first bitrate value of the image block under the first candidate prediction method is determined based on a preset token probability table. Calculate the reconstructed pixel value and the distorted pixel value of the image patch under the first candidate prediction method; Calculate the first distortion value based on the reconstructed pixel value and the distorted pixel value of the image block under the first candidate prediction method; The rate-distortion cost of the image block under the first candidate prediction method is determined based on the first rate cost and the first distortion cost.

7. The method according to claim 4, characterized in that, The rate-distortion cost of the image patch under the second candidate prediction method is determined in the following way: The second code rate value of the image block under the second candidate prediction method is determined based on the token probability table; Calculate the reconstructed pixel value and the distorted pixel value of the image patch under the second candidate prediction method; The second distortion value is calculated based on the reconstructed pixel value and the distorted pixel value of the image block under the second candidate prediction method; The rate-distortion cost of the image block under the second candidate prediction method is determined based on the second rate cost and the second distortion cost.

8. The method according to claim 1, characterized in that, The image patch is encoded based on the second target prediction method and the token probability table to obtain the following encoding results: The pixel values ​​of the image block are predicted based on the second target prediction method. The difference between the predicted pixel values ​​and the reference pixel values ​​is transformed and quantized to obtain the coding transformation coefficients. Entropy encoding is performed on the tokens of the encoding transform coefficients based on the token probability table to obtain the encoding result of the image block.

9. An image processing method, characterized in that, include: The encoder receives an image sent by a data source, wherein the image includes multiple image blocks; The encoder is used to traverse the image. In each traversal of an image block of the image, it determines the prediction method of the pixel value of the image block to obtain a first target prediction method. Based on the first target prediction method, it determines a token probability table for encoding the image block. Based on the token probability table, it updates the prediction method of the image block to obtain a second target prediction method. Based on the second target prediction method and the token probability table, it encodes the image block to obtain an encoding result. Based on the encoding results of all image blocks of the image, it determines the compressed image. The encoder transmits the compressed image to the image receiver, wherein the image receiver decodes the compressed image to obtain the image.

10. A streaming media server, characterized in that, include: A hardware encoder is used to receive a video file and perform the image processing method according to any one of claims 1 to 8 on the frame images in the video file to obtain a bitstream; A storage system for storing the video file and the bitstream from the compressed image; A streaming media processing engine is used to transmit the bitstream to a user terminal via a network, wherein the user terminal is used to decode the bitstream and display the decoded frame image.

11. An image processing apparatus, characterized in that, include: The first determining unit is used to traverse the image and, in the case of traversing an image block of the image each time, determine the prediction method of the pixel value of the image block to obtain the first target prediction method. The second determining unit is configured to determine a token probability table for encoding the image block based on the first target prediction method, wherein the token probability table includes the probability of tokens encoding transform coefficients; The third determining unit is used to update the prediction method of the image block according to the token probability table to obtain the second target prediction method; The encoding unit is used to encode the image block based on the second target prediction method and the token probability table to obtain the encoding result; A compression unit is used to determine the compressed image based on the encoding results of all image blocks of the image.

12. A computer terminal, characterized in that, include: Memory, which stores executable programs; A processor for running the program, wherein the program, when running, performs the method according to any one of claims 1 to 9.

13. A computer program product, characterized in that, The method includes a non-volatile computer-readable storage medium storing a computer program that, when executed by a processor, implements the method of any one of claims 1 to 9.

14. An image processing system, characterized in that, include: processor; A memory, connected to the processor, is used to provide the processor with instructions to perform the following processing steps: Traverse the image, and in each instance of traversing an image block, determine the prediction method for the pixel values ​​of that image block to obtain the first target prediction method; A token probability table for encoding the image patch is determined based on the first target prediction method, wherein the token probability table includes the probability of tokens encoding transform coefficients; The prediction method for the image patch is updated according to the token probability table to obtain the second target prediction method; The image block is encoded based on the second target prediction method and the token probability table to obtain the encoding result; The compressed image is determined based on the encoding results of all image blocks of the image.

Images