Encoding method, decoding method, device, storage medium and program product
By adjusting the arrangement order of subpixels and using the reconstructed subpixels in the same channel as a reference for prediction, the problem of difficulty in balancing prediction accuracy and timing in hardware implementation is solved, achieving higher compression efficiency and better prediction accuracy.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- HUNAN GOKE MICROELECTRONICS CO LTD
- Filing Date
- 2026-04-21
- Publication Date
- 2026-05-19
AI Technical Summary
Existing image compression algorithms based on predictive coding suffer from low compression efficiency because they are difficult to balance prediction accuracy and hardware processing timing when implemented in hardware.
By rearranging the order of multiple consecutive sub-pixels to arrange consecutive sub-pixels in the same channel at intervals, and using the reconstructed sub-pixels of the surrounding sub-pixels in the same channel as a reference for prediction, a bitstream is generated.
It improves prediction accuracy and compression efficiency, reduces the difficulty of hardware implementation, and optimizes hardware processing timing.
Smart Images
Figure CN122069355A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of data compression technology, and in particular to an encoding method, a decoding method, an apparatus, a storage medium, and a program product. Background Technology
[0002] For common multimedia processing chips, such as security chips and display chips, bandwidth and memory are two very important indicators, which are related to the chip's performance and usage cost, and need to be carefully considered.
[0003] Image compression technology mainly compresses the original image data into a smaller amount of data, thereby saving bandwidth and reducing memory usage. At the same time, it can also reduce the power consumption of the system accessing external DDR (Double Data Rate Synchronous Dynamic Random Access Memory), making it an indispensable technology for multimedia processing chips.
[0004] The aforementioned image compression techniques are divided into lossy compression and lossless compression. Lossy compression has a wider range of applications. However, due to limitations in chip area, performance, and power consumption, such image compression algorithms should not be too complex. A balance needs to be struck between compression efficiency and hardware implementation cost, striving to achieve higher compression efficiency under the premise of the same area and performance.
[0005] Among numerous image compression algorithms, those based on predictive coding are highly effective. Lossy compression algorithms based on predictive coding generally consist of three steps: Step 1 involves predicting the pixel value of the sub-pixels to be compressed, thus reducing spatial redundancy. The predicted pixel value is then subtracted from the original pixel value to obtain the prediction error. Step 2 involves quantizing the prediction error to further reduce redundancy. Step 3 involves entropy coding the quantized prediction error to generate a bitstream.
[0006] In hardware implementation, the first prediction stage typically utilizes the local correlation of neighboring pixels, using the reconstructed subpixels adjacent to the current subpixel as a reference for prediction. However, this becomes difficult to implement if the reconstructed subpixels immediately adjacent to the current subpixel have not yet been reconstructed. If a similar scheme is adopted, such as the prediction scheme in the DSC (Display Stream Compression) and AVS (Digital Audio and Video Codec Standard) perceptual compression algorithms, prediction accuracy is lost, and compression efficiency is reduced. Summary of the Invention
[0007] This invention provides an encoding method, a decoding method, a device, a storage medium, and a program product, which provide a pixel prediction scheme that balances prediction accuracy and hardware processing timing.
[0008] This invention provides an encoding method, comprising the following steps: Obtain multiple consecutive sub-pixels; The arrangement order of multiple consecutive sub-pixels is readjusted so that the consecutive sub-pixels in the same channel are arranged with intervals after the adjustment; With reference to the reconstructed subpixels of the subpixels in the same channel surrounding each adjusted subpixel, prediction is performed on each subpixel in turn to obtain the predicted value of each subpixel. The bitstream is obtained by encoding the predicted value of each sub-pixel.
[0009] Optionally, when multiple consecutive sub-pixels belong to multi-channel sub-pixels, the adjusted consecutive sub-pixels in the same channel are spaced apart by at least one sub-pixel from another channel.
[0010] Optionally, the step of readjusting the arrangement order of multiple consecutive sub-pixels so that the adjusted consecutive sub-pixels in the same channel are arranged at intervals includes: When multiple consecutive sub-pixels belong to a single channel, the multiple consecutive sub-pixels are divided into multiple segments, and sub-pixels with the same index position are extracted from each segment and rearranged and spliced.
[0011] Optionally, the reconstructed subpixels of the subpixels in the same channel surrounding the adjusted subpixel are the reconstructed subpixels of several subpixels from the same channel to the left / left and previous row of the current subpixel being predicted.
[0012] Optionally, when the current sub-pixel does not meet the upper reference row disable condition, the reconstructed sub-pixel of the surrounding sub-pixels in the same channel is the reconstructed sub-pixel of several sub-pixels from the same channel to the left and in the previous row of the current sub-pixel being predicted. When the current sub-pixel meets the upper reference row disable condition, the reconstructed sub-pixel of the surrounding sub-pixels in the same channel is the reconstructed sub-pixel of several sub-pixels from the same channel to the left of the current sub-pixel being predicted.
[0013] Optionally, after acquiring multiple consecutive sub-pixels, the method further includes: When multiple consecutive sub-pixels have met the channel interleaving condition, each sub-pixel is predicted with reference to a reconstructed sub-pixel that is at least one sub-pixel apart from each sub-pixel. If multiple consecutive sub-pixels do not meet the channel interleaving condition, the following step is performed: readjust the arrangement order of the multiple consecutive sub-pixels so that the adjusted consecutive sub-pixels in the same channel are arranged with intervals.
[0014] This invention provides a decoding method, comprising the following steps: Obtain the bitstream, which is obtained by the encoding method described above; The bitstream is decoded to obtain multiple pixel data; The arrangement order of multiple pixel data is adjusted according to an inverse adjustment rule, which is the opposite of the adjustment method for multiple consecutive sub-pixels in the above encoding method. The adjusted pixel data is combined into an image.
[0015] The present invention also provides an encoding device, comprising the following modules: The acquisition module is used to acquire multiple consecutive sub-pixels; An adjustment module is used to readjust the arrangement order of multiple consecutive sub-pixels so that the consecutive sub-pixels in the same channel are arranged at intervals after adjustment. The prediction module is used to predict each sub-pixel sequentially, taking the reconstructed sub-pixels of the sub-pixels in the same channel around each of the adjusted sub-pixels as a reference, to obtain the predicted value of each sub-pixel. An encoding module is used to encode the predicted value of each sub-pixel to obtain a bitstream.
[0016] The present invention also provides a decoding device, comprising the following modules: An acquisition module is used to acquire a bitstream, which is obtained by the encoding method described above; A decoding module is used to decode the bitstream to obtain multiple pixel data; An adjustment module is used to adjust the arrangement order of multiple pixel data according to an inverse adjustment rule, which is the opposite of the adjustment method for multiple consecutive sub-pixels in the above-described encoding method. The combination module is used to combine adjusted pixel data into an image.
[0017] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement any of the above-described encoding methods or the above-described decoding methods.
[0018] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-described encoding methods or decoding methods.
[0019] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements any of the above-described encoding methods or decoding methods.
[0020] The encoding method, decoding method, device, storage medium, and program product provided by this invention involve: acquiring multiple consecutive sub-pixels; readjusting the arrangement order of the multiple consecutive sub-pixels so that the adjusted consecutive sub-pixels in the same channel are arranged at intervals; using the reconstructed sub-pixels of the sub-pixels in the same channel surrounding each adjusted sub-pixel as a reference, predicting each sub-pixel sequentially to obtain the predicted value of each sub-pixel; and encoding based on the predicted value of each sub-pixel to obtain a bitstream.
[0021] In this method, when predicting the corresponding predicted value for each sub-pixel, the reconstructed sub-pixels of the surrounding sub-pixels in the same channel are used as a reference. Since the adjusted, consecutive sub-pixels in the same channel are arranged with intervals, it can be understood that for each sub-pixel, there is at least one sub-pixel between it and the reference surrounding consecutive sub-pixels in the same channel. Therefore, when each sub-pixel is predicted, the reference reconstructed sub-pixels have sufficient time to reconstruct. This sub-pixel adjustment and prediction method reduces the hardware implementation difficulty of lossy image compression methods based on predictive coding. Furthermore, compared to similar prediction schemes used in DSC or AVS perceptual lossless algorithms, this method suffers less loss of prediction accuracy, more closely approximates the ideal image prediction reference relationship, and has higher compression efficiency. In summary, this scheme provides a pixel prediction method that balances prediction accuracy (compression efficiency) and hardware processing timing. Attached Figure Description
[0022] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.
[0023] Figure 1 This is a schematic diagram of pixel prediction based on DPCM in an embodiment of the method of the present invention.
[0024] Figure 2 This is a schematic diagram of sub-pixel prediction in the DSC standard in the embodiment of the method of the present invention.
[0025] Figure 3 This is a schematic diagram of sub-pixel prediction in the AVS-sensory lossless compression standard in an embodiment of the method of the present invention.
[0026] Figure 4 This is a schematic diagram of an optional process for the encoding method provided by the present invention.
[0027] Figure 5 This is one of the pixel prediction diagrams in the encoding method provided by the present invention.
[0028] Figure 6 This is the second schematic diagram of pixel prediction in the encoding method provided by the present invention.
[0029] Figure 7 This is the third schematic diagram of pixel prediction in the encoding method provided by the present invention.
[0030] Figure 8 This is the fourth schematic diagram of pixel prediction in the encoding method provided by the present invention.
[0031] Figure 9 This is the fifth schematic diagram of pixel prediction in the encoding method provided by the present invention.
[0032] Figure 10 This is the Raw RGGB diagram involved in the encoding method provided by this invention.
[0033] Figure 11 This is the sixth schematic diagram of pixel prediction for the encoding method provided by the present invention.
[0034] Figure 12 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation
[0035] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.
[0036] As stated above, lossy compression algorithms based on predictive coding often face significant timing pressures during hardware implementation. For example, please refer to... Figure 1 The diagram illustrates a prediction method based on DPCM (Differential Pulse Code Modulation), which utilizes the local correlation between neighboring pixels to predict the current sub-pixel. Figure 1 In the current sub-pixel, x3, pixel prediction utilizes the reconstructed sub-pixel at its immediate left neighbor, x2. However, the reconstructed sub-pixel at position x2 may not have been reconstructed yet, making it difficult to implement at the hardware level. Therefore, other hardware prediction methods must be used, representing a compromise between hardware feasibility and compression efficiency.
[0037] See Figure 2 , Figure 2This is an optional example of subpixel prediction in the DSC standard. In this scheme, x3 to x5 are grouped into a basic unit, and the predicted values of all positions within this basic unit are obtained by referring to the reconstructed subpixel of the last position in the previous basic unit, namely the reconstructed subpixel of x2.
[0038] See Figure 3 , Figure 3 This is an optional example of subpixel prediction in the AVS-aware lossless compression standard. In this example, when predicting subpixels x0 / x2 / x4 / x6 (i.e., the green quadrilaterals), the reconstructed subpixels y0 / y2 / y4 / y6 (i.e., the yellow quadrilaterals) corresponding to the subpixels in the row above are used as references. After obtaining the reconstructed pixel (value) of subpixels x0 / x2 / x4 / x6, the average value of the reconstructed subpixels of the left and right adjacent subpixels of subpixels x1 / x3 / x5 can be used as the predicted value of subpixels x1 / x3 / x5 (i.e., the blue quadrilaterals).
[0039] However, the aforementioned prediction methods based on DSC or AVS often sacrifice prediction accuracy and reduce compression efficiency.
[0040] To address at least one of the aforementioned technical problems, this application provides a pixel prediction scheme that balances prediction accuracy (compression efficiency) and hardware processing timing. Specifically, it provides an encoding method, a decoding method, a device, a storage medium, and a program product. Before pixel prediction, the arrangement order of multiple consecutive sub-pixels is adjusted so that the adjusted consecutive sub-pixels in the same channel are spaced apart. Therefore, during pixel prediction, each sub-pixel can reference the reconstructed sub-pixels of its surrounding sub-pixels in the same channel, and there is an interval between the referenced consecutive sub-pixels in the same channel and the sub-pixel being predicted. This allows sufficient time for hardware to reconstruct the pixel, and the reconstructed sub-pixels of the referenced consecutive sub-pixels in the same channel are always adjacent to the surrounding sub-pixels in the same channel (i.e., reconstructed sub-pixels of the sub-pixels adjacent to the predicted pixel before adjustment). Compared to similar prediction methods used in DSC and AVS algorithms, this improves prediction accuracy and compression efficiency. This solves the technical problem of balancing hardware processing timing and prediction accuracy in related technologies.
[0041] The following describes an optional embodiment of an encoding method provided in this application.
[0042] Figure 4 This is a schematic diagram of an optional flowchart of the encoding method provided by the present invention, such as... Figure 4 As shown, the method includes the following: S101, Obtain multiple consecutive sub-pixels; S102. Rearrange the order of multiple consecutive sub-pixels so that the consecutive sub-pixels in the same channel are arranged at intervals after the adjustment. S103, with reference to the reconstructed sub-pixels of the sub-pixels in the same channel surrounding each of the adjusted sub-pixels, predict each sub-pixel in turn to obtain the predicted value of each sub-pixel. S104, Encode the predicted value of each sub-pixel to obtain a bitstream.
[0043] In this embodiment, when predicting each sub-pixel to obtain the corresponding predicted value, the reconstructed sub-pixels of the surrounding sub-pixels in the same channel are used as a reference. Since the adjusted, consecutive sub-pixels in the same channel are arranged with intervals, it can be understood that for each sub-pixel, there is at least one sub-pixel between it and the reference surrounding consecutive sub-pixels in the same channel. Therefore, when each sub-pixel is predicted, the reference reconstructed sub-pixels have sufficient time to be reconstructed. This sub-pixel adjustment and prediction method reduces the hardware implementation difficulty of coding methods based on predictive coding principles. Furthermore, compared to similar prediction methods used in DSC and AVS algorithms, this pixel prediction method suffers less loss in prediction accuracy and has higher compression efficiency. In summary, this solution provides a pixel prediction scheme that balances prediction accuracy and hardware processing timing.
[0044] It should be noted that the above pixel prediction / encoding scheme can be applied to chips or electronic devices equipped with chips, such as multimedia processing chips. This process is a step in a lossy image compression algorithm, which can be a line-by-line compression or frame-by-frame lossy image compression algorithm.
[0045] When compressing an image, multiple consecutive sub-pixels to be compressed can be obtained from the image. For example, during line-by-line compression, a segment of consecutive sub-pixels can be selected from the current line of the image for pixel prediction. Optionally, this segment of consecutive sub-pixels can be used as sample points to form a basic prediction unit, and subsequent pixel prediction can be performed on each sample point in the basic prediction unit.
[0046] After obtaining the predicted value of each sub-pixel, the prediction error can be obtained by subtracting the predicted value from the original pixel value of each sub-pixel. The prediction error is then quantized and entropy encoded to generate a bitstream.
[0047] After multiple consecutive sub-pixels have completed pixel prediction, quantization, and entropy coding, multiple new consecutive sub-pixels can be acquired until all sub-pixels in the current image / frame have been compressed.
[0048] The following describes the pixel prediction process for sub-pixels. After determining multiple consecutive sub-pixels, their arrangement can be rearranged. For example, if pixel prediction is performed on a basic prediction unit basis, the arrangement of sample points within that basic prediction unit can be rearranged. In some examples, the arrangement of multiple consecutive sub-pixels can be adjusted according to a certain rearrangement rule, where consecutive sub-pixels in the same channel are arranged alternately.
[0049] This rearrangement rule can be understood in two aspects. First, there is a gap between the sub-pixels of the same channel after adjustment, that is, there is a gap of at least one sub-pixel. Second, for a single sub-pixel, its originally adjacent sub-pixels are still in adjacent positions after adjustment. That is to say, the sub-pixels of the same channel are still arranged in order.
[0050] After adjustment, each adjusted sub-pixel can be predicted one by one according to the reference to obtain the predicted value of each sub-pixel.
[0051] In this context, the reference for pixel prediction for each sub-pixel is the reconstructed sub-pixel of the surrounding sub-pixels in the same channel. The "same channel" refers to the same color channel; for example, if the reference reconstructed sub-pixel is in the Y channel, the corresponding sub-pixel for prediction will also be in the Y channel.
[0052] Because the subpixels in the same channel are arranged in a continuous spacing after adjustment, the reference subpixel is spaced apart from the current subpixel being predicted in the row direction. At this time, the reference for each subpixel when performing pixel prediction includes the reconstructed subpixel of the subpixel in the same channel that is spaced apart from the subpixel in the row where the subpixel is located.
[0053] The aforementioned spacing refers to the fact that the reference sub-pixel and the predicted sub-pixel are separated by other sub-pixels. These other sub-pixels can be sub-pixels of different color channels, or sub-pixels of the same color channel as the reference or predicted sub-pixel. The number of these other intervening sub-pixels can be single or multiple.
[0054] Meanwhile, the aforementioned reference sub-pixels in the same channel can also be several sub-pixels in the previous row or several rows. In the same row, the reference sub-pixels can also be arranged at intervals.
[0055] It should be noted that the aforementioned reconstructed subpixel refers to the reconstructed pixel value of a subpixel that has completed pixel prediction and reconstruction (including quantization and dequantization processes) when the current subpixel is performing pixel prediction. Specifically, pixel prediction can be performed with reference to the reconstructed pixel value obtained from its reconstruction.
[0056] When predicting any sub-pixel, the reconstructed pixel values of surrounding sub-pixels that have already been reconstructed can be referenced. In the row direction, the reference sub-pixel belongs to the same channel as the sub-pixel being predicted and is separated by at least one sub-pixel. Therefore, when each sub-pixel finds a reconstructed sub-pixel in the same channel, there is at least a one-pixel gap, providing sufficient processing time in hardware implementation and solving the hardware timing limitations mentioned earlier in pixel prediction. Furthermore, since the reconstructed sub-pixels referenced by each sub-pixel are originally located in the adjacent surrounding sub-pixels, the original prediction accuracy of the current sub-pixel is not lost, greatly improving the compression efficiency of lossy compression algorithms based on predictive coding methods.
[0057] It should be noted that the process of rearranging multiple consecutive sub-pixels described above can be determined based on the arrangement of sub-pixels in the current frame, as long as the original natural processing order is eventually disrupted to form a continuous arrangement of sub-pixels in the same channel.
[0058] For example, if the current frame contains sub-pixels from multiple channels, the adjustment can be made by referring to the situation of sub-pixels within multiple channels. That is, when multiple consecutive sub-pixels belong to multiple channels, after adjustment, consecutive sub-pixels in the same channel should be separated by at least one sub-pixel from another channel, and the sub-pixels from different channels should be arranged in an alternating manner.
[0059] When multiple consecutive subpixels belong to multi-channel subpixels, it can be understood in two ways. First, after adjustment, the subpixels of each channel are still arranged according to the original arrangement order of each channel, that is, consecutive subpixels are arranged with intervals after adjustment. Second, after adjustment, subpixels of the same channel are separated by at least one subpixel of a different channel, that is, subpixels of different channels are arranged alternately.
[0060] The number of channels mentioned above can be 2, 3 or 4.
[0061] For example, please see Figure 5 , Figure 5 The multiple consecutive sub-pixels involved include sub-pixels x0 to x7 and y0 to y7, a total of 16 sub-pixels. These 16 sub-pixels all come from the same row of the image frame. x and y indicate that they come from different image channels. The number of channels is 2. The number after x or y indicates the arrangement order of the sub-pixels in their respective channels.
[0062] In the Figure 5 When multiple consecutive sub-pixels are adjusted, it can form Figure 5 As shown in (1). In Figure 5In (1), the sub-pixels of different image channels are arranged alternately, while the sub-pixels of the same channel are arranged at intervals, and the sub-pixels of the same channel are also arranged sequentially.
[0063] In the Figure 5 (1) When performing pixel prediction for each sub-pixel, the reference reconstructed sub-pixel can be found by referring to the direction indicated by the arrow. For example, x1 can be referenced by the reconstructed pixel value of x0, while x1 and x0 are separated by y0 of other channels; y1 can be referenced by the reconstructed pixel value of y0, while y1 and y0 are separated by x1 of other channels. And so on, pixel prediction for each sub-pixel in the x and y channels can be completed.
[0064] against Figure 5 (1) shows the adjusted sub-pixel arrangement diagram, in which sub-pixels of two image channels can share a single prediction circuit structure to achieve pixel prediction, and the effect of pixel prediction is almost equivalent to Figure 5 (2) The ideal predictive reference relationship.
[0065] For example, please see Figure 6 , Figure 6 (1) The multiple consecutive sub-pixels involved include sub-pixels x0 to x3, y0 to y3, and z0 to z3, totaling 12 sub-pixels. These 12 sub-pixels all come from the same row of the image frame. x, y, and z indicate that they come from different image channels. The number of channels is 3. The numbers after x, y, and z indicate the arrangement order of the sub-pixels within their respective channels. For example, x, y, and z in the figure can be the Y, U, or V channels in the YUV420 planar format.
[0066] In the Figure 6 When multiple consecutive sub-pixels are adjusted, it can form Figure 6 As shown in (1). In Figure 6 In (1), the sub-pixels of different image channels are arranged in an alternating manner, while the sub-pixels in the same channel are arranged sequentially and at intervals. Overall, after adjustment, the sub-pixels of different channels are arranged in an alternating manner point by point.
[0067] In the Figure 6 (1) When performing pixel prediction for each sub-pixel, the reference reconstructed sub-pixel can be found by referring to the direction indicated by the arrow. For example, x1 can be referenced by the reconstructed pixel value of x0, while x1 and x0 are separated by y0 and z0 of other channels; y1 can be referenced by the reconstructed pixel value of y0, while y1 and y0 are separated by z0 and x1 of other channels; z1 can be referenced by the reconstructed pixel value of z0, while z1 and z0 are separated by x1 and y1 of other channels. And so on, pixel prediction for each sub-pixel in the x, y, and z channels can be completed.
[0068] against Figure 6 (1) The adjusted sub-pixel arrangement structure shown allows sub-pixels in the three image channels to share a single prediction circuit structure for pixel prediction, and the effect of pixel prediction is almost equivalent to... Figure 6 (2) The ideal predictive reference relationship.
[0069] The above illustrates the case where the reconstructed subpixel of a subpixel in the same channel surrounding the current subpixel is the reconstructed subpixel of a single subpixel in the same channel to the left of the current subpixel. In other examples, the number of reconstructed subpixels can also be multiple, and / or, the reconstructed subpixels can also be the reconstructed subpixels of several subpixels in the row above the current subpixel.
[0070] In some examples, the reconstructed subpixel of the surrounding subpixels of the same channel is the reconstructed subpixel to the left of the current subpixel being predicted, or the reconstructed subpixel to the left and the previous row, from several subpixels of the same channel.
[0071] For example, please continue reading Figure 6 In (1), when y3 performs pixel prediction, the reference reconstructed sub-pixels can be y2 and y1 and their reconstructed pixel values, and when z3 performs pixel prediction, the reference reconstructed sub-pixels can be z2 and z1 and their reconstructed pixel values.
[0072] For example, please refer to Figure 7 This illustrates the case where the reconstructed subpixels of the same channel surrounding the current subpixel are the subpixels to its left and in the row above it. In this case, the total number of reconstructed subpixels can be three. Taking the current subpixel y1 as an example (green box), its reference reconstructed subpixels (yellow box) include y0 and y1 in the row above it, as well as y0 in the row to its left. The current subpixel is separated from its reference y0 in the same row by a subpixel x1 in the x-channel. Similarly, the reference reconstructed subpixels y0 and y1 in the row above the current subpixel are also separated from each other by a subpixel x1 in the row above it.
[0073] When using the reconstructed subpixels of the previous row and the same channel on the left as references, compared to using only the reconstructed subpixels of the same channel on the left, it can adapt to the local structure of the image frame and significantly reduce the prediction error.
[0074] In summary, the above examples demonstrate a multi-channel hybrid prediction method that interleaves sub-pixels from multiple channels according to a set pattern for prediction. This solves the hardware timing issues faced in the prediction process while ensuring the principle of predicting the same color channel, resulting in optimal prediction performance. Furthermore, the interleaving ensures that the subsequent entropy-encoded output stream is interleaved, eliminating the distinction between different color components. This approach simplifies the stream storage structure, makes it more convenient to use, and allows for a smaller chip hardware area.
[0075] In some examples, considering that some row pixels are located at the first row of the image frame to be compressed, there is no upper reference row; or, some image compression schemes stipulate that the upper reference row is not allowed to be used, for example, some chips do not allow the use of row buffer in order to save area. Therefore, in some examples of this application, the upper reference row is also disabled.
[0076] The conditions for disabling the top reference line include mandating that the top reference line is not allowed and that the subpixel is located in at least one of the first lines of the image frame.
[0077] When the current sub-pixel meets the upper reference row disable condition, the reconstructed sub-pixel of the surrounding sub-pixels of the same channel can be the reconstructed sub-pixels of several sub-pixels from the same channel to the left of the current sub-pixel being predicted.
[0078] When the current sub-pixel does not meet the upper reference row disable condition, the reconstructed sub-pixels of the surrounding sub-pixels in the same channel are the reconstructed sub-pixels of several sub-pixels from the same channel to the left and in the previous row of the current sub-pixel.
[0079] In the example above, by setting the upper reference row disable condition, the reconstructed subpixel corresponding to each subpixel during pixel prediction can be flexibly set, making the most of the local structure of the image frame to be compressed and reducing prediction error.
[0080] In some examples, the aforementioned image frame or multiple consecutive sub-pixels may also contain only a single-channel sub-pixel, i.e., in the above... Figure 4 Step S102 may include the following steps: When multiple consecutive sub-pixels belong to a single channel, the multiple consecutive sub-pixels are divided into multiple segments, and sub-pixels with the same index position are extracted from each segment and rearranged and spliced.
[0081] The main difference between this example and the previous example is that it can adjust multiple consecutive sub-pixels of a single channel so that even sub-pixels containing only a single channel can achieve pixel prediction by referring to step S103.
[0082] In this example, multiple consecutive sub-pixels in a single channel are divided into multiple segments, such as two parts, front and back. The sub-pixels in the front part can be placed in odd-numbered positions, and the sub-pixels in the back part can be placed in even-numbered positions. Alternatively, they can be divided into three sub-segments, and then the sub-pixels at the same index positions in each sub-segment are extracted and reassembled.
[0083] For example, please see Figure 8 ,in Figure 8(1) There are a total of 16 sub-pixels from x0 to x15. These 16 sub-pixels are all sub-pixels of the same channel, and they constitute multiple consecutive sub-pixels.
[0084] When making multiple consecutive subpixel adjustments, the subpixels from x0 to x7 in the first half can be treated as one sub-segment, in which each subpixel is inserted at an odd number of times; the subpixels from x8 to x15 in the second half can be treated as another sub-segment, in which each subpixel is inserted at an even number of times.
[0085] See Figure 8 (2) The order of the sub-pixels within the multiple consecutive sub-pixels after arrangement is as follows: x0, x8, x1, x9, x2, x10, x3, x11, x4, x12, x5, x13, x6, x14, x7, x15.
[0086] For example, when performing pixel prediction, x12 can be referenced to the reconstructed subpixel of x11 at an interval of x4, and x3 can be referenced to the reconstructed subpixel of x2 at an interval of x10.
[0087] For example, when performing pixel prediction, x14 can be referenced to the reconstructed subpixel of x13 separated by one subpixel and the reconstructed subpixel of x12 separated by three subpixels, and x3 can be referenced to the reconstructed subpixel of x2 separated by one subpixel and the reconstructed subpixel of x1 separated by three subpixels.
[0088] For example, please see Figure 9 It shows Figure 8 A portion consisting of multiple consecutive sub-pixels. Figure 9 In the context of pixel prediction, when using the reconstructed subpixels of the same channel as the subpixels in the previous row and to the left as references, taking x3 as the current subpixel undergoing prediction as an example, its referenced reconstructed subpixels of the same channel include the reconstructed pixel value of x2, which is separated by one subpixel in the current row, and y3 and y2 in the previous row. Here, x and y belong to the same pixel channel, x represents the subpixel of the row containing multiple consecutive subpixels, and y represents the reconstructed subpixel of the subpixel of the row preceding the row containing multiple consecutive subpixels.
[0089] This example provides a pixel prediction scheme for single-channel sub-pixels. Before pixel encoding compression, by shuffling the original processing order, multiple consecutive sub-pixels with the adjusted order can meet the pixel prediction requirements of hardware timing, while ensuring that the prediction accuracy is almost the same as the original prediction accuracy, thereby improving image compression efficiency.
[0090] In some other examples, considering that the pixel arrangement in some image frames naturally satisfies the condition of pixel prediction with a reference at least one sub-pixel interval, a channel interleaving condition can be set. When the sub-pixels within multiple consecutive sub-pixels meet the channel interleaving condition, sub-pixel rearrangement / adjustment of multiple consecutive sub-pixels is not performed; instead, the original arrangement order of the sub-pixels within multiple consecutive sub-pixels is directly followed to achieve the interleaved prediction of sub-pixels.
[0091] That is, after obtaining multiple consecutive sub-pixels in step S101, the method further includes: S104, when multiple consecutive sub-pixels have met the channel interleaving condition, each sub-pixel is predicted with reference to a reconstructed sub-pixel that is at least one sub-pixel apart from each sub-pixel. Step S102 is performed only if multiple consecutive sub-pixels do not meet the channel interleaving condition.
[0092] The aforementioned channel interleaving condition assumes that the pixel arrangement in the image frame naturally satisfies pixel prediction with a reference at least one sub-pixel interval. For example, the image frame can be in Raw Bayer format or an image containing only a single-channel sub-pixel.
[0093] Please refer to Figure 10 , Figure 10 A schematic diagram of the Raw Bayer format is shown. When predicting pixels in this format row by row, after determining multiple consecutive sub-pixels, the above-mentioned S103 can be directly referenced to perform pixel prediction using the reconstructed sub-pixel to its left that is separated by one other channel pixel and / or several reconstructed sub-pixels in the row above it, so as to obtain the predicted value.
[0094] Please refer to Figure 11 , Figure 11 This illustrates the pixel prediction process for multiple consecutive sub-pixels in a single channel without executing step S102. In this case, the current sub-pixel in the multiple consecutive sub-pixels no longer uses the immediately adjacent reconstructed sub-pixel to its left for prediction, but instead uses the reconstructed sub-pixel at a distance of N points to its left for prediction. Optionally, N is greater than or equal to 1, and preferably N=1.
[0095] Of course, in other examples, prediction can also be made using the reconstructed subpixels at N points intervals to the left and the reconstructed subpixels from the previous row.
[0096] This ensures that there are no timing issues in the hardware implementation when each sub-pixel is performing pixel prediction, and also guarantees prediction accuracy and compression efficiency to a certain extent.
[0097] Finally, it should be noted that the process of encoding based on the predicted value of each sub-pixel to obtain the compressed bitstream has been described in the background section and can also be performed with reference to existing technologies in this field, and will not be repeated here.
[0098] In the scheme of this application, as long as the predicted value of each pixel can be obtained based on the above-described encoding method, and then the compressed bitstream can be obtained, the compressed bitstream can be transmitted to the decoding end, and the decoding end can restore the original image. Based on the above embodiments, this application also provides a decoding method, which may include: A201, Obtain the bitstream, which is obtained as described in the above encoding method embodiment; A202, decode the bitstream to obtain multiple pixel data; A203, the arrangement order of multiple pixel data is adjusted according to the inverse adjustment rule, which is the opposite of the adjustment method of multiple consecutive sub-pixels in the above encoding method; A204 combines the adjusted pixel data into an image.
[0099] The inverse adjustment rule is the opposite of the adjustment method for multiple consecutive sub-pixels in the above encoding method embodiment; the specific inverse adjustment scheme is also determined based on the composition of multiple consecutive sub-pixels.
[0100] It should be noted that if the arrangement order of multiple consecutive sub-pixels is adjusted during the image encoding and compression process, the original arrangement order of the sub-pixels in the image is disrupted. If the image is decompressed and restored in this way, the restored image will not be the original image.
[0101] Therefore, if the decoding end wants to obtain the original image, it also needs to refer to the inverse adjustment rule that is the opposite of the adjustment method during pixel prediction, so that the pixel data can be restored to the arrangement position of pixels in the original image, and then the image can be formed. The encoding device provided by the present invention is described below. The encoding device described below can be referred to in correspondence with the encoding method described above.
[0102] The acquisition module is used to acquire multiple consecutive sub-pixels; An adjustment module is used to readjust the arrangement order of multiple consecutive sub-pixels so that the consecutive sub-pixels in the same channel are arranged at intervals after adjustment. The prediction module is used to predict each sub-pixel sequentially, taking the reconstructed sub-pixels of the sub-pixels in the same channel around each of the adjusted sub-pixels as a reference, to obtain the predicted value of each sub-pixel. An encoding module is used to encode the predicted value of each sub-pixel to obtain a bitstream.
[0103] Optionally, when multiple consecutive sub-pixels belong to multi-channel sub-pixels, the adjusted consecutive sub-pixels in the same channel are spaced apart by at least one sub-pixel from another channel.
[0104] Optionally, the adjustment module includes: The setting unit is used to divide the multiple consecutive sub-pixels into multiple segments when they belong to a single channel, extract sub-pixels with the same index position from each segment, and rearrange and splice them.
[0105] Optionally, the reconstructed subpixels of the subpixels in the same channel surrounding the adjusted subpixel are the reconstructed subpixels of several subpixels from the same channel to the left / left and previous row of the current subpixel being predicted. Among them, the reconstructed subpixels to the left of the current subpixel are separated from the current subpixel by at least one subpixel.
[0106] Optionally, when the current sub-pixel does not meet the upper reference row disable condition, the reconstructed sub-pixel of the surrounding sub-pixels in the same channel is the reconstructed sub-pixel of several sub-pixels from the same channel to the left and in the previous row of the current sub-pixel being predicted. When the current sub-pixel meets the upper reference row disable condition, the reconstructed sub-pixel of the surrounding sub-pixels in the same channel is the reconstructed sub-pixel of several sub-pixels from the same channel to the left of the current sub-pixel being predicted.
[0107] Optionally, during the row-by-row compression process of the setting module, if multiple consecutive sub-pixels have met the channel interleaving condition, the prediction module makes a prediction for each sub-pixel with reference to the reconstructed sub-pixel that is at least one sub-pixel apart from each sub-pixel. If multiple consecutive sub-pixels do not meet the channel interleaving condition, the adjustment module readjusts the arrangement order of the multiple consecutive sub-pixels so that the adjusted consecutive sub-pixels in the same channel are arranged with intervals. The present invention also provides a decoding device, the decoding device described below and the decoding method described above can be referred to in correspondence. The decoding device includes: An acquisition module is used to acquire a bitstream, which is obtained by the encoding method described above; A decoding module is used to decode the bitstream to obtain multiple pixel data; An adjustment module is used to adjust the arrangement order of multiple pixel data according to an inverse adjustment rule, which is the opposite of the adjustment method for multiple consecutive sub-pixels in the above-described encoding method. The combination module is used to combine adjusted pixel data into an image. Figure 12An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 12 As shown, the electronic device may include a processor 1210, a communications interface 1220, a memory 1230, and a communication bus 1240. The processor 1210, communications interface 1220, and memory 1230 communicate with each other via the communication bus 1240. The processor 1210 can call logical instructions from the memory 1230 to execute the aforementioned encoding or decoding methods.
[0108] Furthermore, the logical instructions in the aforementioned memory 1230 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0109] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the encoding or decoding methods provided by the above methods.
[0110] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform the encoding or decoding methods provided by the above methods.
[0111] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.
[0112] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.
[0113] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. An encoding method, characterized in that, include: Obtain multiple consecutive sub-pixels; The arrangement order of multiple consecutive sub-pixels is readjusted so that the consecutive sub-pixels in the same channel are arranged with intervals after the adjustment; With reference to the reconstructed subpixels of the subpixels in the same channel surrounding each adjusted subpixel, prediction is performed on each subpixel in turn to obtain the predicted value of each subpixel. The bitstream is obtained by encoding the predicted value of each sub-pixel.
2. The method according to claim 1, characterized in that, When multiple consecutive sub-pixels belong to multi-channel sub-pixels, the adjusted consecutive sub-pixels in the same channel are spaced apart by at least one sub-pixel from another channel.
3. The method according to claim 1, characterized in that, The step of readjusting the arrangement order of multiple consecutive sub-pixels, so that the adjusted consecutive sub-pixels in the same channel are arranged with intervals, includes: When multiple consecutive sub-pixels belong to a single channel, the multiple consecutive sub-pixels are divided into multiple segments, and sub-pixels with the same index position are extracted from each segment and rearranged and spliced.
4. The method according to any one of claims 1 to 3, characterized in that, The reconstructed subpixels of the subpixels in the same channel surrounding the adjusted subpixel are the reconstructed subpixels of several subpixels from the same channel to the left / left and previous row of the current subpixel.
5. The method according to claim 4, characterized in that, When the current sub-pixel does not meet the upper reference row disable condition, the reconstructed sub-pixel of the surrounding sub-pixels in the same channel is the reconstructed sub-pixel of several sub-pixels from the same channel to the left and in the previous row of the current sub-pixel. When the current sub-pixel meets the upper reference row disable condition, the reconstructed sub-pixel of the surrounding sub-pixels in the same channel is the reconstructed sub-pixel of several sub-pixels from the same channel to the left of the current sub-pixel being predicted.
6. The method according to any one of claims 1 to 3, characterized in that, After acquiring multiple consecutive sub-pixels, the process further includes: When multiple consecutive sub-pixels have met the channel interleaving condition, each sub-pixel is predicted with reference to a reconstructed sub-pixel that is at least one sub-pixel apart from each sub-pixel. If multiple consecutive sub-pixels do not meet the channel interleaving condition, the following step is performed: readjust the arrangement order of the multiple consecutive sub-pixels so that the adjusted consecutive sub-pixels in the same channel are arranged with intervals.
7. A decoding method, characterized in that, The method includes: Acquire a bitstream, wherein the bitstream is obtained by the encoding method as described in any one of claims 1 to 6; The bitstream is decoded to obtain multiple pixel data; The arrangement order of the multiple pixel data is adjusted according to the inverse adjustment rule, which is the opposite of the adjustment method of multiple consecutive sub-pixels in the encoding method according to any one of claims 1 to 6; The adjusted pixel data is combined into an image.
8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the encoding method as described in any one of claims 1 to 6 or the decoding method as described in claim 7.
9. A computer program product, characterized in that, It includes a computer program that, when executed by a processor, implements the encoding method as described in any one of claims 1 to 6 or the decoding method as described in claim 7.
10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the encoding method as described in any one of claims 1 to 6 or the decoding method as described in claim 7.