Image encoding and decoding method, device and storage medium
By directly determining the prediction mode of video data from the index table based on the target prediction mode and combining the residual encoding mode, the problem of low encoding and decoding efficiency in the prior art is solved, and efficient code streaming and low complexity decoding are realized. .
Patent Information
- Application Number
- CN202210016199.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-01-07
- Publication Date
- 2024-11-01
- Estimated Expiration
- 2042-01-07
AI Technical Summary
The prior art is difficult to efficiently code large-scale video data, resulting in increased storage space and transmission bandwidth usage and high decoding complexity.
The target prediction mode is determined from the index table directly based on the target prediction mode, which reduces the decoding complexity, and saves code streaming overhead by combining the residual encoding mode and the prediction mode as the new prediction mode.
It improves image encoding and decoding efficiency, reduces the overhead of streaming and decoding complexity, and improves encoding and decoding efficiency.
Smart Images

Figure CN116074529B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of image coding and decoding technology, and in particular to an image coding and decoding method, device and storage medium. Background Art
[0002] A complete image in a video is usually called a "frame", and a video composed of multiple frames in chronological order is called a video sequence. Video sequences contain a series of redundant information, including spatial redundancy, temporal redundancy, visual redundancy, information entropy redundancy, structural redundancy, knowledge redundancy, and importance redundancy. In order to remove redundant information in video sequences as much as possible and reduce the amount of data representing the video, video coding technology has been proposed to achieve the effect of reducing storage space and saving transmission bandwidth. Video coding technology is also called video compression technology.
[0003] With the continuous development of technology, it is becoming more and more convenient to collect video data, and the scale of collected video data is also getting larger and larger. Therefore, how to effectively encode and decode video data has become an urgent problem that needs to be solved. Summary of the invention
[0004] The present application provides an image encoding and decoding method, device and storage medium for effectively encoding and decoding video data, thereby improving encoding and decoding efficiency.
[0005] In order to achieve the above objectives, this application adopts the following technical solutions:
[0006] In a first aspect, an image decoding method is provided, comprising: parsing a bitstream to obtain a first syntax element, the first syntax element comprising an index of a target prediction mode of a unit to be decoded; determining the target prediction mode from an index table based on the index of the target prediction mode; wherein the index table comprises a correspondence between the indexes of multiple prediction modes and the multiple prediction modes; and reconstructing the unit to be decoded at least based on the target prediction mode to obtain a reconstructed block. The technical solution can directly determine the target prediction mode from the index table based on the index of the target prediction mode without parsing the flag org_flag, thereby reducing the decoding complexity of the decoding end, thereby improving the decoding efficiency.
[0007] In a possible implementation, the indexes of the multiple prediction modes are generated in a truncated unary code manner, which helps to further reduce the bitstream transmission overhead.
[0008] In a possible implementation, indexes of multiple prediction modes are generated in a binary tree manner.
[0009] In a possible implementation, the multiple prediction modes include the original value mode and other prediction modes, and the codeword length of the index of the original value mode is greater than or equal to the codeword length of the index of one or more prediction modes in the other prediction modes. Since some other prediction modes are used more frequently than the original value mode, the codeword length of their index is set shorter, which helps to further reduce the code stream transmission overhead.
[0010] In a possible implementation, the other prediction modes include at least one of the following: a point prediction mode, an intra-frame prediction mode, or a block copy mode.
[0011] In a possible implementation, the method further includes: parsing the bitstream to obtain a second syntax element; wherein the second syntax element includes an index of a residual coding mode of a unit to be decoded; reconstructing the unit to be decoded at least based on a target prediction mode to obtain a reconstructed block, including: if the index of the residual coding mode is used to indicate skipping the residual coding mode, predicting the unit to be decoded to obtain a prediction block, and determining the prediction block of the unit to be decoded as the reconstructed block of the unit to be decoded; if the index of the residual coding mode is used to indicate a normal residual coding mode, parsing a residual quantization related value of the unit to be decoded to obtain a residual block; and reconstructing the unit to be decoded based on the target prediction mode and the residual block of the unit to be decoded to obtain a reconstructed block.
[0012] In one possible implementation, other prediction modes include at least one of the following: point-by-point prediction mode, intra-frame prediction mode, block copy mode based on normal residual coding, point-by-point prediction mode, intra-frame prediction mode or block copy mode based on skipped residual coding.
[0013] In one possible implementation, the unit to be decoded is reconstructed at least based on a target prediction mode to obtain a reconstructed block, including: if the target prediction mode is a point-by-point prediction mode, an intra-frame prediction mode, or a block copy mode based on skipping residual coding, the prediction block of the unit to be decoded is determined as the reconstructed block of the unit to be decoded; if the target prediction mode is a point-by-point prediction mode, an intra-frame prediction mode, or a block copy mode based on normal residual coding, the residual quantization related value of the unit to be decoded is parsed to obtain the residual block of the unit to be decoded, and the unit to be decoded is reconstructed based on the target prediction mode and the residual block of the unit to be decoded to obtain the reconstructed block of the unit to be decoded.
[0014] In one possible implementation, the unit to be decoded includes at least one prediction group; any one of the at least one prediction group includes a plurality of continuous pixels in the same row / column; any one of the prediction groups includes a first designated pixel area and a second designated pixel area, the first designated pixel area includes a plurality of pixel groups, the plurality of pixel groups are obtained by dividing the second designated pixel area, the prediction methods of the first designated pixel area and the second designated pixel area are different, the plurality of pixel groups can be predicted in parallel, and the pixel group includes one or a plurality of continuous pixels.
[0015] In one possible implementation, if any prediction group includes multiple consecutive elements in the same row, then: the first designated pixel area adopts vertical prediction, and the second designated pixel area adopts horizontal prediction; or, the first designated pixel area adopts vertical prediction, and the second designated pixel area adopts vertical mean prediction.
[0016] In one possible implementation, if any prediction group includes multiple consecutive elements in the same column, then: the first designated pixel area uses horizontal prediction, and the second designated pixel area uses vertical prediction; or, the first designated pixel area uses horizontal prediction, and the second designated pixel area uses horizontal average prediction.
[0017] In a possible implementation manner, the difference between the numbers of pixels included in different pixel groups is equal to or smaller than a threshold.
[0018] In a possible implementation, if the unit to be decoded does not have a reference block, the reconstructed value of the first pixel of the unit to be decoded is a value obtained by shifting the bit width of the unit to be decoded left by 1 bit.
[0019] In one possible implementation, the unit to be decoded includes at least one prediction group; any one of the at least one prediction group contains a plurality of continuous pixels; the unit to be decoded is reconstructed at least based on a target prediction mode to obtain a reconstructed block, including: when the target prediction mode satisfies a preset condition, based on a reference value of a first pixel among the plurality of continuous pixels and a residual value of every two adjacent pixels between the first pixel and the target pixel, determining a reconstructed value of a target pixel; wherein the target pixel is any non-first pixel among the plurality of continuous pixels, and the reconstructed block of the unit to be decoded contains the reconstructed value of the target pixel.
[0020] In a possible implementation, the first pixel adopts the first prediction method, and the non-first pixel adopts the second prediction method; the target prediction mode includes the first prediction method and the second prediction method, and the target prediction mode satisfies the preset conditions including: the first prediction method and the second prediction method are both horizontal predictions; or, the first prediction method and the second prediction method are both vertical predictions; or, one of the first prediction method and the second prediction method is horizontal prediction, and the other is vertical prediction; or, the first prediction method is a reference prediction method using pixel values of adjacent decoding units to the unit to be decoded, or independent decoding units adjacent to the independent decoding unit where the unit to be decoded is located, and the second prediction method is horizontal prediction or vertical prediction.
[0021] In a second aspect, an image encoding method is provided, including: determining a target prediction mode of a unit to be encoded; determining an index of the target prediction mode from an index table based on the target prediction mode; wherein the index table contains a correspondence between the indexes of multiple prediction modes and the multiple prediction modes; and encoding the index of the target prediction mode into a bitstream. This technical solution determines the index of the target prediction mode from an index table directly based on the target prediction mode, and this process may not include an index (such as a flag bit org_flag) indicating whether the target prediction mode is an original value in the bitstream. Since in most cases, the target prediction mode does not use the original value mode, for example, a point-by-point prediction mode or an intra-frame prediction mode is usually used, therefore, in most cases, there is no need to encode the flag bit (org_flag) in the traditional scheme that represents whether the original value mode is used, which helps to save bitstream transmission overhead, thereby improving encoding efficiency.
[0022] In a possible implementation, the indexes of the multiple prediction modes are generated in a truncated unary code manner.
[0023] In a possible implementation, indexes of multiple prediction modes are generated in a binary tree manner.
[0024] In a possible implementation, the multiple prediction modes include an original value mode and other prediction modes, and the codeword length of the index of the original value mode is greater than or equal to the codeword length of the index of one or more prediction modes in the other prediction modes.
[0025] In a possible implementation, the multiple prediction modes include an original value mode and other prediction modes, and the other prediction modes include at least one of the following: a point prediction mode, an intra-frame prediction mode, or a block copy mode.
[0026] In one possible implementation, other prediction modes include at least one of the following: point-by-point prediction mode, intra-frame prediction mode, block copy mode based on normal residual coding, point-by-point prediction mode, intra-frame prediction mode or block copy mode based on skipped residual coding.
[0027] In a third aspect, an image reconstruction method is provided, which can be applied to an image encoding method or an image decoding method. The image reconstruction method includes: determining a residual coding mode of a current image block; if the residual coding mode is a skip residual coding mode, predicting the current image block to obtain a predicted block, and determining the predicted block as a reconstructed block of the current image block. If the residual coding mode is a normal residual coding mode, obtaining a residual quantization related value of the current image block to obtain a residual block; and reconstructing the current image block based on the residual block to obtain a reconstructed block of the current image block.
[0028] In the present technical solution, the residual coding mode is first determined, and then the residual quantization related values of the current image block are obtained when the residual coding mode is the normal residual coding mode. When the residual coding mode is the skip residual coding mode, it is not necessary to obtain the residual quantization related values of the current image block. In this way, when the residual coding mode is the normal residual coding mode, the encoding end does not need to encode the residual quantization related values of the current image block into the bitstream, and the decoding end does not need to parse the residual quantization related values of the current image block, which helps to save bitstream transmission overhead, thereby improving coding efficiency; and helps to reduce decoding complexity, thereby improving decoding efficiency.
[0029] In a fourth aspect, an image reconstruction method is provided, which can be applied to an image encoding method or an image decoding method. The image reconstruction method includes: determining a target prediction mode of a current image block; if the target prediction mode is a prediction mode based on skip residual coding, predicting a previous image block to obtain a prediction block, and determining the prediction block as a reconstructed block; if the target prediction mode is a prediction mode based on normal residual coding, obtaining a residual quantization related value of the previous image block to obtain a residual block of the previous image block, and reconstructing the previous image block based on the target prediction mode and the residual block to obtain a reconstructed block.
[0030] In this technical solution, on the one hand, the traditional residual coding mode is combined with the prediction mode as a new prediction mode provided by the embodiment of the present application. In this way, there is no need to encode (decode) the residual coding mode and the prediction mode separately, and only the new prediction mode needs to be uniformly encoded (decoded). The implementation logic is simple and helps to save the code stream transmission overhead. On the other hand, for the encoding end (or decoding end), when reconstructing the image block in this way, the residual coding mode information (such as the index of the residual coding mode, such as res_skip_flag) can be encoded (or decoded) first, and then based on the residual coding mode, it is determined whether to encode (or decode) the residual quantization related value (such as the near value, or QP value). Compared with the traditional technology, when the residual coding mode is the skip residual coding mode, it is not necessary to encode (or decode) the residual quantization related value, which helps to save the code stream transmission overhead.
[0031] In a possible implementation manner, the point-by-point prediction mode based on skipping residual coding includes: a point-by-point prediction mode based on skipping residual coding, an intra-frame prediction mode, or a block copy mode.
[0032] In a possible implementation manner, the prediction mode based on normal residual coding includes: a point-by-point prediction mode, an intra-frame prediction mode or a block copy mode based on normal residual coding.
[0033] In a fifth aspect, an image reconstruction method is provided, which can be applied to an image encoding method or an image decoding method. The image reconstruction method includes: determining a prediction mode of at least one prediction group into which the current image block is divided; wherein any one of the at least one prediction group includes a plurality of continuous pixels in the same row / column; any one of the prediction groups includes a first designated pixel area and a second designated pixel area, the first designated pixel area includes a plurality of pixel groups, the plurality of pixel groups are obtained according to the division of the second designated pixel area, the prediction mode of the first designated pixel area is different from that of the second designated pixel area, the plurality of pixel groups can be predicted in parallel, and the pixel group includes one or a plurality of continuous pixels; based on the prediction mode of at least one prediction group, the current image block is reconstructed to obtain a reconstructed block. In this technical solution, the plurality of pixel groups in a prediction group can be predicted in parallel, thus helping to shorten the prediction time of the prediction group.
[0034] In one possible implementation, if any prediction group includes multiple consecutive elements in the same row, then: the first designated pixel area adopts vertical prediction, and the second designated pixel area adopts horizontal prediction; or, the first designated pixel area adopts vertical prediction, and the second designated pixel area adopts vertical mean prediction.
[0035] In one possible implementation, if any prediction group includes multiple consecutive elements in the same column, then: the first designated pixel area adopts horizontal prediction, and the second designated pixel area adopts vertical prediction; or, the first designated pixel area adopts horizontal prediction, and the second designated pixel area adopts horizontal average prediction.
[0036] In a possible implementation manner, the difference between the numbers of pixels included in different pixel groups is equal to or smaller than a threshold.
[0037] In a possible implementation, if there is no reference block for the current image block, the reconstructed value of the first pixel of the current image block is a value obtained by shifting the bit width of the current image block left by 1 bit.
[0038] In a sixth aspect, a method that can be applied to an image encoding method or an image decoding method is provided. The image reconstruction method includes: determining a target prediction mode of a current image block; wherein the current image block includes at least one prediction group; any one of the at least one prediction group includes a plurality of continuous pixels; when the target prediction mode satisfies a preset condition, a reconstruction value of a target pixel is determined based on a reference value of the first pixel in the plurality of continuous pixels and a residual value of each two adjacent pixels between the first pixel and the target pixel; wherein the target pixel is any non-first pixel in the plurality of continuous pixels, and the reconstruction block of the current image block includes the reconstruction value of the target pixel.
[0039] With this technical solution, when performing reconstruction, the encoding / decoding end can directly obtain the reconstructed value of the current pixel based on the residual value between the previous pixel and its adjacent pixels, without waiting for the reconstructed value of the previous pixel. This solution can also greatly improve the parallelism of the reconstruction process, thereby improving the decoding parallelism and throughput.
[0040] In a possible implementation, the first pixel adopts the first prediction method, and the non-first pixel adopts the second prediction method; the target prediction mode includes the first prediction method and the second prediction method, and the target prediction mode satisfies the preset conditions including: the first prediction method and the second prediction method are both horizontal predictions; or, the first prediction method and the second prediction method are both vertical predictions; or, one of the first prediction method and the second prediction method is horizontal prediction, and the other is vertical prediction; or, the first prediction method is a method of using a pixel value of an independent decoding unit adjacent to a current image block, or adjacent to an independent decoding unit where the current image block is located, for reference prediction, and the second prediction method is horizontal prediction or vertical prediction.
[0041] In a seventh aspect, an image decoding device is provided, which may be a video decoder or a device including a video decoder. The decoding device includes modules for implementing the method in any possible implementation of the first aspect, the third aspect, the fourth aspect or the fifth aspect. The decoding device has the function of implementing the behavior in the above-mentioned related method instance. The function can be implemented by hardware, or by hardware executing corresponding software implementation. The hardware or software includes one or more modules corresponding to the above-mentioned function. The beneficial effects thereof can be referred to the description in the corresponding method, which will not be repeated here.
[0042] In an eighth aspect, an image encoding device is provided, which may be a video encoder or a device including a video encoder. The encoding device includes modules for implementing the method in any possible implementation of the second aspect, the third aspect, the fourth aspect or the fifth aspect. The encoding device has the function of implementing the behavior in the above-mentioned related method instance. The function can be implemented by hardware, or by hardware executing corresponding software implementation. The hardware or software includes one or more modules corresponding to the above-mentioned function. The beneficial effects thereof can be found in the description of the corresponding method, which will not be repeated here.
[0043] In a ninth aspect, the present application provides an electronic device comprising a processor and a memory, wherein the memory is used to store computer instructions, and the processor is used to call and execute the computer instructions from the memory to implement a method of any one of the implementation methods in the first to sixth aspects.
[0044] For example, the electronic device may refer to a video encoder, or a device including a video encoder.
[0045] For another example, the electronic device may refer to a video decoder, or a device including a video decoder.
[0046] In the tenth aspect, the present application provides a computer-readable storage medium, in which a computer program or instructions are stored. When the computer program or instructions are executed by a computing device or a storage system where the computing device is located, a method of implementing any one of the first to sixth aspects is implemented.
[0047] In the eleventh aspect, the present application provides a computer program product, which includes instructions. When the computer program product runs on a computing device or a processor, the computing device or the processor executes the instructions to implement a method of any one of the implementation methods of the first to sixth aspects.
[0048] In the twelfth aspect, the present application provides a chip including a memory and a processor, wherein the memory is used to store computer instructions, and the processor is used to call and execute the computer instructions from the memory to implement the method of any one of the implementation modes in the first to sixth aspects.
[0049] In a thirteenth aspect, the present application provides an image decoding system, which includes an encoding end and a decoding end, the decoding end is used to implement the corresponding decoding methods provided in the first to sixth aspects, and the encoding end is used to implement the corresponding encoding method.
[0050] Based on the implementation methods provided in the above aspects, the present application can also be further combined to provide more implementation methods. In other words, any possible implementation method of any of the above aspects can be applied to other aspects without conflict, so as to obtain a new embodiment. For example, any image reconstruction method provided in the third to fifth aspects can be applied to any encoding method or decoding method provided in the first or second aspect. For another example, any two reconstruction methods provided in the third to fifth aspects can be combined without conflict, so as to obtain a new reconstruction method. BRIEF DESCRIPTION OF THE DRAWINGS
[0051] Figure 1 A schematic diagram of various prediction methods provided in the embodiments of the present application;
[0052] Figure 2 A schematic diagram of the architecture of the encoding and decoding system used in the embodiments of the present application;
[0053] Figure 3 A schematic block diagram of an example of an encoder for implementing an embodiment of the present application;
[0054] Figure 4 A schematic diagram of a corresponding relationship between an image, a parallel coding unit, an independent coding unit, and a coding unit provided in an embodiment of the present application;
[0055] Figure 5 A schematic diagram of an encoding process provided in an embodiment of the present application;
[0056] Figure 6 A schematic block diagram of an example of a decoder for implementing an embodiment of the present application;
[0057] Fig. 7A A schematic diagram of a process of an image encoding method provided in an embodiment of the present application Figure 1 ;
[0058] Figure 7B A method provided in the embodiment of the present application Fig. 7A A schematic flow chart of the corresponding image decoding method;
[0059] Fig. 8A A schematic diagram of a process of an image encoding method provided in an embodiment of the present application Figure 2 ;
[0060] Figure 8B A method provided in the embodiment of the present application Fig. 8A A schematic flow chart of the corresponding image decoding method;
[0061] Fig. 9A A schematic diagram of a process of an image encoding method provided in an embodiment of the present application Figure 3;
[0062] Fig. 9B A method provided in the embodiment of the present application Fig. 9A A schematic flow chart of the corresponding image decoding method;
[0063] Fig.10 A schematic diagram of an image reconstruction method provided in an embodiment of the present application Figure 1 ;
[0064] Fig.11 A schematic diagram of a method for dividing parallel pixel groups provided in an embodiment of the present application;
[0065] Fig.12 Schematic diagram of the prediction partitioning mode of the current image block of 16x2 Figure 1 ;
[0066] Figure 13A-13B The embodiments of the present application provide Fig.12 A schematic diagram of the corresponding 16x2 prediction partitioning mode of the current image block;
[0067] Fig.14 Schematic diagram of the prediction partitioning mode of the current image block of 16x2 Figure 2 ;
[0068] Figures 15A-15D The embodiments of the present application provide Fig.14 A schematic diagram of the corresponding 16x2 prediction partitioning mode of the current image block;
[0069] Fig.16 A schematic diagram of a prediction partitioning mode for a current image block of 8x2;
[0070] Fig.17 The embodiments of the present application provide Fig.16 A schematic diagram of the corresponding prediction partitioning mode of the current image block of 8x2;
[0071] Fig.18 A schematic diagram of a prediction partitioning mode of a current image block of 8x1;
[0072] Fig.19 The embodiments of the present application provide Fig.18 A schematic diagram of the corresponding prediction partitioning mode of the current image block of 8x1;
[0073] Fig. 20 A schematic diagram of residual block grouping of an 8x1 current image block provided in an embodiment of the present application;
[0074] Fig.21 Schematic diagram of the image reconstruction method provided in the embodiment of the present application Figure 2 ;
[0075] Fig. 22 The embodiments of the present application are provided for illustration Fig.21 Schematic diagram of the prediction mode of an 8x1 image block;
[0076] Figure 23-Figure 28 A schematic diagram of the structure of an encoding device or a decoding device provided in an embodiment of the present application;
[0077] Fig.29 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0078] First, the technical terms involved in the embodiments of the present application are introduced:
[0079] 1) Prediction model
[0080] The combination of prediction methods used to predict the current image block (such as the unit to be encoded / the unit to be decoded) is called a prediction mode. Different prediction methods can be used to predict different pixels in the current image block, or the same prediction method can be used. The prediction methods used to predict all pixels in the current image block can be collectively referred to as the (or corresponding) prediction mode of the current image block.
[0081] Optionally, the prediction mode includes: a point-by-point prediction mode, an intra-frame prediction mode, a block copy mode, and an original value mode (ie, a reconstructed value mode of directly decoding a fixed bit width), etc.
[0082] For example, the point-by-point prediction mode is based on the pixel reconstruction value of the surrounding adjacent points as the prediction value. The point-by-point prediction mode includes a combination of one or more prediction modes such as vertical prediction, horizontal prediction, vertical mean prediction and horizontal mean prediction. Vertical prediction is to use the pixel value on the upper side of the current pixel (which can be the adjacent upper side or the non-adjacent upper side but closer) to obtain the prediction value of the current pixel. One example is: the reconstruction value of the pixel adjacent to the upper side of the current pixel is used as the prediction value of the current pixel. Horizontal prediction is to use the pixel value on the left side of the current pixel (which can be the adjacent left side or the non-adjacent left side but closer) to obtain the prediction value of the current pixel. One example is: the reconstruction value of the adjacent pixel on the left side of the current pixel is used as the prediction value of the current pixel. Vertical mean prediction is to use the pixel values above and below the current pixel to obtain the prediction value of the current pixel. One example is: the prediction value of the current pixel is the average of the reconstruction value of the adjacent pixel vertically above and the reconstruction value of the adjacent pixel vertically below. Horizontal mean prediction is to use the pixel values on the left and right sides of the current pixel to obtain the prediction value of the current pixel. One example is that the predicted value of the current pixel is the average of the reconstructed values of the adjacent pixels on the horizontal left and the reconstructed values of the adjacent pixels on the horizontal right.
[0083] For example, the intra prediction mode uses pixel reconstruction values of surrounding neighboring blocks as prediction values.
[0084] For example, the block copy prediction mode uses pixel reconstruction values of surrounding coded (decoded) blocks (not necessarily adjacent) as prediction values.
[0085] For example, the original value mode is a reconstructed value mode of directly decoding a fixed bit width, that is, a non-reference prediction mode.
[0086] like Figure 1 , which is a schematic diagram of various prediction methods provided in the embodiments of the present application. Figure 1 A small square in the figure represents a pixel. Pixels with different labels are used to represent different prediction methods, specifically vertical prediction, horizontal prediction, vertical mean prediction and horizontal mean prediction. Figure 1 The schematic diagrams of various prediction methods shown can be used to explain any of the related drawings involved in this application, such as Fig.12 wait.
[0087] 2) Residual coding mode
[0088] The way of encoding the residual (i.e., residual block, which is composed of the residual value of each pixel in the current image block) of the current image block (such as the unit to be encoded / the unit to be decoded) is called the residual coding mode. The residual coding mode can include the skip residual coding mode and the normal residual coding mode.
[0089] In skip residual coding mode, there is no need to encode (decode) residual coefficients. At this time, the residual values of the pixels in the current image block are all 0, and the reconstructed value of each pixel is equal to the predicted value of the pixel.
[0090] In the normal residual coding mode, the residual coefficients need to be encoded (decoded). At this time, the residual values of the pixels in the previous image block are not all 0, and the reconstructed value of each pixel can be obtained based on the predicted value and the residual value of the pixel.
[0091] In one example, the residual value of a pixel is equivalent to the residual coefficient of the pixel; in another example, the residual coefficient of a pixel may be obtained by subjecting the residual value of the pixel to certain processing.
[0092] 3) Other terms
[0093] The term "at least one (kind)" in the embodiments of the present application includes one (kind) or more (kinds). "More than one (kind)" refers to two (kinds) or more than two (kinds). For example, at least one of A, B and C includes: A exists alone, B exists alone, A and B exist at the same time, A and C exist at the same time, B and C exist at the same time, and A, B and C exist at the same time. In the description of the present application, unless otherwise specified, " / " means or, for example, A / B can represent A or B; "and / or" in this article is only a description of the association relationship of the associated objects, indicating that there can be three relationships, for example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. "More than one" refers to two or more than two. In addition, in order to facilitate the clear description of the technical solution of the embodiments of the present application, in the embodiments of the present application, the words "first", "second" and so on are used to distinguish the same items or similar items with basically the same functions and effects. Those skilled in the art can understand that the words "first", "second", etc. do not limit the quantity and execution order, and the words "first", "second", etc. do not necessarily limit the differences.
[0094] The following describes the system architecture used in the embodiments of the present application.
[0095] See also Figure 2 , a schematic diagram of the architecture of the encoding and decoding system 10 used in the embodiment of the present application is given. Figure 2 As shown, the coding and decoding system 10 may include a source device 11 and a destination device 12. The source device 11 is used to encode an image, so the source device 11 may be referred to as an image encoding device or a video encoding device. The destination device 12 is used to decode the encoded image data generated by the source device 11, so the destination device 12 may be referred to as an image decoding device or a video decoding device.
[0096] Source device 30 and destination device 31 may include a variety of devices, including desktop computers, mobile computing devices, notebook (e.g., laptop) computers, tablet computers, set-top boxes, telephone handsets such as so-called "smart" phones, televisions, cameras, display devices, digital media players, video game consoles, in-vehicle computers, or the like.
[0097] Optionally, Figure 2 The source device 11 and the destination device 12 may be two separate devices, or the source device 11 and the destination device 12 may be the same device, that is, the source device 11 or the corresponding function and the destination device 12 or the corresponding function may be integrated on the same device.
[0098] The source device 11 and the destination device 12 may communicate with each other, for example, the destination device 12 may receive the encoded image data from the source device 11. In one example, one or more communication media may be included between the source device 11 and the destination device 12, and the encoded image data is transmitted through the one or more communication media, and the one or more communication media may include routers, switches, base stations, or other devices that facilitate communication from the source device 11 to the destination device 12.
[0099] like Figure 2 As shown, the source device 11 includes an encoder 112. Optionally, the source device 11 may also include an image preprocessor 111 and a communication interface 113. The image preprocessor 111 is used to perform preprocessing on the received image to be encoded. For example, the preprocessing performed by the image preprocessor 111 may include refurbishment, color format conversion (for example, from RGB format to YUV format), color adjustment or denoising, etc. The encoder 112 is used to receive the preprocessed image and process the preprocessed image using a related prediction mode to provide encoded image data. In some embodiments, the encoder 112 can be used to perform the encoding process in each embodiment described below. The communication interface 113 can be used to transmit the encoded image data to the destination device 12 or any other device (such as a memory) for storage or direct reconstruction. The other device can be any device for decoding or storage. The communication interface 113 can also encapsulate the encoded image data into a suitable format before transmission.
[0100] Optionally, the image preprocessor 111 , the encoder 112 , and the communication interface 113 may be hardware components in the source device 11 , or may be software programs in the source device 11 , which is not limited in the embodiment of the present application.
[0101] Continue as Figure 2 As shown, the destination device 12 includes a decoder 122. Optionally, the destination device 12 may also include a communication interface 121 and an image post-processor 123. The communication interface 121 may be used to receive encoded image data from the source device 11 or any other source device, such as a storage device. The communication interface 121 may also decapsulate the data transmitted by the communication interface 113 to obtain the encoded image data. The decoder 122 is used to receive the encoded image data and output the decoded image data (also referred to as reconstructed image data or reconstructed image data). In some embodiments, the decoder 122 may be used to perform the decoding process described in the various embodiments described below.
[0102] The image post-processor 123 is used to perform post-processing on the decoded image data to obtain post-processed image data. The post-processing performed by the image post-processor 123 may include: color format conversion (for example, from YUV format to RGB format), color adjustment, repair or resampling, or any other processing, and may also be used to transmit the post-processed image data to a display device for display.
[0103] Similarly, optionally, the communication interface 121, decoder 122 and image post-processor 123 may be hardware components in the destination device 12, or may be software programs in the destination device 12, which is not limited in the embodiment of the present application.
[0104] Below Figure 2 The structure of the encoder and decoder in is briefly introduced.
[0105] See also Figure 3 , Figure 3 1 is a schematic block diagram of an example of an encoder 20 for implementing an embodiment of the present application. Figure 3 In the embodiment, the encoder includes a prediction processing unit 201, a residual calculation unit 202, a residual coding unit 203, a quantization unit 204, a coefficient coding unit 205, an inverse quantization unit (also referred to as an inverse quantization unit) 206, a residual decoding unit 207, a reconstruction unit (or referred to as a reconstruction unit) 208, and a filter unit 209. Optionally, the encoder 20 may further include a buffer and a decoded image buffer, wherein the buffer is used to cache the reconstructed image block output by the reconstruction unit 208, and the decoded image buffer is used to cache the filtered image block output by the filter unit 209.
[0106] In one example, the input of the encoder 20 is an image block of the image to be encoded (i.e., a current image block, an image block to be encoded, or a coding unit. In another example, the input of the encoder 20 is an image to be encoded, and the encoder 20 may include a segmentation unit ( Figure 3 The encoder 20 is used to encode the image to be encoded block by block, for example, to perform the encoding process on each image block.
[0107] In an example, a method for dividing a to-be-encoded image into a plurality of image blocks may include:
[0108] Step 1: Divide a frame of image into one or more non-overlapping parallel coding units. There is no dependency between the parallel coding units and they can be encoded and decoded in parallel / independently.
[0109] Step 2: For each parallel coding unit, the encoding end may divide it into one or more independent coding units that do not overlap each other. The independent coding units may not depend on each other, but may share some header information of the parallel coding unit.
[0110] Step 3: For each independent coding unit, the encoding end may further divide it into one or more non-overlapping coding units. If the independent coding unit is divided into multiple non-overlapping coding units, the division method may be a horizontal equal division method, a vertical equal division method, or a horizontal and vertical equal division method. Of course, the specific implementation is not shown here. The coding units in the independent coding unit may be interdependent, that is, they may refer to each other during the process of executing the prediction step.
[0111] The width of the coding unit is w_cu and the height is h_cu. Optionally, the width is greater than the height (unless it is an edge area). Usually, the coding unit can be a fixed w_cu x h_cu, where w_cu and h_cu are both 2N powers (N is greater than or equal to 0), such as 16x4, 8x4, 16x2, 8x2, 4x2, 8x1, 4x1, etc.
[0112] The coding unit can include three components: luminance Y, chrominance Cb, and chrominance Cr (or red R, green G, and blue B), or only one of them. If it includes three components, the sizes of the components can be exactly the same or different, which is related to the image input format.
[0113] like Figure 4 , which is a schematic diagram of the correspondence between an image, a parallel coding unit, an independent coding unit, and a coding unit. Figure 4 In the description, an example is taken in which an image is divided into parallel coding unit 1 and parallel coding unit 2 according to 3:1, and an independent coding unit includes 4 coding units.
[0114] The prediction processing unit 201 is used to receive or obtain the real value of the image block and the reconstructed image data, and predict the current image block based on the relevant data in the reconstructed image data to obtain a prediction block of the current image block.
[0115] The residual calculation unit 202 is used to calculate the residual value between the real value of the image block and the prediction block of the image block to obtain the residual block, for example, by subtracting the pixel value of the prediction block from the pixel value of the image block pixel by pixel.
[0116] In one example, the residual coding unit 203 is used to determine the residual coefficient of the current image block based on the residual block. Optionally, in this process, it can include: performing a transformation such as discrete cosine transform (DCT) or discrete sine transform (DST) on the residual block to obtain a transformation coefficient in the transformation domain, and the transformation coefficient can also be called a transformation residual coefficient, and the transformation residual coefficient can represent the residual block in the transformation domain. Of course, the transformation step may not be included.
[0117] The quantization unit 204 is used to quantize the transform coefficients by applying scalar quantization or vector quantization to obtain quantized transform coefficients, which may also be referred to as quantized residual coefficients. The quantization process may reduce the bit depth associated with some or all of the transform coefficients. For example, an n-bit transform coefficient may be rounded down to an m-bit transform coefficient during quantization, where n is greater than m. The degree of quantization may be modified by adjusting a quantization parameter (QP). For example, for scalar quantization, different scales may be applied to achieve finer or coarser quantization. A smaller quantization step size corresponds to finer quantization, while a larger quantization step size corresponds to coarser quantization. The appropriate quantization step size may be indicated by the QP.
[0118] The coefficient coding unit 205 is used to encode the quantized residual coefficients and output the encoded image data (i.e., the encoding result of the current image block to be encoded) in the form of an encoded bit stream, and then the encoded bit stream can be transmitted to a decoder, or stored and subsequently transmitted to a decoder or used for retrieval. The coefficient coding unit 205 can also be used to encode other syntax elements of the current image block, such as encoding the index of the prediction mode into the bit stream, etc.
[0119] In one example, the coefficient coding unit 205 encodes the residual coefficient, and a feasible method is: semi-fixed length coding. First, the maximum value of the absolute value of the residual in a residual block (RB) is defined as modifiedmaximum (mm). Determine the number of coding bits for the residual coefficient in the RB (the number of coding bits for the residual coefficient in the same RB is consistent). For example, if the CL of the current RB is 2 and the current residual coefficient is 1, then 2 bits are required to encode the residual coefficient 1, which is expressed as 01. If the CL of the previous RB is 7, it means encoding an 8-bit residual coefficient and a 1-bit sign bit. The determination of CL is to find the minimum M value that satisfies that all residuals of the current sub-block are within the range of [-2^(M-1), 2^(M-1)]. If both -2^(M-1) and 2^(M-1) exist, M should be increased by 1, that is, M+1 bits are required to encode all residuals of the current RB; if only one of the two boundary values of -2^(M-1) and 2^(M-1) exists, a Trailing bit needs to be encoded to determine whether the boundary value is -2^(M-1) or 2^(M-1); if all residuals do not exist in either -2^(M-1) or 2^(M-1), there is no need to encode the Trailing bit.
[0120] Of course, other residual coefficient encoding methods may also be used, such as an exponential Golomb encoding method, a Golomb-Rice encoding method, a truncated unary code encoding method, a run-length encoding method, directly encoding the original residual value, and the like.
[0121] In addition, for some special cases, the original value can be directly encoded instead of the residual value.
[0122] The dequantization unit 206 is used to dequantize the quantized coefficients to obtain dequantized coefficients, where the dequantization is the reverse application of the quantization unit 204, for example, based on or using the same quantization step size as the quantization unit 204, and applying an inverse quantization scheme of the quantization scheme applied by the quantization unit 204. The dequantized coefficients may also be referred to as dequantized residual coefficients.
[0123] The residual decoding unit 207 is used to decode the above-mentioned inverse quantized residual coefficients into residual blocks. Optionally, the process may include: inverse transforming the above-mentioned inverse quantized coefficients. For example, the inverse transform may include inverse discrete cosine transform (discretecosine transform, DCT) or inverse discrete sine transform (discrete sine transform, DST) to obtain an inverse transform block in the pixel domain (or sample domain). The inverse transform block may also be called an inverse transform inverse quantized block or an inverse transform residual block. Of course, the inverse transform step may not be included.
[0124] The reconstruction unit 208 is used to add the inverse transform block (i.e., the inverse transform residual block) to the prediction block to obtain a reconstructed block in the sample domain. The reconstruction unit 208 can be a summer, for example, adding the sample value (i.e., pixel value) of the residual block to the sample value of the prediction block. The reconstructed block output by the reconstruction unit 208 can be subsequently used to predict other image blocks.
[0125] The filter unit 209 (or simply “filter”) is used to filter the reconstructed block to obtain a filtered block, so as to smoothly perform pixel conversion or improve image quality.
[0126] In one example, an encoding process such as Figure 5 As shown. Specifically, the encoder determines whether to adopt the point-by-point prediction mode. If so, prediction is performed based on the point-by-point prediction mode, and inverse quantization and reconstruction steps are performed to realize the encoding process; if not, it is determined whether the original value mode is adopted. If yes, the original value mode is adopted for encoding; if not, it is determined to adopt other prediction modes such as intra-frame prediction or block copy mode for prediction. Subsequently, when it is determined to perform residual skipping, the reconstruction step is directly performed; when it is determined not to perform residual skipping, the inverse quantization step is first performed, and it is determined whether the block copy prediction mode is adopted. If it is determined to adopt the block copy prediction mode, in one case, when it is determined to perform transform skipping, the reconstruction step is directly performed, and in another case, when it is determined that transform skipping is not performed, the encoding process is realized by performing inverse transform and reconstruction steps. If it is determined not to adopt the block copy mode (in this case, one of the prediction modes adopted is the intra-frame prediction mode), the inverse transform step and the reconstruction step are performed to realize the encoding process.
[0127] Specifically, in the embodiment of the present application, the encoder 20 is used to implement the encoding method described in the embodiments below.
[0128] See also Figure 6 , Figure 6 A schematic block diagram of an example of a decoder 30 for implementing an embodiment of the present application is shown. The decoder 30 is used to receive, for example, encoded image data (i.e., an encoded bit stream, for example, an encoded bit stream including an image block and associated syntax elements) encoded by the encoder 20 to obtain a decoded image. The decoder 30 includes a bit stream parsing unit 301, a dequantization unit 302, a residual decoding unit 303, a prediction processing unit 304, a reconstruction unit 305, and a filter unit 306. In some examples, the decoder 30 may perform operations substantially similar to those of the decoder 30. Figure 2 The decoding pass is the inverse of the encoding pass described by the encoder 20. Optionally, the decoder 30 may further include a buffer and a decoded image buffer, wherein the buffer is used to cache the reconstructed image block output by the reconstruction unit 305, and the decoded image buffer is used to cache the filtered image block output by the filter unit 306.
[0129] The code stream parsing unit 301 is used to decode the encoded image data to obtain quantized coefficients and / or decoded encoding parameters (for example, the decoded parameters may include any one or all of inter-frame prediction parameters, intra-frame prediction parameters, filter parameters and / or other syntax elements). The code stream parsing unit 301 is also used to forward the decoded encoding parameters to the prediction processing unit 304, so that the prediction processing unit performs a prediction process according to the encoding parameters.
[0130] The function of the inverse quantization unit 302 may be the same as that of the inverse quantization unit 206 of the encoder 20 , and is used to inverse quantize (ie, inverse quantize) the quantized coefficients decoded by the code stream parsing unit 301 .
[0131] The function of the residual decoding unit 303 may be the same as that of the residual decoding unit 207 of the encoder 20, and the function of the reconstruction unit 305 (e.g., a summer) may be the same as that of the reconstruction unit 208 of the encoder 20, and is used to inversely transform the above-mentioned quantized coefficients (e.g., inverse DCT, inverse integer transform, or a conceptually similar inverse transform process) to obtain an inverse transform block (also referred to as an inverse transform residual block), which is the residual block of the current image block in the pixel domain.
[0132] The prediction processing unit 304 is used to receive or obtain the encoded image data (for example, the encoded bit stream of the current image block) and the reconstructed image data. The prediction processing unit 301 can also receive or obtain prediction-related parameters and / or information about the selected prediction mode (that is, decoded encoding parameters) from, for example, the bit stream parsing unit 302, and predict the current image block based on the relevant data in the reconstructed image data and the decoded encoding parameters to obtain a prediction block of the current image block.
[0133] The reconstruction unit 305 is used to add the inverse transform block (ie, the inverse transform residual block) to the prediction block to obtain a reconstructed block in the sample domain, for example, adding the sample values of the inverse transform residual block to the sample values of the prediction block.
[0134] The filter unit 306 is configured to filter the reconstructed block to obtain a filtered block, where the filtered block is a decoded image block.
[0135] Specifically, in the embodiment of the present application, the decoder 30 is used to implement the decoding method described in the embodiments below.
[0136] It should be understood that in the encoder 20 and decoder 30 of the embodiment of the present application, the processing result of a certain link can also be output to the next link after further processing. For example, after the interpolation filtering, motion vector derivation or filtering and other links, the processing results of the corresponding link are further subjected to operations such as Clip or shift.
[0137] In one example, a decoding process may include the following steps:
[0138] Step 1: The code stream parsing unit 301 parses the prediction mode and the residual coding mode;
[0139] Step 2: The code stream parsing unit 301 parses quantization related values (such as near values, or QP values, etc.) based on the prediction mode and the residual coding mode;
[0140] Step 3: The inverse quantization unit 302 analyzes the residual coefficients based on the prediction mode and the quantization correlation value;
[0141] Step 4: The prediction processing unit 304 obtains the prediction value of each pixel of the current image block based on the prediction mode;
[0142] Step 5: The residual value decoding unit 303 obtains the residual value of each pixel of the current image block based on the residual coefficient decoding;
[0143] Step 6: The reconstruction unit 305 obtains the reconstructed value of each pixel in the current coding unit based on the predicted value and residual value of each pixel.
[0144] Below, in conjunction with the accompanying drawings, the embodiment of the present application provides the following image encoding and decoding method for illustration.
[0145] It should be noted that the encoding end in any embodiment of the present application may be the above Figure 2 or Figure 3 The encoder in can also be the above Figure 2 The source device in any embodiment can be the above-mentioned decoding end Figure 2 or Figure 6 The decoder in can also be the above Figure 2 The destination device in.
[0146] Embodiment 1
[0147] like Fig. 7A The figure is a flow chart of an image encoding method provided in an embodiment of the present application. Fig. 7A The method shown includes the following steps:
[0148] S101: The encoding end determines a target prediction mode of a unit to be encoded.
[0149] The unit to be encoded may be any image block to be encoded (ie, encoding unit) in the image to be encoded.
[0150] The target prediction mode may be one of a plurality of predefined prediction modes, specifically, an original value mode, or other prediction modes except the original value mode.
[0151] In one example, the other prediction modes include: any one or more combinations of at least one point-by-point prediction mode, at least one intra-frame prediction mode, and at least one block copy mode. For example, the other modes include three prediction modes, namely, point-by-point prediction mode, intra-frame prediction mode, and block copy mode. For another example, the other modes include five prediction modes, namely, point-by-point prediction mode 0, point-by-point prediction mode 1, intra-frame prediction mode 0, block copy mode 0, and block copy mode 1.
[0152] In another example, the other prediction modes include: at least one point-by-point prediction mode based on normal residual coding, at least one intra-frame prediction mode, at least one block copy mode, at least one point-by-point prediction mode based on skip residual coding, at least one intra-frame prediction mode, and any one or more combinations of at least one block copy mode. This example can be considered as combining the prediction mode in the traditional technology with the residual coding mode to obtain a new prediction mode defined in the embodiment of the present application.
[0153] When the prediction mode is a point-by-point prediction mode based on normal residual coding, the encoder uses the normal residual coding mode to encode residual coefficients and uses the point-by-point prediction mode to predict the pixel value of the coding unit. When the prediction mode is the other modes, it is similar to this and will not be described one by one.
[0154] S102: The encoding end determines the index of the target prediction mode from an index table based on the target prediction mode; wherein the index table includes the corresponding relationship between the indexes of multiple prediction modes and the multiple prediction modes.
[0155] Optionally, the correspondence between multiple prediction modes and their respective indexes may be stored via an index table, and may also be stored via other methods, which is not limited in the embodiments of the present application.
[0156] Optionally, the encoding end jointly encodes the indexes of the multiple prediction modes.
[0157] For example, the indexes of the multiple prediction modes are encoded in a truncated unary code manner, that is, the indexes of the multiple prediction modes are generated in a truncated unary code manner, as can be seen from the following Examples 1 and 2.
[0158] For another example, the indexes of the multiple prediction modes are encoded in a binary tree manner, as can be seen in Example 3 below.
[0159] Optionally, the codeword length (ie, the number of binary characters, also referred to as code length) of the index of the original value mode is greater than or equal to the codeword length of the index of other prediction modes.
[0160] Optionally, the codeword length of the index of the prediction mode is related to the frequency of use of the prediction mode. The frequency of use here can be obtained based on big data statistics, and the statistical method is not limited in the embodiment of the present application. For example, the more frequently used prediction mode is, the shorter the codeword of its index is. Of course, for the multiple prediction modes, there may be some prediction modes whose indexes satisfy the rules shown in the optional implementation method. Further optionally, there may be multiple prediction modes whose indexes have the same codeword length.
[0161] S103: The encoding end encodes the index of the target prediction mode into the bitstream.
[0162] like Figure 7B As shown, an image decoding method provided by an embodiment of the present application is shown. The image decoding method is similar to Fig. 7A The image encoding method shown corresponds to. Figure 7B The methods shown include:
[0163] S201: The decoding end parses the bitstream to obtain a syntax element, where the syntax element includes an index of a target prediction mode of a unit to be decoded.
[0164] The unit to be decoded can be Fig. 7A The unit to be encoded in the method shown is obtained after encoding.
[0165] Optionally, the encoding end jointly encodes the indexes of the multiple prediction modes, for example, encodes the indexes of the multiple prediction modes in a truncated unary code manner, that is, the indexes of the multiple prediction modes are generated in a truncated unary code manner.
[0166] Optionally, the codeword length of the index of the original value mode is greater than or equal to the codeword length of the index of other prediction modes.
[0167] Optionally, the codeword length of the index of the prediction mode is related to the usage frequency of the prediction mode. For example, the more frequently used the prediction mode is, the shorter the codeword of its index is.
[0168] S202: The decoding end determines the target prediction mode from an index table based on the index of the target prediction mode; wherein the index table includes indexes of multiple prediction modes and corresponding relationships between the multiple prediction modes.
[0169] S203: The decoding end reconstructs the unit to be decoded based on at least the target prediction mode to obtain a reconstructed block of the unit to be decoded.
[0170] In one example, the decoding end predicts the pixel values of the unit to be decoded based on the target prediction mode to obtain a prediction block. The bitstream is parsed to obtain information related to the residual block of the unit to be decoded, and the residual block of the unit to be encoded is obtained based on the information. Then, the unit to be decoded is reconstructed based on the prediction block and the residual block to obtain a reconstructed block.
[0171] The decoding end reconstructs the decoding unit based on at least the target prediction mode, but is not limited thereto. In another example, the decoding end may be based on the following Fig.21 The method in reconstructs the unit to be decoded.
[0172] It should be noted that in traditional technology, at the encoding end, an additional bit is usually used to indicate whether the prediction mode of the current image block (i.e., the unit to be encoded) is the original value mode (for example, this bit is recorded as org_flag, if org_flag is "1", it means that the prediction mode of the current block is the original value mode, if org_flag is "0", it means that the prediction mode of the current block is not the original value mode). If it is not the original value mode, the index of other prediction modes is encoded in the bitstream. However, since the probability of adopting the original value mode is very low, for most image blocks, 1 0 needs to be encoded in the bitstream, which will cause a waste of transmission resources (i.e., increase the bitstream transmission overhead), and regardless of whether the prediction mode is the original value prediction mode, the decoding end needs to decode this marker bit org_flag first, which increases the decoding complexity of the decoding end.
[0173] In the image encoding method shown in FIG. 7 provided in the embodiment of the present application, the index of the target prediction mode is determined directly from the index table based on the target prediction mode, and the process may not include in the code stream the index of whether the target prediction mode is the original value (such as the flag bit org_flag). Since in most cases, the target prediction mode does not use the original value mode, for example, the point-by-point prediction mode or the intra-frame prediction mode is usually used, therefore, in most cases, there is no need to encode the flag bit (org_flag) representing whether the original value mode is used in the traditional scheme, which helps to save code stream transmission overhead.
[0174] Accordingly, in the embodiments of the present application Fig.10 In the provided image decoding method, the target prediction mode can be determined directly from the index table based on the index of the target prediction mode without parsing the flag bit org_flag, thereby reducing the decoding complexity of the decoding end.
[0175] Figure 7B The image decoding method shown is similar to Fig. 7A The image encoding method shown corresponds to, therefore, the image decoding method helps to save code stream transmission overhead.
[0176] Example 1: Assuming that the above-mentioned multiple prediction modes include the original value mode, point prediction mode, intra-frame prediction mode and block copy mode, then a feasible method is to encode the indexes of these prediction modes in the form of truncated unary codes, and the encoding of the index of the original value mode uses a longer codeword, and the indexes of other prediction modes use a shorter codeword. For example: the codeword of the index of the point-by-point prediction mode is 0, the codeword of the intra-frame prediction mode is 10, the codeword of the block copy mode is 111, and the codeword of the index of the original value mode is 110. Based on this scheme, since in most cases, the point-by-point prediction mode or the intra-frame prediction mode is used, there is no need to encode the flag bit (org_flag) in the traditional scheme that represents whether the original value mode is used.
[0177] Based on this example, the index table can be shown as Table 1:
[0178] Table 1: Index table encoded in the truncated unary code mode
[0179]
[0180] Example 2: Assume that the above multiple prediction modes include: original value mode, point-by-point prediction mode based on normal residual coding, intra-frame prediction mode, block copy mode, point-by-point prediction mode based on skip residual coding, intra-frame prediction mode, block copy mode, and the point-by-point prediction mode, intra-frame prediction mode, and block copy mode based on normal residual coding mode are recorded as: ordinary point-by-point prediction mode (mode 1), ordinary intra-frame prediction mode (mode 2), and ordinary block copy mode (mode 3); the point-by-point prediction mode, intra-frame prediction mode, and block copy mode based on skip residual coding mode are recorded as: special point-by-point prediction mode (mode 4), special intra-frame prediction mode (mode 5), and special block copy mode (mode 6). Including the original value mode (mode 7), there are 7 prediction modes in total. Then, a feasible method is to encode the indexes of these prediction modes in the form of truncated unary codes, and the codewords of modes 1-7 are: 0, 10, 110, 1110, 11110, 111110, 111111.
[0181] Based on this example, the index table may be as shown in Table 2:
[0182] Table 2: Index table encoded in the truncated unary code mode
[0183]
[0184] Example 3: Encoding the indexes of multiple prediction modes in a binary tree manner. Table 3 shows an encoding method.
[0185] Table 3: Index table encoded in binary tree format
[0186]
[0187] Table 3 takes multiple prediction modes including the above 15 prediction modes as an example for explanation. In actual implementation, the index table shown in Table 3 may only include two columns, prediction mode and prediction mode index, or three columns, prediction mode number, prediction mode and prediction mode index.
[0188] The indexes of various prediction modes in Table 3 are generated according to the following rules:
[0189] First, "0" and "1" are respectively marked as the highest bit of the indexes of "point prediction mode, intra block copy mode" and "intra prediction mode, original value mode".
[0190] Secondly, for "point prediction mode, intra block copy mode", "0" and "1" are respectively used as the second highest flags of the indexes of "point prediction mode" and "intra block copy mode".
[0191] Next, for the point prediction mode, "1" and "0" are used as the third flag bit of the index of "point prediction mode 0-1" and "point prediction mode 2-3" respectively.
[0192] Finally, for “point prediction mode 0-1”, “0” and “1” are used as the fourth flag bit of the index of “point prediction mode 0” and “point prediction mode 1” respectively.
[0193] Thus, the indexes of point prediction mode 0 and point prediction mode 1 are respectively “0010” and “0011”. The indexes of other prediction modes are generated in a similar manner and will not be described in detail here.
[0194] For frame prediction modes 0-6, "1" and "0" are used as the second highest bit of the indexes of "frame prediction mode 0" and "frame prediction mode 1-6" respectively. The indexes of other frame prediction modes are not described one by one. The index generation method for the intra-frame block copy mode is similar to this.
[0195] Optionally, the code length of the index of a prediction mode is related to the usage frequency of the prediction mode. In one example, referring to Table 3, considering that the usage frequencies of point prediction modes 0-3 are roughly the same, the code lengths of their indexes are set to be the same; considering that the usage frequency of intra-frame prediction mode 0 is higher than that of intra-frame prediction modes 1-6, the code length of its index is less than the usage frequency of the codewords of intra-frame prediction modes 1-6. Other examples are not listed one by one.
[0196] Optionally, the index of one prediction mode is not used as a prefix of the indexes of other prediction modes. For example, the index for prediction mode 0 is "11", and the prefixes of the indexes of the other 14 prediction modes are not "11". This is a technical solution proposed based on the consideration that "if the prefix of the index of one or more prediction modes is "11", when the decoding end recognizes "11", it cannot determine whether the prediction mode is prediction mode 0 or one or more prediction modes", which helps the decoding end to decode quickly.
[0197] It should be noted that the specific implementation method of generating indexes of multiple prediction modes based on a binary tree is not limited to Table 3.
[0198] For example, in Table 3, the code lengths of the indexes of point prediction modes 0-3 are equal. In specific implementation, the code lengths of the indexes of any two point prediction modes may not be equal, such as the generation method of the indexes of frame prediction modes 0-6.
[0199] For example, the code lengths of the indexes of intra prediction modes 0-6 may also be equal, such as the generation method of the indexes of reference point prediction modes 0-3.
[0200] For example, Table 3 uses "0" and "1" as the highest bit marks of the indexes of "point prediction mode, intra-frame block copy mode" and "intra-frame prediction mode, original value mode" respectively; in specific implementation, "1" and "0" can also be used as the highest bit marks of the indexes of "point prediction mode, intra-frame block copy mode" and "intra-frame prediction mode, original value mode" respectively.
[0201] Other examples are not listed one by one.
[0202] Embodiment 2
[0203] like Fig. 8A , which is a schematic diagram of an image encoding method provided in an embodiment of the present application. Fig. 8A The method shown comprises the following steps:
[0204] S301: The encoding end determines a residual coding mode of a unit to be encoded, wherein the residual coding mode may include a skip residual coding mode or a normal residual coding mode.
[0205] For the relevant explanations about the unit to be encoded, the skip residual encoding mode and the normal residual encoding mode, please refer to the above and will not be repeated here.
[0206] S302: If the residual coding mode is the skip residual coding mode, the encoder encodes the index of the residual coding mode into the bitstream. In this case, it is not necessary to encode the residual quantization related value (such as near value or QP value, etc.) of the unit to be encoded into the bitstream.
[0207] S303: If the residual coding mode is a normal residual coding mode, the encoder encodes the index of the residual coding mode and the residual quantization related value (such as a near value or a QP value, etc.) of the unit to be coded into the bitstream. In one example, the encoder first encodes the index of the residual coding mode and then encodes the residual quantization related value of the unit to be coded.
[0208] S304: If the residual coding mode is the skip residual coding mode, the encoding end predicts the unit to be coded to obtain a prediction block, and determines the prediction block as a reconstructed block of the unit to be coded. In this case, it is not necessary to obtain the residual quantization related value of the unit to be coded.
[0209] S305: If the residual coding mode is a normal residual coding mode, the encoder obtains the residual quantization correlation value of the unit to be coded to obtain a residual block; and reconstructs the unit to be coded based on the residual block to obtain a reconstructed block of the unit to be coded.
[0210] Among them, S304-S305 can be considered as a reconstruction process performed by the encoding end.
[0211] like Figure 8B FIG. 1 is a schematic diagram of an image decoding method provided by an embodiment of the present application. The image decoding method is similar to the Fig. 8A The image encoding method shown corresponds to. Figure 8B The method shown comprises the following steps:
[0212] S401: The decoding end parses the bitstream to obtain a syntax element. The syntax element includes an index of the residual coding mode of the unit to be decoded. The unit to be decoded may be Fig. 8A The method shown encodes the image block obtained by encoding the unit to be encoded.
[0213] S402: If the index of the residual coding mode is used to indicate skipping the residual coding mode, the decoding end predicts the unit to be decoded to obtain a prediction block, and determines the prediction block as a reconstructed block of the unit to be decoded.
[0214] Since the residual coding mode is a skip residual coding mode, that is, the residual values of the units to be decoded are all 0, the predicted block can be directly used as a reconstructed block.
[0215] S403: If the index of the residual coding mode is used to indicate the normal residual coding mode, the decoding end parses the residual quantization correlation value of the unit to be decoded to obtain a residual block; and reconstructs the unit to be decoded at least based on the residual block to obtain a reconstructed block.
[0216] In one example, the syntax element may further include a prediction mode of the unit to be decoded, and the decoding end predicts the unit to be decoded based on the prediction mode to obtain a prediction block, and reconstructs the unit to be decoded based on the residual block and the prediction block to obtain a reconstructed block. The process of reconstructing the unit to be decoded based on at least the residual block is not limited thereto. Fig.21 The reconstruction is performed by the method shown to obtain a reconstructed block.
[0217] It should be noted that in the traditional technology, a flag bit (referred to as res_skip_flag) is usually used to indicate whether the current block adopts the skip residual coding mode, and the encoding (decoding) of the residual quantization related value is before res_skip_flag, so even if res_skip_flag is 1, the residual quantization related value needs to be encoded (decoded). At this time, the residual quantization related value is redundant, so this technical solution not only increases the code stream transmission overhead between the encoding end and the decoding end, but also increases the decoding complexity of the decoding end.
[0218] The embodiment of the present application Fig. 8A In the provided image coding method, considering that "if it is a skip residual coding mode (such as res_skip_flag is 1), there is no need to encode or decode the residual coefficient, so the residual quantization related values (such as near values, or QP values, etc.) do not need to be encoded (decoded)", it is proposed to first encode the residual coding mode information (such as the index of the residual coding mode, such as res_skip_flag) at the encoding end, and then determine whether to encode the residual quantization related values (such as near values, or QP values) based on the residual coding mode. Compared with traditional technologies, when the residual coding mode is a skip residual coding mode, there is no need to encode the residual quantization related values, which helps to save bitstream transmission overhead.
[0219] Accordingly, in the embodiments of the present application Figure 8B In the provided image decoding method, the residual coding mode information (such as the index of the residual coding mode, such as res_skip_flag) can be parsed first, and then based on the residual coding mode, it is determined whether to parse the residual quantization related value (such as the near value, or the QP value), thereby reducing the decoding complexity of the decoding end.
[0220] Embodiment 3
[0221] like Fig. 9A , which is a schematic diagram of an image encoding method provided in an embodiment of the present application. Fig. 9A The method shown comprises the following steps:
[0222] S501: The encoding end determines a target prediction mode of a unit to be encoded, wherein the target prediction mode includes a prediction mode based on skipped residual encoding or a prediction mode based on normal residual encoding.
[0223] The prediction mode based on skipping residual coding and the prediction mode based on normal residual coding can be considered to be a combination of the prediction mode in the traditional technology and the residual coding mode, so as to obtain a new prediction mode defined in the embodiment of the present application.
[0224] In an example, the point-by-point prediction mode based on skipping residual coding includes: a point-by-point prediction mode based on skipping residual coding, an intra-frame prediction mode, or a block copy mode.
[0225] In an example, the prediction mode based on normal residual coding includes: a point-by-point prediction mode, an intra-frame prediction mode or a block copy mode based on normal residual coding.
[0226] S502: The encoder encodes the index of the target prediction mode into the bitstream.
[0227] In one example, the method for generating the index of the target prediction mode can refer to Fig. 7A In the embodiment shown, the indexes of the multiple prediction modes are jointly encoded, for example, the indexes of the multiple prediction modes are encoded in a truncated unary code manner. Of course, the specific implementation is not limited thereto. In another example, the index of the target prediction mode can be encoded into the bitstream in a traditional manner, such as first encoding the flag bit org_flag.
[0228] S503: If the target prediction mode is a prediction mode based on skipping residual coding, the encoding end predicts the unit to be encoded to obtain a prediction block, and determines the prediction block as a reconstructed block of the unit to be encoded.
[0229] S504: If the target prediction mode is a prediction mode based on normal residual coding, the encoder obtains the residual quantization related value of the unit to be coded, obtains the residual block of the unit to be coded, and reconstructs the unit to be coded based on the target prediction mode and the residual block to obtain a reconstructed block.
[0230] Among them, S503-S504 can be considered as a reconstruction process performed by the encoding end.
[0231] In one example, reconstructing the coding unit based on the target prediction mode and the residual block may include: predicting the coding unit based on the target prediction mode to obtain a prediction block, and reconstructing the coding unit based on the prediction block and the residual block. Of course, not limited to this, in another example, based on Fig.21 The reconstruction is performed by the method shown to obtain a reconstructed block.
[0232] like Fig. 9B FIG. 1 is a schematic diagram of an image decoding method provided by an embodiment of the present application. The image decoding method is similar to the Fig. 9A The image encoding method shown corresponds to. Fig. 9B The method shown comprises the following steps:
[0233] S601: The decoding end parses the bitstream to obtain a syntax element; the syntax element includes an index of a target prediction mode of the unit to be decoded, wherein the target prediction mode includes an index of a prediction mode based on skip residual coding or an index of a prediction mode based on normal residual coding.
[0234] S602: If the target prediction mode is a prediction mode based on skipping residual coding, the decoding end predicts the unit to be decoded to obtain a prediction block, and determines the prediction block as a reconstructed block.
[0235] S603: If the target prediction mode is a prediction mode based on normal residual coding, the decoding end parses the residual quantization correlation value of the unit to be decoded to obtain the residual block of the unit to be decoded, and reconstructs the unit to be decoded based on the target prediction mode and the residual block to obtain a reconstructed block.
[0236] In one example, reconstructing the unit to be decoded based on the target prediction mode and the residual block may include: predicting the unit to be decoded based on the target prediction mode to obtain a prediction block, and reconstructing the unit to be decoded based on the prediction block and the residual block. Of course, not limited to this, in another example, based on Fig.21 The reconstruction is performed by the method shown to obtain a reconstructed block.
[0237] Fig. 9A and Fig. 9B The image encoding and decoding method shown has at least the following beneficial effects:
[0238] On the one hand, the traditional residual coding mode is combined with the prediction mode as a new prediction mode provided in the embodiment of the present application. In this way, there is no need to encode (decode) the residual coding mode and the prediction mode separately. It is only necessary to uniformly encode (decode) the new prediction mode. The implementation logic is simple and helps to save the code stream transmission overhead. In other words, the encoding end can encode the residual skip mode as a prediction mode. The encoding end (decoding end) first encodes (decodes) the prediction mode, and then determines whether it is necessary to parse the residual quantization related value (such as the near value, or QP value) based on the prediction mode to save the bit rate.
[0239] On the other hand, the residual coding mode information (such as the index of the residual coding mode, such as res_skip_flag) is first encoded (or decoded), and then based on the residual coding mode, it is determined whether to encode (or decode) the residual quantization-related value (such as the near value, or the QP value). Compared with the traditional technology, when the residual coding mode is the skip residual coding mode, there is no need to encode (or decode) the residual quantization-related value, which helps to save the bitstream transmission overhead.
[0240] For example, based on the encoding method shown in Example 2 above, at the decoding end, based on the prediction mode, it can be known whether the to-be-encoded unit corresponding to the to-be-decoded unit needs to encode the residual. For modes that do not require encoding the residual (such as the above modes 4, 5, 6, and 7), there is no need to encode (decode) the residual quantization related value (such as the near value, or QP value).
[0241] Embodiment 4
[0242] like Fig.10 As shown, an image reconstruction method provided by an embodiment of the present application can be applied to an image encoding method or an image decoding method. Fig.10 The method shown may include the following steps:
[0243] S701: Determine a prediction mode of at least one prediction group into which the current image block is divided. Any one of the at least one prediction group includes a plurality of continuous pixels in the same row; the any one prediction group includes a first designated pixel region and a second designated pixel region, the first designated pixel region includes a plurality of pixel groups, the plurality of pixel groups are obtained by dividing the second designated pixel region, and the prediction modes of the first designated pixel region and the second designated pixel region are different. The plurality of pixel groups can be predicted in parallel, and one pixel group includes one or a plurality of continuous pixels.
[0244] When this embodiment is applied to an image coding method, the execution subject of this embodiment may be a coding end, and the current image block is specifically an image block to be coded or a unit to be coded.
[0245] When this embodiment is applied to an image decoding method, the execution subject of this embodiment may be a decoding end, and the current image block may specifically be an image block to be decoded or a unit to be decoded. S701 may include: parsing a bitstream to obtain an index of a prediction mode of any one or more prediction groups in at least one prediction group into which the current image block is divided.
[0246] The embodiment of the present application does not limit the size of the current image block, for example, the width of the current image block is greater than the height. For example, the size of the current image block is 16*2, 8*2 or 8*1.
[0247] In one example, the current image block may be divided into a prediction group, which is equivalent to not dividing the current image block into prediction groups. In this case, all pixels in the current image block are used as a prediction group.
[0248] In another example, the current image block may be divided into a plurality of prediction groups.
[0249] The embodiment of the present application does not limit the division method of the prediction groups of the current image block and the prediction mode of each prediction group. Hereinafter, the division mode and the prediction mode based on the division mode are collectively referred to as "division prediction mode".
[0250] In an example, a prediction group may be composed of all / part of the pixels in one row / multiple rows of the current image block. Of course, other implementations are also possible. In principle, a prediction group only needs to include multiple consecutive pixels in the same row.
[0251] In one example, the number of pixels of the continuous multiple pixels included in a prediction group is greater than or equal to a preset threshold. This is because when the number of continuous multiple pixels in a prediction group is large, if the pixels in the prediction group adopt serial prediction, it may cause the prediction of the prediction group to take a long time. In this case, if this solution is adopted, multiple pixel groups in a prediction group can be predicted in parallel, which helps to shorten the prediction time of the prediction group.
[0252] If there is no dependency between the pixels in two pixel groups during the prediction process, the two pixel groups can be predicted in parallel. The number of pixels contained in different pixel groups can be equal or unequal.
[0253] The embodiments of the present application do not limit which pixels the first designated pixel area and the second designated pixel area specifically include, nor do they limit the prediction method for these pixels. In principle, any type of "first designated pixel area and second designated pixel area, and prediction method" may be used as long as "multiple pixel groups can be predicted in parallel, and the pixel group includes one or multiple consecutive pixels" is satisfied.
[0254] like Fig.11 As shown, a method for dividing parallel pixel groups provided in an embodiment of the present application is provided. Fig.11 In the example, the continuous pixels in the same row in a prediction group are pixels 1-12, the pixels included in the second designated pixel region are pixels 3, 7, and 11, and the pixels included in the first designated pixel region are pixels 1-2, 4-6, 8-10, and 12. The multiple pixel groups are pixel groups 1-4, wherein pixel groups 1-3 each include multiple pixels, and pixel group 4 includes 1 pixel. The number of pixels included in pixel group 2 is equal to that in pixel group 3, but not equal to that in other pixel groups.
[0255] In one example, the first designated pixel region uses vertical prediction, and the second designated pixel region uses horizontal prediction. In this way, parallel prediction between multiple pixel groups can be achieved.
[0256] In another example, the first designated pixel region adopts vertical prediction, and the second designated pixel region adopts vertical mean prediction. In this way, parallel prediction between multiple pixel groups can be achieved.
[0257] Optionally, the difference in the number of pixels included in different pixel groups is equal or less than a threshold value. This helps to achieve that the number of pixels included in multiple pixel groups is equal or as equal as possible, so that the prediction durations of different pixel groups predicted in parallel are the same or approximately the same, thereby shortening the total prediction duration of a prediction group.
[0258] S702: Reconstruct the current image block based on the prediction mode of the at least one prediction group to obtain a reconstructed block.
[0259] For example, for any one of the at least one prediction group, the prediction group is predicted based on the prediction mode of the prediction group to obtain a prediction block of the prediction group; and a residual block of the prediction group is obtained; then, based on the prediction block and the residual block, the reconstructed block is obtained. Of course, there may be other implementations, for example, when all values in the residual block are 0, the prediction block is directly used as the reconstructed block. For example, based on Fig.21 The reconstruction is performed by the method shown to obtain a reconstructed block.
[0260] Fig.11 In the image reconstruction method shown, when performing prediction, multiple pixel groups in a prediction group can be predicted in parallel, which helps to shorten the total prediction time of the prediction group. Fig.11 The image decoding method shown can be considered as a new point-by-point prediction mode provided by the embodiment of the present application.
[0261] Scalable, Fig.11 The "row" in the image reconstruction method shown can be replaced by "column", thereby forming a new embodiment to solve the problem that the prediction process takes a long time because the prediction process cannot be performed in parallel in the current image block whose height is greater than the width (for example, the size of the current image block is 2*16, 2*8 or 1*8, etc.). In this case, in one example, the first designated pixel area adopts horizontal prediction and the second designated pixel area adopts vertical prediction. In another example, the first designated pixel area adopts horizontal prediction and the second designated pixel area adopts horizontal average prediction. Other related embodiments can be based on the present application. Fig.11 The relevant description can be inferred and will not be repeated here.
[0262] The following is a specific example of Fig.11 The image reconstruction method shown is described:
[0263] For the 16x2 current image block, it may be a 16x2 luminance block or a 16x2 chrominance block.
[0264] 1) A prediction partitioning mode of a 16x2 current image block is as follows Fig.12 As shown, the current image block includes Group1-2. Specifically:
[0265] Group 1 consists of the second row of pixels and uses horizontal prediction.
[0266] Group 2 consists of the first row of pixels and uses vertical mean prediction. The pixels of group 2 can refer to the last row of pixels of the upper adjacent 16x2 image block.
[0267] like Fig.12 As shown in the figure, the encoding of the second pixel in group2 depends on the reconstructed value of the first pixel, so the encoding of the first pixel and the second pixel cannot be performed in parallel, and the decoding is similar. In addition, the encoding of the first pixel in group1 depends on the reconstructed value of the first pixel in group2, so encoding or decoding cannot be performed at the same time. It can be seen that this prediction partitioning mode has the problem of low parallelism, which is not conducive to hardware implementation.
[0268] In order to improve the parallelism in the prediction process, the following examples are used in this application: Fig.11 In the embodiment shown, the prediction partitioning mode of the current image block of 16x2 can be as follows: Fig.13A or Fig. 13B As shown, Fig.13A and Fig. 13B The prediction group division mode of the current image block shown is the same as Fig.12 Same, but different prediction modes. Specifically:
[0269] Any 16x2 current image block can refer to the values of the left column (2 pixels) and the top row (16 pixels). The leftmost pixel of the 16x2 current image block is called the first pixel. Then:
[0270] Solution 1: Fig.13A As shown, the prediction modes include:
[0271] a) For the kN+1th (N is preferably 4, k=1, 2 or 3) pixel of Group1 and Group2, the reconstructed value of the pixel above it is used as the predicted value of the current pixel.
[0272] b) Among the pixels of Group 2, except for the kN+1th pixel (N is preferably 4, k=1, 2 or 3), the other pixels use the reconstructed value of the pixel on its left as the predicted value of the current pixel.
[0273] c) Optionally, for pixels in Group1, except for the kN+1th pixel (N is preferably 4, k=1, 2 or 3), the predicted values of other pixels are preset values (i.e., default values). Optionally, the other pixels use the average of the reconstructed values of the pixels above them (the reconstructed values of other adjacent blocks above, denoted as P0) and the reconstructed values of the pixels below them (the reconstructed value of group2 is denoted as P1) (e.g., (P0+P1+1)>>1) as the predicted value of the current pixel. Among them, ">>" indicates an integer right shift. "a>>N" means a is shifted right by N bits, similar to a divided by 2 to the Nth power, so ">>1" is equivalent to dividing by 2.
[0274] Solution 2: Fig. 13B As shown, the prediction modes include:
[0275] d) For the kN+2th (N is preferably 5, k=1, 2) pixel of Group 1 and Group 2, the reconstructed value of the pixel above it is used as the predicted value of the current pixel.
[0276] e) Among the pixels of Group 2, except for the kN+2th pixel (N is preferably 5, k=1, 2), the other pixels use the reconstructed value of the pixel on its left as the predicted value of the current pixel.
[0277] f) Optionally, for pixels in Group1, except for the kN+2th (N is preferably 5, k=1, 2) pixel, the other pixels use the average of the reconstructed value of the pixel above it (the reconstructed value of other adjacent blocks above, denoted as P0) and the reconstructed value of the pixel below it (the reconstructed value of group2 is denoted as P1) (such as: (P0+P1+1)>>1, shifted right by one position, that is, divided by 2) as the predicted value of the current pixel.
[0278] 2) A prediction partitioning mode of a 16x2 current image block is as follows Fig.14 As shown, the current image block includes Group1-2. Specifically:
[0279] Group 1 consists of the first row of pixels and uses horizontal prediction.
[0280] Group 2 consists of the second row of pixels and uses horizontal prediction.
[0281] In order to improve the parallelism in the prediction process, the following examples are used in this application: Fig.11 In the embodiment shown, a prediction partitioning mode of a 16x2 current image block is as follows: Figures 15A-15D As shown, Figures 15A-15D The prediction group division mode of the current image block shown is the same as Fig.14 Same, but different prediction modes. Specifically:
[0282] Solution 1: Fig.15AAs shown, the prediction modes include:
[0283] a) For the kN+1th (N is preferably 4, k=1, 2, 3) pixel of Group1 and Group2, the reconstructed value of the pixel above it is used as the predicted value of the current pixel.
[0284] b) For other pixels in Group 1 and Group 2, the reconstructed value of the pixel on its left is used as the predicted value of the current pixel.
[0285] Solution 2: Fig. 15B As shown, the prediction modes include:
[0286] The kN+1th (N is preferably 4, k=2, 3) pixel of Group1 and Group2 uses the reconstructed value of the pixel above it as the predicted value of the current pixel. The other pixels use the reconstructed value of the pixel on the left as the predicted value of the current pixel.
[0287] Solution 3: Fig. 15C As shown, the prediction modes include:
[0288] The kN+2th (N is preferably 5, k=1, 2) pixel of Group1 and Group2 uses the reconstructed value of the pixel above it as the predicted value of the current pixel. The other pixels use the reconstructed value of the pixel on the left as the predicted value of the current pixel.
[0289] Solution 4: Fig.15D As shown, the prediction modes include:
[0290] The kN+1th (N is preferably 8, k=1) pixel of Group1 and Group2 uses the reconstructed value of the pixel above it as the predicted value of the current pixel. The other pixels use the reconstructed value of the pixel on the left as the predicted value of the current pixel.
[0291] It should be noted that in the above embodiments 1-2, N and k may take other values. Furthermore, when the number of consecutive pixels in the prediction group is different, the values of N and k may also be different accordingly.
[0292] It should also be noted that Figure 13A-13B , Figures 15A-15D The reconstructed values of the current image block surrounding the current image block of the pixels marked with black shadows may be set to default values if unavailable.
[0293] For the 8x2 current image block, it can be an 8x2 luminance block or an 8x2 chrominance block.
[0294] A group prediction mode for the current image block of 8x2 is as follows Fig.16As shown. Among them, any 8x2 current image block can refer to the reconstruction values of the pixels in the left column (2 pixels) and the upper row (8 pixels). Then:
[0295] like Fig.16 As shown in Figure a, in prediction partition mode A, all pixels use the reconstructed value of the upper pixel as the prediction value of the current pixel.
[0296] like Fig.16 As shown in Figure b, in prediction partition mode B, all pixels use the reconstructed value of the pixel on the left as the prediction value of the current pixel.
[0297] In order to improve the parallelism in the prediction process, the following examples are used in this application: Fig.11 In the embodiment shown, a prediction partitioning mode of a current image block of 8x2 is as follows: Fig.17 shown.
[0298] exist Fig.17 In the example, the kN+1th (N is preferably 4, k=1) pixel of Group1 and Group2 uses the reconstructed value of the pixel above it as the predicted value of the current pixel. The other pixels use the reconstructed value of the pixel on the left as the predicted value of the current pixel.
[0299] For the 8x1 current image block, it can be an 8x1 luminance block or an 8x1 chrominance block.
[0300] A group prediction mode for the current image block of 8x1 is as follows Fig.18 As shown. Among them, any 8x1 current image block can refer to the reconstruction values of the pixels in the left column (1 pixel) and the upper row (8 pixels). Then:
[0301] like Fig.18 As shown in Figure a, in prediction partition mode A, all pixels use the reconstructed value of the upper pixel as the prediction value of the current pixel.
[0302] like Fig.18 As shown in Figure b, in the prediction partition mode B, all pixels use the reconstructed value of the left pixel as the prediction value of the current pixel.
[0303] exist Fig.18 In , the 8x1 current image block is divided into a prediction group.
[0304] In order to improve the parallelism in the prediction process, the following examples are used in this application: Fig.11 In the embodiment shown, a prediction partitioning mode of a current image block of 8x2 is as follows: Fig.19 shown.
[0305] exist Fig.19In the example, the kN+1th pixel (N is preferably 4, k=1) of the current image block of 8x1 uses the reconstructed value of the pixel above it as the predicted value of the current pixel. The other pixels use the reconstructed value of the pixel on the left as the predicted value of the current pixel.
[0306] It should be noted that:
[0307] In one example, in any of the above schemes, if the current image block is located in the first column of a frame image or an independent encoding (decoding) unit, there is no pixel on the left side of the current image block, and a default value may be used to replace its reconstructed value. For example, the default value may be 1<<(bit_depth-1).
[0308] Similarly, for any of the above schemes, if the current image block is located in the first row of the image or independent encoding (decoding) unit, there is no pixel above the current image block, and a default value can be used to replace its reconstructed value. For example, the default value can be 1<<(bit_depth-1).
[0309] Among them, bit_depth represents the bit width of the current image block, for example, it can be 8, 9, 10, 12 or 16. 1<< (bit_depth-1) means 1 is shifted left by bit_depth-1 bits. It should be noted that the reconstructed values of pixels in this application can be considered as the pixel values of the pixels in a certain component. For example, the components here can be the red R component, the green G component or the blue B component in the RGB format; or, they can be the luminance Y component, the chromaticity blue Cb component or the chromaticity red Cr component in the YCbCr format. The size of each component can be the same as the image size or smaller than the image size. For example, in the YUV420 format, the size of Y is equal to the image size, and the width and height of U or V are only half of Y, that is, the size is only 1 / 4 of the image. Based on this, the bit_depth here can represent the bit width of the current image block in a certain component.
[0310] It should also be noted that: in any of the above solutions, for each Group, in addition to the above horizontal or vertical prediction mode, other prediction modes may also be used. A feasible prediction mode is: using pixel values of adjacent image blocks or adjacent independent image blocks for reference prediction.
[0311] Embodiment 5
[0312] In some embodiments of the present application, the encoding end may group the residual blocks of the coding unit. Specifically, each row of pixels of the coding unit is used as a prediction group, and the residual block of the prediction group is divided into at least one residual small block.
[0313] Division mode 1: The residual block of a prediction group is divided into a small residual block. This is equivalent to not grouping the residual block, such as Fig. 20 As shown in Figure a.
[0314] Division mode 2: The residual block of a prediction group is divided into multiple residual small blocks. Specifically, the residual block of a prediction group is divided into multiple residual small blocks according to a preset ratio. This application does not limit the prediction ratio. The sizes of the multiple residual small blocks can be equal, or the sizes of some residual small blocks can be equal.
[0315] For example, the residual block of a prediction group is divided into two residual small blocks according to a 1:1 ratio, such as Fig. 20 As shown in Figure b.
[0316] For example, the residual block of a prediction group is divided into three residual blocks according to the ratio of 2:1:1, such as Fig. 20 As shown in Figure c.
[0317] For example, the residual block of a prediction group is divided into three residual blocks according to the ratio of 1:1:2, such as Fig. 20 As shown in Figure d.
[0318] For example, the residual block of a prediction group is divided into four residual blocks according to the ratio of 1:1:1:1, such as Fig. 20 As shown in Figure e.
[0319] in, Fig. 20 Figures a to e in the figure are all described by taking the coding unit as 16x2 and each row as a prediction group. The 16x2 coding unit can be a luminance block or a chrominance block. Each small square in each figure represents a pixel, and the same type of small squares (such as all white small squares, or all small squares marked with the same shade) represent pixels in the same residual small block.
[0320] In addition, for the 8×1 prediction group, the residual block of the prediction group can be divided into one residual small block or divided equally into two residual small blocks. Of course, the specific implementation is not limited to this.
[0321] In an example, the coefficients of pixels in the same residual block are encoded in the same manner, or the coding length (CL), that is, the number of bits occupied by the residual coefficients after encoding, of the pixels in the same residual block is the same.
[0322] In one example, the above Fig. 20 Five partition modes of the residual block are illustrated.
[0323] In an example, 1 bit or 3 bits may be used to represent the partition mode GPM (as shown in Table 4). GP0-GP4 may correspond to Fig. 20 Figure a-e in the figure.
[0324] Table 4: GPM index encoding method (conventional binary encoding method)
[0325]
[0326] It should be noted that the index encoding method of the residual block partition mode shown in Table 4 can also be applied to the partition modes of the residual blocks of prediction groups of other sizes, as well as other partition modes of the above-mentioned prediction groups.
[0327] It should be noted that when a residual block is divided into a large number of residual sub-blocks, more division mode information needs to be encoded; when a residual block is divided into a small number of residual sub-blocks, residuals with different characteristics are grouped together, and the encoding efficiency of the residual coefficients is not high. Taking at least these two aspects into consideration, the embodiment of the present application provides the above-mentioned residual block division method.
[0328] Embodiment 6
[0329] like Fig.21 As shown, it is a schematic diagram of an image reconstruction method provided in an embodiment of the present application, which can be applied to an image encoding method or an image decoding method. Fig.21 The method shown may include the following steps:
[0330] S801: Determine a target prediction mode of a current image block; wherein the current image block includes at least one prediction group; any one prediction group of the at least one prediction group includes a plurality of continuous pixels.
[0331] The plurality of continuous pixels may be a plurality of continuous pixels in a row, or a plurality of continuous pixels in a column.
[0332] When this embodiment is applied to an image coding method, the execution subject of this embodiment may be a coding end, and the current image block is specifically an image block to be coded or a unit to be coded.
[0333] When this embodiment is applied to an image decoding method, the execution subject of this embodiment may be a decoding end, and the current image block may be a to-be-decoded image block or a to-be-decoded unit. S801 may include: parsing a bitstream to obtain an index of a target prediction mode of the current image block.
[0334] S802: When the target prediction mode satisfies a preset condition, a reconstruction value of the target pixel is determined based on a reference value of the first pixel among the plurality of continuous pixels and a residual value of every two adjacent pixels between the first pixel and the target pixel; wherein the target pixel is any non-first pixel among the plurality of continuous pixels, and the reconstructed block of the current image block includes the reconstruction value of the target pixel.
[0335] In one example, the first pixel in the plurality of consecutive pixels adopts the first prediction mode, and the non-first pixel adopts the second prediction mode. The target prediction mode includes the first prediction mode and the second prediction mode. In this case, the target prediction mode satisfies the preset conditions including any of the following conditions:
[0336] Condition 1: Both the first prediction method and the second prediction method are horizontal predictions;
[0337] Condition 2: Both the first prediction method and the second prediction method are vertical predictions;
[0338] Condition 3: One of the first prediction mode and the second prediction mode is horizontal prediction, and the other is vertical prediction. For example, the first prediction mode is horizontal prediction, and the second prediction mode is vertical prediction. For another example, the first prediction mode is vertical prediction, and the second prediction mode is horizontal prediction.
[0339] Condition 4: The first prediction mode is to use the pixel values of the image blocks adjacent to the current image block or the independent coding (decoding) unit adjacent to the independent coding (decoding) unit where the current image block is located for reference prediction. The second prediction mode is horizontal prediction or vertical prediction.
[0340] The image reconstruction method provided in this embodiment allows the encoding / decoding end to obtain the reconstructed value of the current pixel directly based on the residual value between the previous pixel and its adjacent pixels when performing reconstruction, without waiting for the reconstructed value of the previous pixel. This solution can also greatly improve the parallelism of the reconstruction process, thereby improving the decoding parallelism and throughput.
[0341] For simplicity, the following Fig. 22 The prediction mode of the 8×1 image block shown is used as an example for explanation. Other image blocks or other prediction modes can be simply deduced by analogy and will not be described in detail.
[0342] The reconstruction value of the current 8x1 image block is R(i), the residual value is Res(i), and the predicted value is Pred(i), i=0, 1, ...7. The reconstruction value of the 8x1 block above the current 8x1 block is RA(i)i=0, 1, ...7, and the reconstruction value of the left pixel is RLi, i=0, 1, such as Fig. 22 As shown. Fig. 22 is Fig.19 Drawn on the basis of.
[0343] Based on the traditional method, the prediction value of the 8x1 image block is:
[0344] If i=0, then Pred(i)=RL0;
[0345] If i = 1, 2 or 3, then Pred(i) = R(i-1);
[0346] If i=4, then Pred(i)=RA(4);
[0347] If i=5, 6 or 7, then Pred(i)=R(i-1).
[0348] In summary, in the traditional method, the reconstructed value of the 8x1 image block is: R(i)=Pred(i)+Res(i)==R(i-1) / RA(4)+Res(i), i=0, 1,…7.
[0349] based on Fig.21 The method shown:
[0350] For i=0; R(0)=Pred(0)+Res(0)=RL0+Res(0);
[0351] For i=1, R(1)=Pred(1)+Res(1), and since Pred(1)=R(0), R(1)=R(0)+Res(1)=RL0+Res(0)+Res(1).
[0352] Similarly, for i = 2 or 3, R(2) = RL0 + Res(0) + Res(1) + Res(2), R(3) = RL0 + Res(0) + Res(1) + Res(2) + Res(3).
[0353] In summary, for any value of i=0~3, Based on this method, the reconstruction value of the pixel with i=1, 2 or 3 does not need to wait for the reconstruction value of the pixel on its left to be generated. It only needs to obtain the pixel value RL0 on the left side of the current image block and the residual value between the current pixel and the pixel on its left to directly obtain the reconstruction value of the current pixel. This solution can also greatly improve decoding parallelism and throughput.
[0354] And so on, i=4~7, , the reconstructed values of the pixels with i=5~7 can be obtained without waiting for the reconstructed values of the pixels on their left.
[0355] It should be noted that some or all of the technical features in any of the embodiments provided above can be combined without conflict to form a new embodiment. In addition, the technical solution provided in any of the embodiments above can be applied to 16x2, 8x2, 8x1 image blocks, and of course can also be applied to other MxN image blocks, such as 4x2, 16x4, etc.
[0356] It is understandable that, in order to implement the functions in the above-mentioned embodiments, the encoder / decoder includes hardware structures and / or software modules corresponding to the execution of each function. It should be easily appreciated by those skilled in the art that, in combination with the units and method steps of each example described in the embodiments disclosed in this application, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application scenario and design constraints of the technical solution.
[0357] The following describes a decoding device and an encoding unit provided in an embodiment of the present application. In one example, any of the decoding devices provided below may be Figure 2 The destination device 12 or decoder 122 in the embodiment of the present invention may be Figure 6 In another example, any of the encoding devices provided below may be Figure 2 The source device 11 or encoder 112 in the embodiment of the present invention may be Figure 3 The encoder 20 in the embodiment is described uniformly here and will not be described in detail below.
[0358] Fig.23 A structural schematic diagram of a decoding device 3000 provided in the present application, any of the above decoding method embodiments can be executed by the decoding device. The decoding device 3000 includes a code stream parsing unit 3001, a determination unit 3002 and a reconstruction unit 3003. The code stream parsing unit 3001 is used to parse the code stream to obtain a first syntax element, and the first syntax element includes an index of a target prediction mode of a unit to be decoded. The determination unit 3002 is used to determine a target prediction mode from an index table based on the index of the target prediction mode; wherein the index table contains a correspondence between the indexes of multiple prediction modes and the multiple prediction modes; the reconstruction unit 3003 is used to reconstruct the unit to be decoded based at least on the target prediction mode to obtain a reconstructed block.
[0359] In one example, the code stream parsing unit 3001 can Figure 6 The decoding and parsing unit 301 in the embodiment of the present invention is implemented. The determining unit 3002 can be Figure 6 The prediction processing unit 304 in the reconstruction unit 3003 can be realized by Figure 6 The prediction processing unit 304 is implemented together with the reconstruction unit 305. Figure 6 The current image block in may be the unit to be decoded in this embodiment.
[0360] For a more detailed description of the above-mentioned code stream parsing unit 3001, determination unit 3002 and reconstruction unit 3003, as well as a more detailed description of each technical feature therein, and a description of the beneficial effects, etc., please refer to the above-mentioned corresponding method embodiment part, which will not be repeated here.
[0361] Fig.24 The present application provides a schematic diagram of the structure of a coding device 3100, which includes a prediction unit 3101 and a coding unit 3102. The prediction unit 3101 is used to determine a target prediction mode of a unit to be coded, and based on the target prediction mode, determine the index of the target prediction mode from an index table; wherein the index table contains the corresponding relationship between the indexes of multiple prediction modes and the multiple prediction modes. The coding unit 3102 is used to encode the index of the target prediction mode into a bitstream.
[0362] In one example, the prediction unit 3101 may Figure 3 The prediction processing unit 201 in the embodiment can be realized by the encoding unit 3102 Figure 3 The coefficient encoding unit 205 in is implemented. Figure 3 The current image block in can be the encoding unit of this embodiment.
[0363] For a more detailed description of the prediction unit 3101 and the encoding unit 3102, as well as a more detailed description of each technical feature therein and a description of the beneficial effects, etc., please refer to the corresponding method embodiment part above, which will not be repeated here.
[0364] Fig.25 This is a structural schematic diagram of a coding device / decoding device 3200 provided in the present application. The coding device / decoding device 3200 includes: a determination unit 3201, a prediction unit 3202 and a reconstruction unit 3203.
[0365] The determination unit 3201 is used to determine the residual coding mode of the current image block. If the residual coding mode is the skip residual coding mode, the prediction unit 3202 is used to predict the current image block to obtain a prediction block, and the reconstruction unit 3203 is used to determine the prediction block as the reconstructed block of the current image block. If the residual coding mode is the normal residual coding mode, the determination unit 3201 is also used to obtain the residual quantization correlation value of the current image block to obtain the residual block, and the reconstruction unit 3203 is used to reconstruct the current image block based on the residual block to obtain the reconstructed block of the current image block.
[0366] In one example, the determining unit 3201 may Figure 6The code stream parsing unit 301 in the prediction unit 3202 can be realized by Figure 6 The prediction processing unit 304 is implemented, and the reconstruction unit 3203 can be Figure 6 The reconstruction unit 305 in is implemented.
[0367] In another example, the determining unit 3201 may Figure 3 The residual coding unit 203 in the prediction unit 3202 can be realized by Figure 3 The prediction processing unit 201 in the embodiment can be realized by the reconstruction unit 3203 Figure 3 The reconstruction unit 208 in is implemented.
[0368] For a more detailed description of the above-mentioned determination unit 3201, prediction unit 3202 and reconstruction unit 3203, as well as a more detailed description of each technical feature therein, and a description of the beneficial effects, etc., please refer to the above-mentioned corresponding method embodiment part, which will not be repeated here.
[0369] Fig.26 A structural schematic diagram of a coding device / decoding device 3300 provided in the present application, the coding device / decoding device 3300 includes: a determination unit 3301, a prediction unit 3302, a reconstruction unit 3303 and an acquisition unit 3304. The determination unit 3301 is used to determine the target prediction mode of the current image block. If the target prediction mode is a prediction mode based on skip residual coding, the prediction unit 3302 is used to predict the previous image block to obtain a prediction block, and the reconstruction unit 3303 is used to determine the prediction block as a reconstructed block. If the target prediction mode is a prediction mode based on normal residual coding, the acquisition unit 3304 is used to obtain the residual quantization correlation value of the previous image block to obtain the residual block of the previous image block, and the reconstruction unit 3303 is used to reconstruct the previous image block based on the target prediction mode and the residual block to obtain a reconstructed block.
[0370] In one example, the determining unit 3301 may Figure 6 The code stream parsing unit 301 in the embodiment can be realized by the prediction unit 3302 Figure 6 The prediction processing unit 304 is implemented, and the reconstruction unit 3303 can be Figure 6 The reconstruction unit 305 in the embodiment can be realized by obtaining the unit 3304 Figure 6 The code stream parsing unit 301 and the residual decoding unit 303 are implemented.
[0371] In another example, the determining unit 3301 and the predicting unit 3302 may be Figure 3 The prediction processing unit 201 in the embodiment can be realized by the reconstruction unit 3303 Figure 3 The reconstruction unit 208 in the embodiment can be realized by obtaining the unit 3304 Figure 3 The residual decoding unit 303 in is implemented.
[0372] For a more detailed description of the above-mentioned determination unit 3301, prediction unit 3302, reconstruction unit 3303 and acquisition unit 3304, as well as a more detailed description of each technical feature therein, and a description of the beneficial effects, etc., please refer to the above-mentioned corresponding method embodiment part, which will not be repeated here.
[0373] Fig. 27 A schematic diagram of the structure of a coding device / decoding device 3400 provided in the present application, the coding device / decoding device 3400 includes: a determination unit 3401 and a reconstruction unit 3402. The determination unit 3401 is used to determine the prediction mode of at least one prediction group into which the current image block is divided; wherein any one of the at least one prediction group includes a plurality of continuous pixels in the same row / column; any one of the prediction groups includes a first designated pixel area and a second designated pixel area, the first designated pixel area includes a plurality of pixel groups, the plurality of pixel groups are obtained according to the division of the second designated pixel area, the prediction modes of the first designated pixel area and the second designated pixel area are different, the plurality of pixel groups can be predicted in parallel, and the pixel group includes one or a plurality of continuous pixels. The reconstruction unit 3402 is used to reconstruct the current image block based on the prediction mode of at least one prediction group to obtain a reconstructed block.
[0374] In one example, the determining unit 3401 may Figure 6 The code stream parsing unit 301 in the embodiment can be realized by the reconstruction unit 3402 Figure 6 The prediction processing unit 304 and the reconstruction unit 305 are implemented together.
[0375] In another example, the determining unit 3401 may Figure 3 The prediction processing unit 201 in the embodiment can be realized by the reconstruction unit 3402 Figure 3 The reconstruction unit 208 in is implemented together.
[0376] For a more detailed description of the above-mentioned determination unit 3401 and reconstruction unit 3402, as well as a more detailed description of each technical feature therein, and a description of the beneficial effects, etc., please refer to the above-mentioned corresponding method embodiment part, which will not be repeated here.
[0377] Fig.28A schematic diagram of the structure of a coding device / decoding device 3500 provided in the present application, the coding device / decoding device 3500 includes: a determination unit 3501 and a reconstruction unit 3502. The determination unit 3501 is used to determine a target prediction mode of a current image block; wherein the current image block includes at least one prediction group; any one of the at least one prediction group contains a plurality of continuous pixels. The reconstruction unit 3502 is used to determine a reconstruction value of the target pixel based on a reference value of the first pixel in the plurality of continuous pixels and a residual value of every two adjacent pixels between the first pixel and the target pixel when the target prediction mode meets a preset condition; wherein the target pixel is any non-first pixel in the plurality of continuous pixels, and the reconstructed block of the current image block includes the reconstruction value of the target pixel.
[0378] In one example, the determining unit 3501 may Figure 6 The code stream parsing unit 301 in the embodiment can be realized by the reconstruction unit 3502 Figure 6 The prediction processing unit 304 and the reconstruction unit 305 are implemented together.
[0379] In another example, the determining unit 3501 may Figure 3 The prediction processing unit 201 in the embodiment can be realized by the reconstruction unit 3502 Figure 3 The reconstruction unit 208 in is implemented together.
[0380] For a more detailed description of the above-mentioned determination unit 3501 and reconstruction unit 3502, as well as a more detailed description of each technical feature therein, and a description of the beneficial effects, etc., please refer to the above-mentioned corresponding method embodiment part, which will not be repeated here.
[0381] The present application also provides an electronic device for executing any of the above decoding method embodiments. Fig.29 As shown, Fig.29 The electronic device 3600 is a schematic diagram of a structure of an electronic device provided by the present application, and includes a processor 3610 and a communication interface 3620. The processor 3610 and the communication interface 3620 are coupled to each other. It is understood that the communication interface 3620 can be a transceiver or an input-output interface. In an example, the electronic device 3600 may also include a memory 3630 for storing instructions executed by the processor 3610 or storing input data required by the processor 3610 to run the instructions or storing data generated after the processor 3610 runs the instructions.
[0382] The specific connection medium between the communication interface 3620, the processor 3610 and the memory 3630 is not limited in the embodiment of the present application. Fig.29The communication interface 3620, the processor 3610 and the memory 3630 are connected via a bus 3640. Fig.29 The connection between the other components is only for schematic illustration and is not intended to be limiting. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in FIG35, but it does not mean that there is only one bus or one type of bus.
[0383] The memory 3630 can be used to store software programs and modules, such as program instructions / modules corresponding to the decoding method or encoding method provided in the embodiment of the present application. The processor 3610 executes the software programs and modules stored in the memory 3630 to perform various functional applications and data processing to implement any of the decoding methods or encoding methods provided above. The communication interface 3620 can be used to communicate signaling or data with other devices. In the present application, the electronic device 3600 can have multiple communication interfaces 3620.
[0384] It is understandable that the processor in the embodiments of the present application may be a central processing unit (CPU), a neural processing unit (NPU) or a graphic processing unit (GPU), or may be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. The general-purpose processor may be a microprocessor or any conventional processor.
[0385] The method steps in the embodiments of the present application can be implemented by hardware or by a processor executing software instructions. The software instructions can be composed of corresponding software modules, and the software modules can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable ROM (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, mobile hard disks, CD-ROMs or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an ASIC. In addition, the ASIC can be located in a network device or a terminal device. Of course, the processor and the storage medium can also be present in a network device or a terminal device as discrete components.
[0386] An embodiment of the present application also provides a coding and decoding system, including a coding end and a decoding end, wherein the coding end can be used to execute any one of the coding methods provided above, and the decoding end is used to execute the corresponding decoding method.
[0387] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer programs or instructions. When the computer program or instruction is loaded and executed on a computer, the process or function described in the embodiment of the present application is executed in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, a network device, a user device or other programmable device. The computer program or instruction may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer program or instruction may be transmitted from one website site, computer, server or data center to another website site, computer, server or data center by wired or wireless means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server, data center, etc. that integrates one or more available media. The available medium may be a magnetic medium, for example, a floppy disk, a hard disk, a tape; it may also be an optical medium, for example, a digital video disc (DVD); it may also be a semiconductor medium, for example, a solid state drive (SSD).
[0388] In the various embodiments of the present application, if there is no special explanation and logical conflict, the terms and / or descriptions between different embodiments are consistent and can be referenced to each other, and the technical features in different embodiments can be combined to form new embodiments according to their inherent logical relationships. In the present application, "at least one" means one or more, and "more" means two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. In the text description of the present application, the character " / " generally indicates that the previous and next associated objects are in an "or" relationship; in the formula of the present application, the character " / " indicates that the previous and next associated objects are in a "division" relationship.
[0389] It is understood that the various numbers involved in the embodiments of the present application are only for the convenience of description and are not used to limit the scope of the embodiments of the present application. The size of the sequence number of the above-mentioned processes does not mean the order of execution, and the execution order of each process should be determined by its function and internal logic.
Claims
1. An image encoding and decoding method, characterized in that: The method comprises: Determine a prediction mode of at least one prediction group into which the current image block is divided; wherein any one of the at least one prediction group includes a plurality of continuous pixels in the same row / column; the any one prediction group includes a first designated pixel region and a second designated pixel region, the first designated pixel region includes a plurality of separated pixel groups, the plurality of separated pixel groups are obtained by dividing the second designated pixel region, the first designated pixel region and the second designated pixel region have different prediction modes, the plurality of pixel groups can be predicted in parallel, and the pixel group includes one or a plurality of continuous pixels; Based on the prediction mode of the at least one prediction group, the current image block is reconstructed to obtain a reconstructed block.
2. The method according to claim 1, characterized in that: If any one of the prediction groups includes multiple consecutive elements in the same row, then: The first designated pixel region adopts vertical prediction, and the second designated pixel region adopts horizontal prediction; Alternatively, the first designated pixel region adopts vertical prediction, and the second designated pixel region adopts vertical mean prediction; And / or, if any one of the prediction groups includes a plurality of consecutive elements in the same column, then: The first designated pixel region adopts horizontal prediction, and the second designated pixel region adopts vertical prediction; Alternatively, the first designated pixel region adopts horizontal prediction, and the second designated pixel region adopts horizontal average prediction.
3. The method according to claim 1 or 2, characterized in that: The difference between the numbers of pixels included in different pixel groups is equal to or smaller than a threshold.
4. The method according to claim 1 or 2, characterized in that: If there is no reference block for the current image block, the reconstructed value of the reference block of the current image block is 1 and left-shifted by bit_depth-1, where bit_depth represents the bit width of the current image block.
5. The method according to claim 1 or 2, characterized in that: The current image block includes at least one prediction group; any one prediction group in the at least one prediction group includes a plurality of continuous pixels; Reconstructing the current image block based on the prediction mode of the at least one prediction group to obtain a reconstructed block includes: When the target prediction mode satisfies a preset condition, a reconstruction value of the target pixel is determined based on a reference value of a first pixel among the plurality of continuous pixels and a residual value of every two adjacent pixels between the first pixel and the target pixel; wherein the target pixel is any non-first pixel among the plurality of continuous pixels, and the reconstructed block of the current image block includes the reconstructed value of the target pixel.
6. An image encoding / decoding device, characterized in that: include: A determination unit, configured to determine a prediction mode of at least one prediction group into which the current image block is divided; wherein any one of the at least one prediction group includes a plurality of continuous pixels in the same row / column; the any one prediction group includes a first designated pixel region and a second designated pixel region, the first designated pixel region includes a plurality of separated pixel groups, the plurality of separated pixel groups are obtained by dividing the second designated pixel region, the first designated pixel region and the second designated pixel region have different prediction modes, the plurality of pixel groups can be predicted in parallel, and the pixel group includes one or a plurality of continuous pixels; The reconstruction unit is used to reconstruct the current image block based on the prediction mode of the at least one prediction group to obtain a reconstructed block.
7. An electronic device, characterized in that: The invention comprises a processor and a memory, wherein the memory is used to store computer instructions, and the processor is used to call and execute the computer instructions from the memory to implement the method according to any one of claims 1 to 5.
8. A computer-readable storage medium, characterized in that: The storage medium stores a computer program or instruction. When the computer program or instruction is executed by an electronic device, the method according to any one of claims 1 to 5 is implemented.
