Image decoding and coding method and device, equipment and storage medium

By determining the non-rectangular scanning area in image encoding and decoding and encoding, the redundant encoding problem caused by insufficiently tight scanning area division in the prior art is solved, and the compression performance is improved.

CN118945367AActive Publication Date: 2024-11-12HANGZHOU HIKVISION DIGITAL TECHNOLOGY CO LTD
View PDF 6 Cites 0 Cited by

Patent Information

Application Number
CN202310550107.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-05-12
Publication Date
2024-11-12
Estimated Expiration
2043-05-12

AI Technical Summary

Technical Problem

In the prior art, the division of the scanning areas during image encoding is not tight enough, resulting in increased encoding of redundancy coefficients and unsatisfactory compression performance.

Method used

By extracting the area syntax parameters of the transformed block to be decoded from the image code stream, the non-rectangular scanning area is determined, and decoded and encoded based on the non-zero data, the reconstructed image block is constructed.

Benefits of technology

This reduces the bit cost of image encoding, improves compression performance, and achieves a more flexible and compact expression of scanning areas.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118945367A_ABST
    Figure CN118945367A_ABST
Patent Text Reader

Abstract

The invention belongs to the technical field of image processing, and discloses an image decoding and coding method and device, equipment and a storage medium. The method comprises the following steps: extracting a to-be-decoded transform block from an image code stream, and obtaining a region grammar parameter corresponding to the to-be-decoded transform block; a scanning area in the to-be-decoded conversion block is determined according to the area grammar parameters, the scanning area is an area matched with non-zero data in quantized residual data corresponding to the to-be-decoded conversion block, and the scanning area comprises a non-rectangular area; decoding the scanning regions in sequence according to a region scanning sequence corresponding to the to-be-decoded transform block to obtain quantized residual data; and constructing a reconstructed image block corresponding to the transform block to be decoded according to the quantized residual data. The scanning area determined during decoding can be the non-rectangular area adaptive to the non-zero data in the quantized residual data, so that a more flexible and compact scanning area expression mode can be selected when the image is compressed, the bit cost of image coding is reduced, and the compression performance is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to an image decoding and encoding method, device, equipment and storage medium. Background Art

[0002] In the process of encoding images (such as videos), a prediction process is required to remove spatial and temporal redundancy. The encoder obtains a predicted value through prediction, and the original pixel minus the predicted value obtains the residual. The residual is transformed and quantized to obtain quantized residual data (also called coefficients). The quantized residual data and other information (such as block division information, mode information, transformation parameters, quantization parameters, etc.) are entropy encoded to obtain a bit stream.

[0003] In the process of encoding coefficients, the coefficient range to be encoded is generally identified by scanning an area, and the current scanning area is a rectangular area, which is not optimal in expressing the scanning area. For example, when the lower right corner of the rectangle is all zero, the encoding of redundant zero coefficients will be increased.

[0004] The above contents are only used to assist in understanding the technical solution of the present invention and do not constitute an admission that the above contents are prior art. Summary of the invention

[0005] The main purpose of the present invention is to provide an image decoding and encoding method, device, equipment and storage medium, aiming to solve the technical problem that the division of the scanning area in the prior art is not compact enough, and the encoding of redundant coefficients will be included, resulting in unsatisfactory compression performance.

[0006] To achieve the above object, the present invention provides an image decoding method, which comprises the following steps:

[0007] Extracting a transform block to be decoded from an image code stream, and obtaining a regional syntax parameter corresponding to the transform block to be decoded;

[0008] Determine a scanning area in the transform block to be decoded according to the area syntax parameter, the scanning area being an area adapted to non-zero data in the quantized residual data corresponding to the transform block to be decoded, and the scanning area including a non-rectangular area;

[0009] Decoding the scanned areas in sequence according to the area scan order corresponding to the to-be-decoded transform block to obtain quantized residual data;

[0010] A reconstructed image block corresponding to the to-be-decoded transform block is constructed according to the quantized residual data.

[0011] In a possible implementation of the present application, the area scanning order is at least one of horizontal scanning, vertical scanning, zigzag scanning and reverse zigzag scanning; the area scanning order is a preset order, adaptively selected according to the area shape type of the scanning area, or extracted from the image code stream through syntax; the syntax is sequence level, frame level, Patch level, Slice level, Tile level, CTU level, CTB level, CU level, CB level or TB level.

[0012] In a possible implementation of the present application, determining the scanning area in the to-be-decoded transform block according to the area syntax parameter includes:

[0013] Obtaining a region shape type corresponding to the transform block to be decoded;

[0014] A scanning area in the to-be-decoded transform block is determined according to the area syntax parameter and the area shape type.

[0015] In a possible implementation of the present application, the area shape type is a polygon, a point or a line.

[0016] In a possible implementation of the present application, the area shape type is a combined shape; the combined shape is formed by combining at least two shapes of a polygon, a point or a line.

[0017] In a possible implementation of the present application, the image code stream corresponds to at least one type of region shape;

[0018] The obtaining the region shape type corresponding to the to-be-decoded transform block includes:

[0019] Determining the region shape type corresponding to the to-be-decoded transform block according to a preset type value;

[0020] or,

[0021] Derived according to the regional syntax parameters, determining the regional shape type corresponding to the to-be-decoded transform block;

[0022] or,

[0023] Obtaining a shape indication parameter corresponding to the transform block to be decoded;

[0024] The region shape type corresponding to the to-be-decoded transform block is selected from at least one region shape type corresponding to the image code stream according to the shape indication parameter.

[0025] In a possible implementation of the present application, the shape indication parameter is a preset parameter or is obtained from a video bit stream through a syntax, and the syntax is at the TB level.

[0026] In a possible implementation of the present application, the region shape type is a non-isosceles right triangle;

[0027] The determining the scanning area in the to-be-decoded transform block according to the area syntax parameter and the area shape type comprises:

[0028] Constructing a region in the shape of a non-isosceles right triangle according to the region syntax parameters, and obtaining the region integer coordinates and non-integer coordinates on the hypotenuse of the triangle included in the constructed region;

[0029] Rounding the non-integer coordinates according to a rounding method to obtain hypotenuse integer coordinates;

[0030] A scanning area in the to-be-decoded transform block is determined according to the area integer coordinates and the hypotenuse integer coordinates.

[0031] In a possible implementation of the present application, the rounding method is rounding off, rounding down or rounding up; the rounding method is a preset value or read from an image bitstream through syntax; the syntax is sequence level, frame level, Patch level, Slice level, Tile level, CTU level, CTB level, CU level, CB level or TB level.

[0032] In a possible implementation of the present application, the region shape type is a polygon with more than three sides;

[0033] The determining the scanning area in the to-be-decoded transform block according to the area syntax parameter and the area shape type comprises:

[0034] Get the angle value of the hypotenuse of the area shape in the scan area;

[0035] A scanning area in the to-be-decoded transform block is determined according to the angle value and the area syntax parameter.

[0036] In a possible implementation of the present application, the regional syntax parameter is used to indicate the regional range of the scanning area.

[0037] In a possible implementation of the present application, the regional syntax parameter includes a parameter value for indicating a position of a hypotenuse of a regional shape in the scanning region.

[0038] In a possible implementation of the present application, the regional syntax parameters include horizontal parameters, vertical parameters and diagonal parameters;

[0039] The horizontal parameter is the horizontal coordinate of the non-zero quantized residual data on the rightmost side of the scanning area, or the length of the right-angled side in the horizontal direction;

[0040] The vertical parameter is the ordinate of the non-zero quantized residual data at the bottom of the scanning area, or the length of the right-angle side in the vertical direction;

[0041] The diagonal parameter is the sum of the horizontal coordinate and the vertical coordinate of the non-zero quantized residual data at the lower right side of the scanning area, or the horizontal coordinate of the intersection of the extension line of the hypotenuse of the area shape and the upper boundary of the to-be-decoded transform block.

[0042] In a possible implementation of the present application, the region syntax parameter is extracted from the image code stream through syntax based on the context model;

[0043] The context model is related to at least one of the following information: width, height, area, color channel, luminance component channel, chrominance component channel, block transform type, and sub-block transform type of the transform block to be decoded.

[0044] In a possible implementation of the present application, the angle value is a preset value, or is derived according to the regional syntax parameter, or is extracted from the image code stream through syntax; the syntax is at the TB level.

[0045] In a possible implementation of the present application, the decoding the scanned areas in sequence according to the area scanning order corresponding to the to-be-decoded transform block to obtain quantized residual data includes:

[0046] If the scanned area is an area of ​​a preset shape type, the scanned area is decoded in sequence according to the area scanning order corresponding to the to-be-decoded transform block based on a preset coding skip rule to obtain quantized residual data.

[0047] In addition, to achieve the above object, the present invention also proposes an image encoding method, the image encoding method comprising:

[0048] Converting the to-be-encoded conversion block corresponding to the target image to obtain quantized residual data corresponding to the to-be-encoded conversion block;

[0049] Marking a scan area in the to-be-encoded conversion block according to non-zero data in the quantized residual data, wherein the scan area includes a non-rectangular area;

[0050] Encoding the quantized residual data corresponding to the scanned area in sequence according to the area scanning order corresponding to the to-be-encoded conversion block, to generate an image code stream corresponding to the target image;

[0051] A region syntax parameter corresponding to the to-be-encoded conversion block is constructed according to the scanned region, and the region syntax parameter is written into the image code stream.

[0052] In a possible implementation of the present application, the area scanning order is at least one of horizontal scanning, vertical scanning, Z-shaped scanning, and reverse Z-shaped scanning; the area scanning order is a preset order and is adaptively selected according to the area shape type of the scanning area.

[0053] In a possible implementation of the present application, the area scanning order is written into the image code stream through syntax; the syntax is sequence level, frame level, Patch level, Slice level, Tile level, CTU level, CTB level, CU level, CB level or TB level.

[0054] In a possible implementation of the present application, the area shape type of the scanning area is a polygon, a point or a line.

[0055] In a possible implementation of the present application, the area shape type is a combined shape; the combined shape is formed by combining at least two shapes of a polygon, a point or a line.

[0056] In a possible implementation of the present application, encoding the quantized residual data corresponding to the scanned area in sequence according to the area scanning order corresponding to the to-be-encoded conversion block to generate an image code stream corresponding to the target image includes:

[0057] If the scanned area is an area of ​​a preset shape type, the quantized residual data corresponding to the scanned area is encoded in sequence according to the area scanning order corresponding to the to-be-encoded conversion block based on a preset encoding skip rule to generate an image code stream corresponding to the target image.

[0058] In addition, to achieve the above object, the present invention further provides an image decoding device, the image decoding device comprising:

[0059] A decoding module, used for extracting a transform block to be decoded from an image code stream, and obtaining a regional syntax parameter corresponding to the transform block to be decoded;

[0060] A construction module, configured to determine a scanning area in the to-be-decoded transform block according to the area syntax parameter, wherein the scanning area is a non-rectangular area adapted to non-zero data in the quantized residual data corresponding to the image code stream;

[0061] A scanning module, used for sequentially decoding the scanning areas according to the area scanning order corresponding to the transform block to be decoded, to obtain quantized residual data;

[0062] A reconstruction module is used to construct a reconstructed image block corresponding to the to-be-decoded transform block according to the quantized residual data.

[0063] In addition, to achieve the above object, the present invention further provides an image encoding device, the image encoding device comprising:

[0064] A conversion module, used for converting the to-be-encoded conversion block corresponding to the target image to obtain quantized residual data corresponding to the to-be-encoded conversion block;

[0065] A construction module, configured to mark a scanning area in the to-be-encoded conversion block according to non-zero data in the quantized residual data, wherein the scanning area includes a non-rectangular area;

[0066] A generating module, used for sequentially encoding the quantized residual data corresponding to the scanned area according to the area scanning order corresponding to the to-be-encoded conversion block, to generate an image code stream corresponding to the target image;

[0067] The encoding module is used to construct a region syntax parameter corresponding to the to-be-encoded conversion block according to the scanned region, and write the region syntax parameter into the image code stream.

[0068] In addition, to achieve the above-mentioned purpose, the present invention also proposes a decoding device, which includes: a processor, a memory, and an image decoding program stored in the memory and executable on the processor, and the image decoding program implements the image decoding method described above when executed by the processor.

[0069] In addition, to achieve the above-mentioned purpose, the present invention also proposes a coding device, which includes: a processor, a memory, and an image coding program stored in the memory and runnable on the processor, and when the image coding program is executed by the processor, it implements the image coding method as described above.

[0070] In addition, to achieve the above-mentioned purpose, the present invention also proposes a storage medium, on which an image decoding program and / or an image encoding program are stored, and when the image decoding program is executed, the image decoding method as described above is implemented, and when the image encoding program is executed, the image encoding method as described above is implemented.

[0071] The present invention extracts a transform block to be decoded from an image code stream, and obtains a regional syntax parameter corresponding to the transform block to be decoded; determines a scanning area in the transform block to be decoded according to the regional syntax parameter, the scanning area is an area adapted to non-zero data in quantized residual data corresponding to the transform block to be decoded, and the scanning area includes a non-rectangular area; decodes the scanning area in sequence according to the regional scanning order corresponding to the transform block to be decoded, and obtains quantized residual data; and constructs a reconstructed image block corresponding to the transform block to be decoded according to the quantized residual data. Since the scanning area determined during decoding can be a non-rectangular area adapted to non-zero data in the quantized residual data, a more flexible and compact scanning area expression method can be selected when compressing the image, thereby reducing the bit cost of image coding and improving compression performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] Figure 1 It is a schematic diagram of the structure of an electronic device in a hardware operating environment involved in an embodiment of the present invention;

[0073] Figure 2 A schematic diagram of a flow chart of a first embodiment of an image decoding method of the present invention;

[0074] Figure 3 A schematic diagram of a video encoding framework structure according to an embodiment of the present invention;

[0075] Figure 4 A schematic diagram of a flow chart of a second embodiment of an image decoding method according to the present invention;

[0076] Figure 5 A schematic diagram of dividing a triangular scanning area according to an embodiment of the present invention;

[0077] Figure 6 A schematic diagram of a right-angle trapezoidal area division according to an embodiment of the present invention;

[0078] Figure 7 A schematic diagram of pentagonal area division according to an embodiment of the present invention;

[0079] Figure 8 A schematic diagram of combining shape regions according to an embodiment of the present invention;

[0080] Fig. 9 A schematic diagram of a flow chart of a first embodiment of an image encoding method according to the present invention;

[0081] Fig.10 is a structural block diagram of a first embodiment of an image decoding device according to the present invention;

[0082] Fig.11 FIG. 4 is a structural block diagram of the first embodiment of the image encoding device of the present invention.

[0083] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0084] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0085] Reference Figure 1 , Figure 1 The present invention is a schematic diagram of the structure of a decoding device or encoding device in a hardware operating environment involved in an embodiment of the present invention.

[0086] like Figure 1As shown, the electronic device may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a wireless fidelity (Wireless-Fidelity, WI-FI) interface). The memory 1005 may be a high-speed random access memory (Random Access Memory, RAM), or a stable non-volatile memory (Non-Volatile Memory, NVM), such as a disk storage. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0087] Those skilled in the art will understand that Figure 1 The structure shown in the figure does not constitute a limitation on the electronic device, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0088] like Figure 1 As shown, the memory 1005 as a storage medium may include an operating system, a network communication module, a user interface module, and an image decoding program and / or an image encoding program.

[0089] exist Figure 1 In the electronic device shown, the network interface 1004 is mainly used for data communication with a network server; the user interface 1003 is mainly used for data interaction with a user; the processor 1001 and the memory 1005 in the electronic device of the present invention can be set in a decoding device or an encoding device, and the electronic device calls the image decoding program stored in the memory 1005 through the processor 1001, and executes the image decoding method provided by the embodiment of the present invention; the electronic device calls the image encoding program stored in the memory 1005 through the processor 1001, and executes the image encoding method provided by the embodiment of the present invention.

[0090] The embodiment of the present invention provides an image decoding method, referring to Figure 2 , Figure 2 The figure is a flowchart of a first embodiment of an image decoding method according to the present invention.

[0091] In this embodiment, the image decoding method includes the following steps:

[0092] Step S10: extracting a transform block to be decoded from the image code stream, and obtaining a regional syntax parameter corresponding to the transform block to be decoded.

[0093] It should be noted that the executor of this embodiment can be a decoding device when encoding image data. The decoding device can be an electronic device such as a personal computer or a server. Of course, it can also be other devices that can achieve the same or similar functions. This embodiment does not limit this. In this embodiment and the following embodiments, the image decoding method of the present invention is explained by taking a decoding device as an example.

[0094] Among them, since during the image encoding process, the encoding device generally decodes the encoded image code stream after the encoding is completed, and determines whether the parameters used in the encoding need to be adjusted according to the image quality of the decoded image, the executor of this embodiment can also be the encoding device.

[0095] It should be noted that the image code stream may be a code stream generated after the encoding device encodes the image data to be compressed and encoded. When compressing the image data, the image data is generally divided into at least one image transform block (TB), and then the divided image transform blocks are encoded / decoded in sequence. The transform block to be decoded may be the image transform block currently being decoded and extracted from the image code stream.

[0096] The region syntax parameter may be a parameter for indicating the region range of the scan region, for example, a parameter including the region boundary position coordinates of the scan region. The encoding method used in encoding the region syntax parameter into the image code stream may be group encoding, using truncated unary code to encode the group number, using fixed-length code to encode the offset within the group, using context-based encoding for truncated unary code, and using bypass encoding for fixed-length code.

[0097] Obtaining the regional syntax parameters corresponding to the transform block to be decoded can be based on the context model to extract the regional syntax parameters corresponding to the transform block to be decoded from the image code stream through syntax. The syntax used here can be a syntax at the sequence level, frame level, block (Patch) level, slice (Slice) level, Tile level, coding tree unit (CTU) level, luminance coding tree block (CTB) level, coding unit (CU) level, coding block (CB) level or transform block (TB) level. The context model can be related to at least one of the width, height, area, color channel, luminance component channel, chrominance component channel, block transform type, and sub-block transform type of the transform block to be decoded.

[0098] Figure 3 FIG. 1 is a schematic diagram of the video encoding framework structure of this embodiment. In the image encoding or image decoding process, the operation process of the commonly used video encoding framework is as follows: Figure 3As shown, it mainly involves prediction, transformation, quantization, entropy coding and filtering modules, among which:

[0099] The prediction module includes intra-frame prediction (such as Figure 3 Intra-frame prediction units) and inter-frame prediction (such as Figure 3 Intra-frame prediction uses the reconstructed pixels around the current block to make predictions and remove spatial redundancy; inter-frame prediction uses the reconstructed pixels on the time domain reference frame to make predictions and remove temporal redundancy.

[0100] The transformation module linearly maps the residual information in the spatial domain to the transformation domain (such as the frequency domain), with the goal of making the energy more concentrated and removing the frequency domain correlation of the signal. Theoretically, the transformation matrix is ​​reversible and will not cause signal loss.

[0101] The quantization module is a "many-to-one" mapping process, which is irreversible and will cause signal loss; the advantage is that it can greatly reduce the value range of the signal, so that the encoder can use a small number of symbols to give a good approximation of the original signal, thereby improving the compression rate. Among them, the inverse quantization module and the inverse transform module perform the inverse process of the quantization module and the transform module respectively.

[0102] Entropy coding module (such as Figure 3 The entropy encoder shown in FIG. 1 is a lossless coding method based on the principle of information entropy. It converts a series of element symbols (such as transform coefficients and mode information) used to represent a video sequence into a binary code stream, removing the statistical redundancy of these video element symbols.

[0103] Filter module (such as Figure 3 The in-loop filtering module shown in the figure) will enhance the reconstructed image, with the goal of making the reconstructed image closer to the original image, while reducing the impact of block effects and ringing effects, thereby improving the quality of the reconstructed image.

[0104] During the encoding process, the encoding device will attempt to reconstruct the image, and after the reconstructed image is processed by in-loop filtering, it will pass through the reference image buffer to perform motion estimation / motion compensation, and determine the inter-frame mode information based on the motion estimation / motion compensation. Finally, after determining that the encoding performance meets the requirements, entropy encoding is performed through the entropy encoder to generate an image code stream (i.e., bit stream).

[0105] In this process, the technical terms that may be involved include: Residual, Coefficient Coding, Scan Region-based Coefficient Coding (SRCC), Entropy Coding, etc., which are now explained here:

[0106] Residual: In the image (such as video) encoding process, a prediction process is required to remove spatial and temporal redundancy. The encoder obtains the predicted value after prediction, and the original pixel minus the predicted value gets the residual. The residual is transformed and quantized to get the coefficient, and then the coefficient and other information (including block division information, mode information, transformation parameters, quantization parameters, etc.) are entropy encoded to get the bit stream.

[0107] Coefficient Coding: Encoding the quantized coefficients, including determining the range, coding order and coding method of the coding coefficients, where the coding method includes binarization method, context model and entropy coding. The range of the coding coefficient means that only the coefficients within the range are encoded, and the coefficients outside the range are not encoded (the default is zero). The coding order means that all the coefficients to be encoded are arranged into a one-dimensional array in this order, and then encoded one by one. The binarization method means that each coefficient is represented as a binary symbol string, and then each bit of the binary symbol string (0 / 1 binary number, called bin) is written into the bitstream through the entropy encoder. The context model estimates the probability of each bin taking the value 0 or 1 to assist the entropy encoder in better compressing the binary symbol string.

[0108] Scan Region-based Coefficient Coding (SRCC): A scan region is used to represent the range of coefficients to be encoded. The scan region is a rectangular area, expressed by coordinates (sr_x, sr_y), where sr_x refers to the horizontal coordinate of the rightmost non-zero coefficient in the transform block (TB), and sr_y refers to the vertical coordinate of the bottommost non-zero coefficient in the transform block. During the coefficient encoding process, only the coefficients inside and on the boundary of the scan region are encoded, and the coefficients outside the scan region are not encoded (the default is zero). The encoding order (also called the scanning order) uses reverse Z-shaped scanning.

[0109] Entropy Coding: The commonly used entropy coding method is Context-based Adaptive Binary Arithmetic Coding (CABAC), which performs arithmetic coding on each bin of each syntactic element according to its context model to obtain the final output code stream. In the coding, the coding of each symbol is related to the result of the previous coding, and a code word is adaptively assigned to each symbol according to the statistical characteristics of the code stream. CABAC is particularly suitable for symbols with unequal probabilities of occurrence, which can remove the correlation between symbols and further compress the bit rate.

[0110] Step S20: determining a scan area in the to-be-decoded transform block according to the area syntax parameters.

[0111] It should be noted that the scanning area is a non-rectangular area adapted to the non-zero data in the quantized residual data corresponding to the image code stream (including the non-zero quantized residual data as much as possible), such as an area in the shape of a polygon, point or line, and of course, it can also be a combined shape, such as a shape constructed by a combination of at least two of the polygons, points or lines. Among them, the polygon can include a triangle (such as a right triangle, an isosceles right triangle, etc.), a quadrilateral (such as a trapezoid, a rhombus, a parallelogram, etc.), a pentagon or a polygon with more sides.

[0112] Step S30: decoding the scanned areas in sequence according to the area scanning order corresponding to the to-be-decoded transform block to obtain quantized residual data.

[0113] It should be noted that the area scanning sequence may include at least one of horizontal scanning, vertical scanning, zigzag scanning and reverse zigzag scanning.

[0114] In a specific implementation, the administrator of the encoding device and / or decoding device can set multiple to-be-decoded transform blocks in the image code stream to the same area scanning order. In this case, the preset order pre-set by the administrator is used as the area scanning order corresponding to the to-be-decoded transform blocks.

[0115] Of course, it is also possible to set a corresponding area scanning order for each transform block to be decoded separately. In this case, the area scanning order of the transform block to be decoded can be adaptively set from a variety of area scanning orders according to the area shape type of the scan area in the transform block to be decoded, wherein the adaptive rules can be pre-set by the management personnel of the decoding device and / or the encoding device, for example: for a non-isosceles right triangle area (or an area of ​​a combined shape obtained by combining non-isosceles right triangles), when the width is greater than the height, vertical scanning is used, otherwise horizontal scanning is used; for an isosceles right triangle area or a pentagonal area with a hypotenuse of 45 degrees, Z-shaped scanning or reverse Z-shaped scanning is used.

[0116] Of course, it is also possible to set a corresponding area scanning order for each transform block into which the image data is split when encoding the image data, and write the set area scanning order into the image code stream, and then extract the area scanning order corresponding to the transform block to be decoded from the image code stream through the syntax. The syntax used here can be a sequence level, frame level, patch level, slice level, tile level, CTU level, CTB level, CU level, CB level or TB level syntax.

[0117] In actual use, when the quantized residual data is encoded into the image code stream, the quantized residual data is generally binarized, and then the binarized components are encoded into the image code stream. The binarization formula for each quantized residual data (i.e., the coefficient obtained after transformation and quantization) is:

[0118] coef=sign*(sig+gt1+gt2+remain)

[0119] In the formula, coef is the quantized residual data, sig indicates whether the quantized residual data is non-zero; gt1 indicates whether the absolute value of the quantized residual data is greater than 1; gt2 indicates whether the absolute value of the quantized residual data is greater than 2; remain indicates the absolute value of the quantized residual data minus 3, and the exponential Columbus is used for binarization; sign indicates the sign bit of the quantized residual data.

[0120] At this time, the scanned areas are decoded in sequence according to the area scanning order corresponding to the transform block to be decoded, and the quantized residual data can be obtained by obtaining a context model corresponding to the scanned area, decoding the image code stream according to the context model, and obtaining the binary components of the quantized residual data corresponding to the scanned area in sequence according to the area scanning order corresponding to the transform block to be decoded, and then calculating the corresponding quantized residual data according to the above binarization formula.

[0121] Among them, since only the scanned area in the transform block to be decoded is scanned, the data obtained after the scan actually only contains the quantized residual data corresponding to the scanned area in the transform block to be decoded, and since the quantized residual data corresponding to other areas outside the scanned area of ​​the transform block to be decoded is actually zero by default (of course, it can also be other similar values, which are not limited here), therefore, after obtaining the quantized residual data corresponding to the scanned area in the transform block to be decoded, it is completed with zero to obtain the quantized residual data corresponding to the transform block to be decoded.

[0122] Step S40: constructing a reconstructed image block corresponding to the to-be-decoded transform block according to the quantized residual data.

[0123] It should be noted that constructing a reconstructed image block corresponding to the transform block to be decoded based on the quantized residual data can be by inverse quantizing the quantized residual data, adding the residual obtained by inverse quantization to the predicted mean, obtaining reconstructed image features, performing image transformation based on the reconstructed image features, and thus constructing a reconstructed image block corresponding to the transform block to be decoded.

[0124] In actual use, if there are multiple transform blocks to be decoded, after obtaining reconstructed image blocks corresponding to the transform blocks to be decoded in the image code stream, the obtained reconstructed image blocks can be aggregated.

[0125] In specific applications, in some cases, the scanning range can be reduced. In this case, step S30 in this embodiment may include:

[0126] If the scanned area is an area of ​​a preset shape type, the scanned area is decoded in sequence according to the area scanning order corresponding to the to-be-decoded transform block based on a preset coding skip rule to obtain quantized residual data.

[0127] It should be noted that the preset shape type and the preset encoding skip rule can be pre-set by the administrator of the encoding device or the decoding device according to actual needs.

[0128] For example: for preset shape types such as an isosceles right triangle, a right trapezoid with a hypotenuse of 45 degrees, or a pentagon with a hypotenuse of 45 degrees, under a given scanning order, when encoding the quantized residual data on the target edge (right straight edge, lower straight edge, lower right hypotenuse) of the scanning area, if the other quantized residual data on the target edge except the last quantized residual data are all 0, then the last quantized residual data on the target edge must be non-zero, or the sig of the quantized residual data (marking information indicating whether the current quantized residual data value is non-zero) must be 1.

[0129] This embodiment extracts the transform block to be decoded from the image code stream and obtains the regional syntax parameters corresponding to the transform block to be decoded; determines the scanning area in the transform block to be decoded according to the regional syntax parameters, the scanning area is the area corresponding to the non-zero data in the quantized residual data corresponding to the transform block to be decoded, and the scanning area includes a non-rectangular area; decodes the scanning area in sequence according to the regional scanning order corresponding to the transform block to be decoded to obtain the quantized residual data; and constructs the reconstructed image block corresponding to the transform block to be decoded according to the quantized residual data. Since the scanning area determined during decoding can be a non-rectangular area corresponding to the non-zero data in the quantized residual data, a more flexible and compact scanning area expression method can be selected when compressing the image, thereby reducing the bit cost of image encoding and improving compression performance.

[0130] refer to Figure 4 , Figure 4 The figure is a flowchart of a second embodiment of an image decoding method according to the present invention.

[0131] Based on the above first embodiment, step S20 of the image decoding method of this embodiment includes:

[0132] Step S201: Obtain the region shape type corresponding to the transform block to be decoded.

[0133] It should be noted that the region shape type corresponding to the transform block to be decoded may be the region shape type of the scan region in the transform block to be decoded. Depending on the shape of the scan region, the region shape type may include multiple types such as polygons, points or lines, wherein the polygon may include a triangle (such as a right triangle, an isosceles right triangle, etc.), a quadrilateral (such as a trapezoid, a rhombus, a parallelogram, etc.), a pentagon or a polygon with more sides.

[0134] In actual use, in order to adapt to more complex situations, the area shape type can also include combined shapes. The combined shape can be composed of at least two shapes of polygons, points or lines, and the same shapes can be combined. For example, the combined shape can be "triangle + point", "pentagon + point", "triangle + pentagon", "triangle + triangle" and other shapes.

[0135] In practical applications, one video bitstream corresponds to at least one transform block to be decoded, and the shapes of the scan areas in different transform blocks to be decoded may be the same or different, that is, the image bitstream corresponds to at least one type of area shape. In order to ensure that it can adapt to various actual scenarios in actual use, step S201 in this embodiment may include:

[0136] Determining the region shape type corresponding to the to-be-decoded transform block according to a preset type value;

[0137] or,

[0138] Derived according to the regional syntax parameters, determining the regional shape type corresponding to the to-be-decoded transform block;

[0139] or,

[0140] Obtaining a shape indication parameter corresponding to the transform block to be decoded;

[0141] The region shape type corresponding to the to-be-decoded transform block is selected from at least one region shape type corresponding to the image code stream according to the shape indication parameter.

[0142] It should be noted that obtaining the area shape type corresponding to the transform block to be decoded can be to determine the area shape type corresponding to the transform block to be decoded according to a preset type value. For example, if the scanning area shapes in the transform block to be decoded corresponding to a video code stream are the same (such as all using triangles, trapezoids, pentagons or the same type of combined shapes, etc.), then the preset type value can be used to mark the area shape type corresponding to the transform block to be decoded.

[0143] The region shape type corresponding to the transform block to be decoded can also be obtained by deducing based on the region syntax parameters to determine the region shape type corresponding to the transform block to be decoded, such as: if the shape of the scanned region in the transform block to be decoded corresponding to a video code stream is different, but the region syntax parameters corresponding to the shape of the adopted scanned region have obvious differences, then at this time, it can be attempted to deduce based on the number and / or value range of the region syntax parameters to determine the region shape type corresponding to the transform block to be decoded, for example: if the region shape of the scanned region in the transform block to be decoded corresponding to the video code stream may be a triangle or a pentagon, then at this time, the region shape type corresponding to the transform block to be decoded can be derived based on the number of region syntax parameters (if the number of region syntax parameters is 2, then the region shape type corresponding to the transform block to be decoded is determined to be a triangle; if the number of region syntax parameters is greater than 2, then the region shape type corresponding to the transform block to be decoded is determined to be a pentagon).

[0144] Of course, it is also possible to set corresponding shape indication parameters for each transform block to be decoded, and select the region shape type corresponding to the transform block to be decoded from at least one region shape type corresponding to the image code stream according to the shape indication parameters. For example, if the shapes of the scanned regions in the transform block to be decoded corresponding to a video code stream are different, and there is no obvious difference in the region syntax parameters corresponding to the shapes of the adopted scanned regions, then in order to determine the region shape type corresponding to the transform block to be decoded as accurately as possible, it is possible to set corresponding shape indication parameters for each transform block to be decoded in the image code stream, and select the region shape type corresponding to the transform block to be decoded from at least one region shape type corresponding to the image code stream according to the shape indication parameters.

[0145] Among them, the specific setting of the shape indication parameter can be determined according to the number of regional shape types corresponding to the image code stream. If the number of regional shape types corresponding to the image code stream is 2, the regional shape type corresponding to the transform block to be decoded can be determined by using a true / false method.

[0146] For example: if the region shape type corresponding to the image code stream is point / line+triangle (or a combined shape based on a triangle), then at this time, by setting the variable triangle_flag, if the triangle_flag corresponding to the transform block to be decoded is true, it means that the region shape type corresponding to the transform block to be decoded is a triangle (or a combined shape), if the triangle_flag corresponding to the transform block to be decoded is false, it means that the region shape type corresponding to the transform block to be decoded is a point / line;

[0147] Similarly, if the region shape type corresponding to the image code stream is a point / straight line + a pentagon with a hypotenuse of 45 degrees (or a combined shape based on a pentagon), the region shape type corresponding to the transform block to be decoded can be indicated by setting the variable pentagon_flag;

[0148] If the area shape type corresponding to the image code stream is point / line + triangle (or a combined shape constructed based on a triangle) + pentagon (or a combined shape constructed based on a pentagon), the area shape type corresponding to the transform block to be decoded can be detected by first detecting whether it is a point / line through the number of parameters and the value of the parameters in the area syntax parameter. If it is not a point / line, the variable triangle_pentagon_flag can be set to indicate whether the area shape type corresponding to the transform block to be decoded is a triangle (or a combined shape constructed based on a triangle) or a pentagon (or a combined shape constructed based on a pentagon).

[0149] If the number of regional shape types corresponding to the image code stream is greater than 2, the regional shape type corresponding to the transform block to be decoded can be determined by the value of the shape indication parameter. For example, when the value of the shape indication parameter is 0, it indicates that the regional shape type corresponding to the transform block to be decoded is a point / line; when the value of the shape indication parameter is 1, it indicates that the regional shape type corresponding to the transform block to be decoded is a triangle; when the value of the shape indication parameter is 2, it indicates that the regional shape type corresponding to the transform block to be decoded is a quadrilateral; when the value of the shape indication parameter is 3, it indicates that the regional shape type corresponding to the transform block to be decoded is a pentagon, and so on.

[0150] In practical applications, the shape indication parameter may be a preset parameter or may be obtained from a video bitstream through a TB-level syntax, wherein the TB-level syntax may be a value indicating that a shape indication parameter is set for each transform block.

[0151] Step S202: Determine a scan area in the to-be-decoded transform block according to the area syntax parameter and the area shape type.

[0152] It can be understood that after determining the region shape type, the shape of the specific scanning region can be clarified, and then combined with the region syntax parameter, the scanning region in the to-be-decoded transform block can be marked.

[0153] It should be noted that in order to facilitate the division of the scanning area, a coordinate system can be constructed in advance in the transform block to be decoded. For example, the upper left corner of the transform block to be decoded is the origin of the coordinate system, the right direction from the upper boundary of the transform block to be decoded is the positive direction of the x-axis of the coordinate system, and the downward direction from the left boundary of the transform block to be decoded is the positive direction of the y-axis of the coordinate system. The unit length of the coordinate system can be the side length of the 1x1 sub-block.

[0154] In the application process, the regional syntax parameters include at least a horizontal parameter (sr_x) and a vertical parameter (sr_y), wherein the horizontal parameter is the horizontal coordinate of the rightmost non-zero quantized residual data in the scan area; the vertical parameter is the vertical coordinate of the bottommost non-zero quantized residual data in the scan area. The value range of sr_x is [offset, min(tb_w, thres) – 1]; the value range of sr_y is [offset, min(tb_h, thres) – 1];

[0155] Wherein, tb_w and tb_h represent the width and height of the transform block TB respectively. thres is a preset threshold value, which is generally set to 32. The value of offset is: for a point or a line, its value is 0; for a right triangle, its value is 1; for a pentagon, its value is 2.

[0156] In specific applications, when encoding the horizontal parameter (sr_x) and the vertical parameter (sr_y) into the image code stream, the encoding device may directly encode the original value of the parameter into the image code stream, or encode the "original value-offset" into the image code stream, or jointly encode sr_x and sr_y, for example, first encode "min(sr_x,sr_y)" or "min(sr_x,sr_y)-offset", then encode "max(sr_x,sr_y)-min(sr_x,sr_y)", and finally encode the mark information of whether sr_x is greater than sr_y. The decoding device may calculate the horizontal parameter and the vertical parameter based on the data parsed from the image code stream using the reverse calculation method corresponding to the encoding method.

[0157] In actual use, when the region shape type is different, the number of parameters and parameter value range of the region syntax parameter will vary to a certain extent:

[0158] When the area shape type is point or line, the area syntax parameters may include horizontal parameters (sr_x) and vertical parameters (sr_y). When sr_x and sr_y are both 0, the scanning area is a point; when sr_x is 0 and sr_y is greater than 0, the scanning area is a vertical line; when sr_x is greater than 0 and sr_y is 0, the scanning area is a horizontal line.

[0159] When the region shape type is a triangle, the region syntax parameters mainly include horizontal parameters (sr_x) and vertical parameters (sr_y), and the triangular scanning area can be mainly divided into isosceles right triangle and non-isosceles right triangle;

[0160] For an isosceles right triangle, sr_x=sr_y exists at this time, and due to the particularity of an isosceles right triangle, the pixel positions on the hypotenuse of its area shape are all integers, so at this time, the scanning area of ​​the isosceles right triangle shape can be directly divided according to the horizontal parameter (sr_x) and the vertical parameter (sr_y);

[0161] If it is a non-isosceles right triangle, the pixel position on the hypotenuse of the region shape may be a non-integer. In this case, step S202 of this embodiment may include:

[0162] Constructing a region in the shape of a non-isosceles right triangle according to the region syntax parameters, and obtaining the region integer coordinates and non-integer coordinates on the hypotenuse of the triangle included in the constructed region;

[0163] Rounding the non-integer coordinates according to a rounding method to obtain hypotenuse integer coordinates;

[0164] A scanning area in the to-be-decoded transform block is determined according to the area integer coordinates and the hypotenuse integer coordinates.

[0165] It should be noted that when a region with a shape of a non-isosceles right triangle is constructed according to the region syntax parameters, the integer coordinates of the region contained in the constructed region and the non-integer coordinates on the hypotenuse of the triangular region that are not completely covered by the region can be obtained based on the pre-constructed coordinate system. At this time, in order to clarify the specific range of the scanning area, the non-integer coordinates can be rounded to obtain the integer coordinates of the hypotenuse.

[0166] Among them, the rounding method can be rounding off, rounding down or rounding up. The rounding methods in the encoding device and the decoding device need to be consistent. The rounding method can be pre-set in the encoding device and the decoding device by the administrator of the encoding device and / or the decoding device in the form of a preset value, or it can be transmitted through the image code stream. At this time, the decoding device can read it from the image code stream through the syntax. The syntax described here can be sequence level, frame level, Patch level, Slice level, Tile level, CTU level, CTB level, CU level, CB level or TB level.

[0167] In actual use, determining the scan area in the transform block to be decoded according to the area integer coordinates and the hypotenuse integer coordinates may be marking the area corresponding to the area integer coordinates and the hypotenuse integer coordinates as the scan area in the transform block to be decoded.

[0168] For ease of understanding, now combined Figure 5 This is for illustration but not for limitation. Figure 5 The triangular scanning area division diagram of this embodiment is shown in FIG. Figure 5As shown in a, the scanning area of ​​the non-isosceles right triangle shape area is divided as follows Figure 5 As shown in b.

[0169] When the area shape type is a polygon with more than three sides (such as a non-rectangular quadrilateral or pentagon), in order to accurately divide the scanning area, step S202 in this embodiment may include:

[0170] Get the angle value of the hypotenuse of the area shape in the scan area;

[0171] A scanning area in the to-be-decoded transform block is determined according to the angle value and the area syntax parameter.

[0172] It should be noted that the angle value of the hypotenuse of the regional shape can be the angle between the hypotenuse of the regional shape and the upper boundary of the transform block to be decoded. At this time, the regional syntax parameter includes not only the horizontal parameter (sr_x) and the vertical parameter (sr_y), but also a parameter value for indicating the position of the hypotenuse of the regional shape in the scanning area, that is, the diagonal parameter (sr_d). At this time, the horizontal parameter is the horizontal coordinate of the non-zero quantized residual data on the far right of the scanning area, or the length of the right angle in the horizontal direction; the vertical parameter is the vertical coordinate of the non-zero quantized residual data on the lower side of the scanning area, or the length of the right angle in the vertical direction; the diagonal parameter is the sum of the horizontal coordinate and the vertical coordinate of the non-zero quantized residual data on the lower right side of the scanning area, or the horizontal coordinate of the intersection of the extension line of the hypotenuse of the regional shape and the upper boundary of the transform block to be decoded.

[0173] Among them, for the pentagonal area, the value range of the diagonal parameter (sr_d) can be [max(sr_x,sr_y)+1, sr_x+sr_y–1], or [max(sr_x,sr_y), sr_x+sr_y–1]. In particular, when sr_d=max(sr_x,sr_y), the pentagon will degenerate into a right trapezoid or a right triangle. When the encoding device encodes the diagonal parameter (sr_d) into the image code stream, it may directly encode the original value of the diagonal parameter into the image code stream, or it may encode “original value - (max(sr_x, sr_y) + 1)”, “original value - max(sr_x, sr_y)”, “(sr_x + sr_y - 1) - original value” or “(sr_x + sr_y) - original value” into the image code stream. The decoding device may calculate and obtain the diagonal parameter using a reverse calculation method corresponding to the encoding method.

[0174] For some special quadrilaterals or pentagons, sr_d can be derived from sr_x and sr_y without coding sr_d. For example, for a right trapezoid, sr_d can be derived by setting sr_d = max(sr_x, sr_y); and for a pentagon with "sr_x = 2 or sr_y = 2", sr_d can be derived by setting sr_d = sr_x + sr_y - 1.

[0175] In actual use, the scanning area in the transform block to be decoded is determined according to the angle value and the area syntax parameter. The position of the hypotenuse of the area shape is determined according to the angle value and the diagonal parameter (sr_d), the position of the horizontal right-angled side is determined according to the horizontal parameter (sr_x), and the position of the vertical right-angled side is determined according to the vertical parameter (sr_y), and then a polygon is constructed according to the hypotenuse of the area shape, the horizontal right-angled side, and the vertical right-angled side.

[0176] In actual implementation, the angle value of the bevel of the regional shape in the scan area can be obtained by using a preset value as the angle value of the bevel of the regional shape, or by deducing the angle value of the bevel of the regional shape according to the horizontal parameter (sr_x) and the vertical parameter (sr_y) in the regional syntax parameter, or by extracting the angle value from the image code stream through the TB-level syntax. If the angle value is not set by any of the above methods, it means that the angle value is 45°.

[0177] For ease of understanding, now combined Figure 6 , Figure 7 and Figure 8 This is for illustration but not for limitation. Figure 6 Schematic diagram of the right-angle trapezoidal area division of this embodiment, Figure 7 Schematic diagram of pentagonal area division in this embodiment. Figure 8 is a schematic diagram of the combined shape area division of this embodiment. For a right-angled trapezoid, if sr_x>sr_y, the area division schematic diagram of the right-angled trapezoid is as follows: Figure 6 As shown in a, if sr_x<sr_y, then the area division diagram of the right-angled trapezoid is as follows Figure 6 As shown in b, the scanning area of ​​the pentagon with a hypotenuse of 45° is divided as follows Figure 7 As shown in a, the scanning area of ​​the pentagon whose hypotenuse is not 45° is divided as follows Figure 7 As shown in b.

[0178] In the specific implementation, when expressing a combined shape, if the combined shape is a combination of a point and other shapes, when the regional grammar parameters can remain unchanged, for example: when expressing a combined shape of "polygon + point" (such as a triangle + point, pentagon + point, etc.), the regional shape parameters can remain unchanged, and when dividing the polygon according to the regional shape parameters, the point with coordinates (sr_x, sr_y) (such as Figure 5 The Y in the figure is also included in the scanning area (the specific combined shape division, the isosceles right triangle + point shape area division is as follows Figure 8 As shown in a, the shape area of ​​non-isosceles right triangle + point is divided as follows Figure 8 As shown in b, the scanning area of ​​the right-angle trapezoid + point of sr_x>sr_y is divided as follows Figure 8 c, the scanning area of ​​the right-angle trapezoid + point of sr_x<sr_y is divided as follows Figure 8 As shown in d, the scanning area of ​​the pentagon + point with a hypotenuse of 45° is divided as follows Figure 8 As shown in e, the scanning area of ​​the pentagon with a hypotenuse of not 45° + the scanning area of ​​the point is divided as follows Figure 8 f); if the combined shape is a combination of polygons and lines or polygons and polygons, you can try to set multiple sets of regional syntax parameters to divide the regions separately, so as to determine the scanning area.

[0179] In this embodiment, the region shape type corresponding to the transform block to be decoded is obtained; and the scanning region in the transform block to be decoded is determined according to the region syntax parameter and the region shape type. Since the region shape type of the transform block to be decoded is obtained first, and then the scanning region in the transform block to be decoded is determined in combination with the region shape type and the region syntax parameter, it is ensured that the specific scanning region shape can be distinguished when dividing the scanning region, and the correctness of dividing the scanning region is ensured.

[0180] The embodiment of the present invention provides an image encoding method, referring to Fig. 9 , Fig. 9 FIG. 4 is a flow chart of a first embodiment of an image encoding method according to the present invention.

[0181] In this embodiment, the image encoding method includes the following steps:

[0182] Step S100: converting a to-be-encoded conversion block corresponding to a target image to obtain quantized residual data corresponding to the to-be-encoded conversion block.

[0183] It should be noted that the target image may be image data that needs to be encoded, and may be pre-set by the administrator of the encoding device. The to-be-encoded conversion block corresponding to the target image may be converted to obtain the quantized residual data corresponding to the to-be-encoded conversion block. The target image may be divided into at least one to-be-encoded conversion block, and then features of the to-be-encoded conversion block are extracted and mean predicted. The residual is obtained based on the extracted features and subtracted from the predicted mean, and then the residual is transformed and quantized to obtain the quantized residual data corresponding to the to-be-encoded conversion block.

[0184] Step S200: marking a scan area in the to-be-encoded conversion block according to non-zero data in the quantized residual data.

[0185] It should be noted that the scanning area may include a non-rectangular area, such as an area in the shape of a polygon, a point or a line, and of course, it may also be a combined shape, such as a shape constructed by a combination of at least two of the polygons, points or lines. Among them, the polygon may include a triangle (such as a right triangle, an isosceles right triangle, etc.), a quadrilateral (such as a trapezoid, a rhombus, a parallelogram, etc.), a pentagon or a polygon with more sides. The scanning area in the to-be-encoded conversion block may be marked according to the non-zero data in the quantized residual data, so that the scanning area includes the non-zero data in the quantized residual data as much as possible.

[0186] Step S300: sequentially encoding the quantized residual data corresponding to the scanned area according to the area scanning order corresponding to the to-be-encoded conversion block, to generate an image code stream corresponding to the target image.

[0187] It should be noted that the area scanning order may include at least one of horizontal scanning, vertical scanning, zigzag scanning and reverse zigzag scanning. The quantized residual data corresponding to the scanned area is sequentially encoded according to the area scanning order corresponding to the to-be-encoded conversion block, and the image code stream corresponding to the target image is generated by sequentially binarizing the quantized residual data contained in the scanned area according to the area scanning order corresponding to the to-be-encoded conversion block, and encoding the binarized components into the image code stream, thereby generating the image code stream corresponding to the target image.

[0188] Step S400: constructing a region syntax parameter corresponding to the to-be-encoded conversion block according to the scanned region, and writing the region syntax parameter into the image code stream.

[0189] It should be noted that the horizontal parameter (sr_x) and the vertical parameter (sr_y) are generated by constructing the region syntax parameters corresponding to the to-be-encoded conversion block according to the scan region, and the diagonal parameter (sr_d) may also be included if necessary, and the region syntax parameters may be encoded and written into the image bitstream for the convenience of decoding. The encoding method used in encoding the region syntax parameters into the image bitstream may be group encoding, specifically, the group number may be encoded by truncated unary code, the offset within the group may be encoded by fixed-length code, the truncated unary code may be encoded based on context, and the fixed-length code may be encoded by bypass.

[0190] When encoding the diagonal parameter (sr_d), the original value of the diagonal parameter can be directly encoded into the image code stream, or "original value-(max(sr_x,sr_y)+1)", "original value-max(sr_x,sr_y)", "(sr_x+sr_y-1)-original value" or "(sr_x+sr_y)-original value" can be encoded into the image code stream, and the decoding device can use the inverse calculation method corresponding to the encoding method to calculate and obtain the diagonal parameter.

[0191] In actual use, if the area scanning order is not set using a preset value, then in order to ensure that the area scanning order used during encoding and decoding is consistent, the area scanning order can be written into the image bitstream through syntax. The syntax used here can be sequence level, frame level, Patch level, Slice level, Tile level, CTU level, CTB level, CU level, CB level or TB level syntax.

[0192] Among them, in order to facilitate the decoding device to perform decoding, more data can be encoded into the image code stream, such as rounding method, angle value and other data. For the specific time when encoding is required, please refer to the specific situation description in any of the above-mentioned image decoding method embodiments, which will not be repeated here.

[0193] In a possible implementation of this embodiment, in order to reduce unnecessary scanning, step S300 in this embodiment may include:

[0194] If the scanned area is an area of ​​a preset shape type, the quantized residual data corresponding to the scanned area is encoded in sequence according to the area scanning order corresponding to the to-be-encoded conversion block based on a preset encoding skip rule to generate an image code stream corresponding to the target image.

[0195] It should be noted that the preset shape type and the preset encoding skip rule can be pre-set by the administrator of the encoding device or the decoding device according to actual needs.

[0196] For example: for preset shape types such as an isosceles right triangle, a right trapezoid with a hypotenuse of 45 degrees, or a pentagon with a hypotenuse of 45 degrees, under a given scanning order, when encoding the quantized residual data on the target edge (right straight edge, lower straight edge, lower right hypotenuse) of the scanning area, if the other quantized residual data on the target edge except the last quantized residual data are all 0, then the last quantized residual data on the target edge must be non-zero, or the sig of the quantized residual data (marking information indicating whether the current quantized residual data value is non-zero) must be 1.

[0197] This embodiment converts the conversion block to be encoded corresponding to the target image to obtain the quantized residual data corresponding to the conversion block to be encoded; marks the scan area in the conversion block to be encoded according to the non-zero data in the quantized residual data, and the scan area includes a non-rectangular area; encodes the quantized residual data corresponding to the scan area in sequence according to the area scan order corresponding to the conversion block to be encoded to generate an image code stream corresponding to the target image; constructs the area syntax parameters corresponding to the conversion block to be encoded according to the scan area, and writes the area syntax parameters into the image code stream. Since the scan area in the conversion block to be encoded is marked according to the non-zero data in the quantized residual data after the quantized residual data is obtained, the scan area includes the non-zero data in the quantized residual data as much as possible, and since the scan area can include a non-rectangular area, a more flexible and compact scan area expression method can be selected according to actual needs during encoding, thereby reducing the bit cost of image encoding and improving compression performance.

[0198] In addition, an embodiment of the present invention also proposes a storage medium, on which an image decoding program and / or an image encoding program is stored. When the image decoding program is executed by a processor, the steps of the image decoding method described above are implemented; when the image encoding program is executed by a processor, the steps of the image encoding method described above are implemented.

[0199] Reference Fig.10 , Fig.10 This is a structural block diagram of the first embodiment of the image decoding device of the present invention.

[0200] like Fig.10 As shown, the image decoding device proposed in the embodiment of the present invention includes:

[0201] The decoding module 10 is used to extract the transform block to be decoded from the image code stream and obtain the regional syntax parameters corresponding to the transform block to be decoded;

[0202] A construction module 20, configured to determine a scanning area in the to-be-decoded transform block according to the area syntax parameter, wherein the scanning area is a non-rectangular area adapted to non-zero data in the quantized residual data corresponding to the image code stream;

[0203] A scanning module 30, configured to sequentially decode the scanned areas according to the area scanning order corresponding to the to-be-decoded transform block to obtain quantized residual data;

[0204] The reconstruction module 40 is used to construct a reconstructed image block corresponding to the to-be-decoded transform block according to the quantized residual data.

[0205] This embodiment extracts the transform block to be decoded from the image code stream and obtains the regional syntax parameters corresponding to the transform block to be decoded; determines the scanning area in the transform block to be decoded according to the regional syntax parameters, the scanning area is the area corresponding to the non-zero data in the quantized residual data corresponding to the transform block to be decoded, and the scanning area includes a non-rectangular area; decodes the scanning area in sequence according to the regional scanning order corresponding to the transform block to be decoded to obtain the quantized residual data; and constructs the reconstructed image block corresponding to the transform block to be decoded according to the quantized residual data. Since the scanning area determined during decoding can be a non-rectangular area corresponding to the non-zero data in the quantized residual data, a more flexible and compact scanning area expression method can be selected when compressing the image, thereby reducing the bit cost of image encoding and improving compression performance.

[0206] In a possible implementation of this embodiment, the area scanning order is at least one of horizontal scanning, vertical scanning, zigzag scanning and reverse zigzag scanning; the area scanning order is a preset order, adaptively selected according to the area shape type of the scanning area, or extracted from the image code stream through syntax; the syntax is sequence level, frame level, Patch level, Slice level, Tile level, CTU level, CTB level, CU level, CB level or TB level.

[0207] In a possible implementation of this embodiment, the construction module 20 is further used to obtain the region shape type corresponding to the transform block to be decoded; and determine the scanning region in the transform block to be decoded according to the region syntax parameter and the region shape type.

[0208] In a possible implementation manner of this embodiment, the area shape type is a polygon, a point or a line.

[0209] In a possible implementation manner of this embodiment, the area shape type is a combined shape; the combined shape is formed by combining at least two shapes of a polygon, a point or a line.

[0210] In a possible implementation of this embodiment, the image code stream corresponds to at least one type of region shape;

[0211] The construction module 20 is further used to determine the region shape type corresponding to the transform block to be decoded according to a preset type value; or, derive according to the region syntax parameter to determine the region shape type corresponding to the transform block to be decoded; or, obtain a shape indication parameter corresponding to the transform block to be decoded; and select the region shape type corresponding to the transform block to be decoded from at least one region shape type corresponding to the image code stream according to the shape indication parameter.

[0212] In a possible implementation of this embodiment, the shape indication parameter is a preset parameter or is obtained from a video bit stream through a syntax, and the syntax is at the TB level.

[0213] In a possible implementation of this embodiment, the region shape type is a non-isosceles right triangle;

[0214] The construction module 20 is further used to construct a region with a shape of a non-isosceles right triangle according to the region syntax parameters, and obtain the region integer coordinates contained in the constructed region and the non-integer coordinates on the hypotenuse of the triangle; round the non-integer coordinates according to a rounding method to obtain the hypotenuse integer coordinates; and determine the scanning area in the transform block to be decoded according to the region integer coordinates and the hypotenuse integer coordinates.

[0215] In a possible implementation of this embodiment, the rounding method is rounding, rounding down or rounding up; the rounding method is a preset value or read from an image code stream through syntax; the syntax is sequence level, frame level, Patch level, Slice level, Tile level, CTU level, CTB level, CU level, CB level or TB level.

[0216] In a possible implementation of this embodiment, the area shape type is a polygon with more than three sides;

[0217] The construction module 20 is further configured to obtain an angle value of a hypotenuse of a region shape in a scanning region; and determine a scanning region in the to-be-decoded transform block according to the angle value and the region syntax parameter.

[0218] In a possible implementation manner of this embodiment, the regional syntax parameter is used to indicate the regional range of the scanning area.

[0219] In a possible implementation manner of this embodiment, the region syntax parameter includes a parameter value for indicating a position of a hypotenuse of a region shape in the scanning region.

[0220] In a possible implementation of this embodiment, the area syntax parameters include a horizontal parameter, a vertical parameter and a diagonal parameter; the horizontal parameter is the horizontal coordinate of the non-zero quantized residual data on the rightmost side of the scanning area, or the length of the right-angled side in the horizontal direction; the vertical parameter is the vertical coordinate of the non-zero quantized residual data on the bottom side of the scanning area, or the length of the right-angled side in the vertical direction; the diagonal parameter is the sum of the horizontal coordinate and the vertical coordinate of the non-zero quantized residual data on the bottom right side of the scanning area, or the horizontal coordinate of the intersection of the extension line of the hypotenuse of the area shape and the upper boundary of the transform block to be decoded.

[0221] In a possible implementation of this embodiment, the regional syntax parameters are extracted from the image code stream through syntax based on a context model; the context model is related to at least one of the width, height, area, color channel, luminance component channel, chrominance component channel, block transform type, and sub-block transform type of the transform block to be decoded.

[0222] In a possible implementation of this embodiment, the angle value is a preset value, or is derived according to the regional syntax parameter, or is extracted from the image code stream through syntax; the syntax is at the TB level.

[0223] In a possible implementation of this embodiment, the scanning module 30 is also used to decode the scanning area in sequence according to the area scanning order corresponding to the transform block to be decoded based on a preset coding skip rule if the scanning area is an area of ​​a preset shape type, so as to obtain quantized residual data.

[0224] refer to Fig.11 , Fig.11 FIG. 4 is a structural block diagram of the first embodiment of the image encoding device of the present invention.

[0225] like Fig.11 As shown, the image encoding device proposed in the embodiment of the present invention includes:

[0226] The conversion module 100 is used to convert the to-be-encoded conversion block corresponding to the target image to obtain quantized residual data corresponding to the to-be-encoded conversion block;

[0227] A construction module 200 is used to mark a scanning area in the to-be-encoded conversion block according to non-zero data in the quantized residual data, wherein the scanning area includes a non-rectangular area;

[0228] A generating module 300 is used to sequentially encode the quantized residual data corresponding to the scanned area according to the area scanning order corresponding to the to-be-encoded conversion block, so as to generate an image code stream corresponding to the target image;

[0229] The encoding module 400 is used to construct a region syntax parameter corresponding to the to-be-encoded conversion block according to the scanned region, and write the region syntax parameter into the image code stream.

[0230] This embodiment converts the conversion block to be encoded corresponding to the target image to obtain the quantized residual data corresponding to the conversion block to be encoded; marks the scan area in the conversion block to be encoded according to the non-zero data in the quantized residual data, and the scan area includes a non-rectangular area; encodes the quantized residual data corresponding to the scan area in sequence according to the area scan order corresponding to the conversion block to be encoded to generate an image code stream corresponding to the target image; constructs the area syntax parameters corresponding to the conversion block to be encoded according to the scan area, and writes the area syntax parameters into the image code stream. Since the scan area in the conversion block to be encoded is marked according to the non-zero data in the quantized residual data after the quantized residual data is obtained, the scan area includes the non-zero data in the quantized residual data as much as possible, and since the scan area can include a non-rectangular area, a more flexible and compact scan area expression method can be selected according to actual needs during encoding, thereby reducing the bit cost of image encoding and improving compression performance.

[0231] In a possible implementation of this embodiment, the area scanning order is at least one of horizontal scanning, vertical scanning, Z-shaped scanning, and reverse Z-shaped scanning; the area scanning order is a preset order and is adaptively selected according to the area shape type of the scanning area.

[0232] In a possible implementation of this embodiment, the area scanning order is written into the image code stream through syntax; the syntax is sequence level, frame level, Patch level, Slice level, Tile level, CTU level, CTB level, CU level, CB level or TB level.

[0233] In a possible implementation manner of this embodiment, the area shape type of the scanning area is a polygon, a point or a line.

[0234] In a possible implementation manner of this embodiment, the area shape type is a combined shape; the combined shape is formed by combining at least two shapes of a polygon, a point or a line.

[0235] In a possible implementation of this embodiment, the generation module 300 is also used to encode the quantized residual data corresponding to the scanned area in sequence according to the area scanning order corresponding to the to-be-encoded conversion block based on a preset encoding skip rule if the scanned area is an area of ​​a preset shape type, so as to generate an image code stream corresponding to the target image.

[0236] It should be understood that the above is only an example and does not constitute any limitation on the technical solution of the present invention. In specific applications, technicians in this field can make settings as needed, and the present invention does not limit this.

[0237] It should be noted that the workflow described above is merely illustrative and does not limit the scope of protection of the present invention. In practical applications, technicians in this field can select part or all of them according to actual needs to achieve the purpose of the present embodiment, and no limitation is made here.

[0238] In addition, for technical details that are not described in detail in this embodiment, reference may be made to the image decoding method and / or image encoding method provided in any embodiment of the present invention, and will not be repeated here.

[0239] In addition, it should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0240] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0241] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as a read-only memory (ROM) / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0242] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. An image decoding method, characterized in that: The image decoding method comprises: Extracting a transform block to be decoded from an image code stream, and obtaining a regional syntax parameter corresponding to the transform block to be decoded; Determine a scanning area in the transform block to be decoded according to the area syntax parameter, the scanning area being an area adapted to non-zero data in the quantized residual data corresponding to the transform block to be decoded, and the scanning area including a non-rectangular area; Decoding the scanned areas in sequence according to the area scan order corresponding to the to-be-decoded transform block to obtain quantized residual data; A reconstructed image block corresponding to the to-be-decoded transform block is constructed according to the quantized residual data.

2. The image decoding method according to claim 1, characterized in that: Include at least one of the following: The area scanning sequence is at least one of horizontal scanning, vertical scanning, zigzag scanning and reverse zigzag scanning; The area scanning order is a preset order, adaptively selected according to the area shape type of the scanning area, or extracted from the image code stream through syntax; The syntax is sequence level, frame level, patch level, slice level, tile level, CTU level, CTB level, CU level, CB level or TB level.

3. The image decoding method according to claim 1, wherein: The determining the scanning area in the to-be-decoded transform block according to the area syntax parameter comprises: Obtaining a region shape type corresponding to the transform block to be decoded; A scanning area in the to-be-decoded transform block is determined according to the area syntax parameter and the area shape type.

4. The image decoding method according to claim 3, characterized in that: The area shape type is polygon, point or line.

5. The image decoding method according to claim 3, wherein: Include at least one of the following: The area shape type is a combined shape; The combined shape is formed by combining at least two shapes of polygons, points or lines.

6. The image decoding method according to claim 3, characterized in that: The image code stream corresponds to at least one type of region shape; The obtaining the region shape type corresponding to the to-be-decoded transform block includes: Determining the region shape type corresponding to the to-be-decoded transform block according to a preset type value; or, Derived according to the regional syntax parameters, determining the regional shape type corresponding to the to-be-decoded transform block; or, Obtaining a shape indication parameter corresponding to the transform block to be decoded; The region shape type corresponding to the to-be-decoded transform block is selected from at least one region shape type corresponding to the image code stream according to the shape indication parameter.

7. The image decoding method according to claim 6, characterized in that: The shape indication parameter is a preset parameter or is obtained from a video bit stream through a syntax, and the syntax is at the TB level.

8. The image decoding method according to claim 4, characterized in that: The shape type of the region is a non-isosceles right triangle; The determining the scanning area in the to-be-decoded transform block according to the area syntax parameter and the area shape type comprises: Constructing a region in the shape of a non-isosceles right triangle according to the region syntax parameters, and obtaining the region integer coordinates and non-integer coordinates on the hypotenuse of the triangle included in the constructed region; Rounding the non-integer coordinates according to a rounding method to obtain hypotenuse integer coordinates; A scanning area in the to-be-decoded transform block is determined according to the area integer coordinates and the hypotenuse integer coordinates.

9. The image decoding method according to claim 8, characterized in that: Include at least one of the following: The rounding method is rounding off, rounding down or rounding up; The rounding mode is a preset value or is read from the image code stream through syntax; The syntax is sequence level, frame level, patch level, slice level, tile level, CTU level, CTB level, CU level, CB level or TB level.

10. The image decoding method according to claim 4, characterized in that: The area shape type is a polygon with more than three sides; The determining the scanning area in the to-be-decoded transform block according to the area syntax parameter and the area shape type comprises: Get the angle value of the hypotenuse of the area shape in the scan area; A scanning area in the to-be-decoded transform block is determined according to the angle value and the area syntax parameter.

11. The image decoding method according to claim 1, wherein: The area syntax parameter is used to indicate the area range of the scanning area.

12. The image decoding method according to claim 10, characterized in that: The region syntax parameters include parameter values ​​for indicating the positions of the hypotenuses of the region shapes in the scan region.

13. The image decoding method according to claim 12, wherein: The regional syntax parameters include horizontal parameters, vertical parameters and diagonal parameters; The horizontal parameter is the horizontal coordinate of the non-zero quantized residual data on the rightmost side of the scanning area, or the length of the right-angled side in the horizontal direction; The vertical parameter is the ordinate of the non-zero quantized residual data at the bottom of the scanning area, or the length of the right-angled side in the vertical direction; The diagonal parameter is the sum of the horizontal coordinate and the vertical coordinate of the non-zero quantized residual data at the lower right side of the scanning area, or the horizontal coordinate of the intersection of the extension line of the hypotenuse of the area shape and the upper boundary of the to-be-decoded transform block.

14. The image decoding method according to claim 13, wherein: The regional grammar parameters are extracted from the image code stream through grammar based on the context model; The context model is related to at least one of the following information: width, height, area, color channel, luminance component channel, chrominance component channel, block transform type, and sub-block transform type of the transform block to be decoded.

15. The image decoding method according to claim 10, wherein: Include at least one of the following: The angle value is a preset value, or is derived according to the regional syntax parameter, or is extracted from the image code stream through syntax; The grammar is on the terabyte scale.

16. The image decoding method according to any one of claims 1 to 15, characterized in that: The decoding of the scanned areas in sequence according to the area scanning order corresponding to the to-be-decoded transform block to obtain quantized residual data comprises: If the scanned area is an area of ​​a preset shape type, the scanned area is decoded in sequence according to the area scanning order corresponding to the to-be-decoded transform block based on a preset coding skip rule to obtain quantized residual data.

17. An image encoding method, characterized in that: The image encoding method comprises: Converting the to-be-encoded conversion block corresponding to the target image to obtain quantized residual data corresponding to the to-be-encoded conversion block; Marking a scan area in the to-be-encoded conversion block according to non-zero data in the quantized residual data, wherein the scan area includes a non-rectangular area; Encoding the quantized residual data corresponding to the scanned area in sequence according to the area scanning order corresponding to the to-be-encoded conversion block, to generate an image code stream corresponding to the target image; A region syntax parameter corresponding to the to-be-encoded conversion block is constructed according to the scanned region, and the region syntax parameter is written into the image code stream.

18. The image encoding method according to claim 17, wherein: Include at least one of the following: The area scanning sequence is at least one of horizontal scanning, vertical scanning, zigzag scanning, and reverse zigzag scanning; The area scanning order is a preset order, which is adaptively selected according to the area shape type of the scanning area.

19. The image encoding method according to claim 18, characterized in that: Include at least one of the following: The area scanning order is written into the image code stream through syntax; The syntax is sequence level, frame level, patch level, slice level, tile level, CTU level, CTB level, CU level, CB level or TB level.

20. The image encoding method according to claim 17, wherein: The area shape type of the scanning area is a polygon, a point or a line.

21. The image encoding method according to claim 20, characterized in that: Include at least one of the following: The area shape type is a combined shape; The combined shape is formed by combining at least two shapes of polygons, points or lines.

22. The image encoding method according to any one of claims 17 to 21, characterized in that: The step of sequentially encoding the quantized residual data corresponding to the scanned area according to the area scanning order corresponding to the to-be-encoded conversion block to generate an image code stream corresponding to the target image includes: If the scanned area is an area of ​​a preset shape type, the quantized residual data corresponding to the scanned area is encoded in sequence according to the area scanning order corresponding to the to-be-encoded conversion block based on a preset encoding skip rule to generate an image code stream corresponding to the target image.

23. An image decoding device, characterized in that: The image decoding device comprises: A decoding module, used for extracting a transform block to be decoded from an image code stream, and obtaining a regional syntax parameter corresponding to the transform block to be decoded; A construction module, configured to determine a scanning area in the to-be-decoded transform block according to the area syntax parameter, wherein the scanning area is a non-rectangular area adapted to non-zero data in the quantized residual data corresponding to the image code stream; A scanning module, used for sequentially decoding the scanning areas according to the area scanning order corresponding to the transform block to be decoded, to obtain quantized residual data; A reconstruction module is used to construct a reconstructed image block corresponding to the to-be-decoded transform block according to the quantized residual data.

24. An image encoding device, characterized in that: The image encoding device comprises: A conversion module, used for converting the to-be-encoded conversion block corresponding to the target image to obtain quantized residual data corresponding to the to-be-encoded conversion block; A construction module, configured to mark a scanning area in the to-be-encoded conversion block according to non-zero data in the quantized residual data, wherein the scanning area includes a non-rectangular area; A generating module, used for sequentially encoding the quantized residual data corresponding to the scanned area according to the area scanning order corresponding to the to-be-encoded conversion block, to generate an image code stream corresponding to the target image; The encoding module is used to construct a region syntax parameter corresponding to the to-be-encoded conversion block according to the scanned region, and write the region syntax parameter into the image code stream.

25. A decoding device, characterized in that: The decoding device comprises: a processor, a memory, and an image decoding program stored in the memory and executable on the processor, wherein the image decoding program implements the image decoding method according to any one of claims 1 to 16 when executed by the processor.

26. A coding device, characterized in that The encoding device comprises: a processor, a memory, and an image encoding program stored in the memory and executable on the processor, wherein the image encoding program implements the image encoding method according to any one of claims 17 to 22 when executed by the processor.

27. A storage medium, characterized in that: The storage medium stores an image decoding program and / or an image encoding program. When the image decoding program is executed, the image decoding method according to any one of claims 1 to 16 is implemented. When the image encoding program is executed, the image encoding method according to any one of claims 17 to 22 is implemented.

Citation Information

Patent Citations

  • Method and apparatus for encoding / decoding image, and recording medium in which bit stream is stored

    CN110024399A

  • Method and device for deciding transform coefficient scan order on basis of high frequency zeroing

    CN111587575A

  • Image coding method on basis of transformation and device therefor

    CN111684809A

  • Method for decoding video and apparatus therefor and method for encoding video and apparatus therefor

    CN111727606A

  • Residual coefficient coding and decoding

    CN114365492A