Video encoding method and device, video decoding method and device, and electronic device

Abstract

The present disclosure provides a video encoding method and device, a video decoding method and device, and an electronic device, and relates to the technical field of digital multimedia. The video encoding method comprises: obtaining a target image block in a video frame; traversing an uncoded image block in the video frame to determine at least one first image block that meets a preset rule; obtaining a first residual matrix of the target image block and the at least one first image block respectively; fusing the first residual matrix of the target image block and the at least one first image block to obtain a second residual matrix; and in a case where the encoding cost of the second residual matrix is less than the sum of the encoding costs of the first residual matrix of the target image block and the at least one first image block, obtaining the encoding data corresponding to the target image block and the at least one first image block based on the second residual matrix. In this way, the expression efficiency of the encoding data can be improved, the code rate of the video quantization output can be greatly reduced, and the video compression efficiency and the video encoding performance are improved as a whole.

Description

Technical Field

[0001] This disclosure relates to the field of digital multimedia technology, specifically to a video encoding method and apparatus, a video decoding method and apparatus, and an electronic device. Background Technology

[0002] Digital multimedia technology utilizes computer and communication technologies to process, store, transmit, and display digital information. Among these technologies, video, as an emerging information carrier, possesses advantages such as vividness, diversity, and ease of understanding, and has broad application prospects in information transmission and social communication. Therefore, video coding technology has become a current research hotspot. Related research institutions have invested significant resources and effort in improving the efficiency of video coding, aiming to minimize data volume while maintaining visual quality.

[0003] Video consists of consecutive video frames, and these frames typically exhibit strong similarity, meaning they contain a significant amount of redundant information. Related techniques use predicted images and residuals to describe video frames, reducing spatial and temporal redundancy and thus improving video compression efficiency. However, residuals still contain some redundancy and internal correlations, impacting video compression efficiency and video coding performance. Summary of the Invention

[0004] In view of this, the present disclosure provides a video encoding method and apparatus, a video decoding method and apparatus, and an electronic device to solve the encoding performance problem in the related art.

[0005] In a first aspect, a video coding method is provided, comprising: acquiring a target image block in a video frame; traversing uncoded image blocks in the video frame to determine at least one first image block that conforms to a preset rule; acquiring a first residual matrix for each of the target image block and the at least one first image block; fusing the first residual matrices of the target image block and the at least one first image block to obtain a second residual matrix; and, if the coding cost of the second residual matrix is ​​less than the sum of the coding costs of the first residual matrices of the target image block and the at least one first image block, obtaining coded data corresponding to the target image block and the at least one first image block based on the second residual matrix.

[0006] In some embodiments, traversing uncoded image blocks in a video frame and determining at least one first image block that conforms to a preset rule includes: traversing uncoded image blocks in a video frame and determining image blocks whose prediction mode is inter-frame prediction as second image blocks; traversing second image blocks in a video frame and determining k second image blocks with the same area as the target image block and adjacent to the target image block as first image blocks, where k is a positive integer.

[0007] In some embodiments, traversing uncoded image blocks in a video frame and determining at least one first image block that conforms to a preset rule includes: traversing uncoded image blocks in a video frame and determining an image block whose prediction mode is inter-frame prediction as a third image block; traversing the third image blocks in a video frame and determining at least one third image block whose area sums with that of the target image block is equal to a preset area and which is adjacent to the target image block as a first image block.

[0008] In some embodiments, the encoding cost is the rate-distortion cost, which is calculated based on the bitstream length of the encoded data of the target image block and the degree of video distortion.

[0009] In some embodiments, the video coding method further includes: if the coding cost of the second residual matrix is ​​greater than or equal to the coding cost of the first residual matrix of the target image block, obtaining the coded data corresponding to the target image block based on the first residual matrix of the target image block.

[0010] In some embodiments, the prediction mode for the target image patch is inter-frame prediction.

[0011] In a second aspect, a video decoding method is provided, comprising: acquiring encoded data corresponding to a target image block and at least one first image block, wherein the encoded data includes a second residual matrix corresponding to the target image block and at least one first image block, and the encoded data is obtained based on the method of the first aspect; acquiring a preset rule corresponding to the target image block; splitting the second residual matrix based on the preset rule to obtain multiple first residual matrices; and decoding the encoded data into a target image block and at least one first image block based on the multiple first residual matrices.

[0012] Thirdly, a video encoding apparatus is provided, comprising: a first acquisition module for acquiring a target image block in a video frame; a determination module for traversing unencoded image blocks in the video frame and determining at least one first image block that conforms to a preset rule; a second acquisition module for acquiring a first residual matrix of the target image block and the at least one first image block respectively; a fusion module for fusing the first residual matrices of the target image block and the at least one first image block to obtain a second residual matrix; and an encoding module for obtaining encoded data corresponding to the target image block and the at least one first image block based on the second residual matrix, provided that the encoding cost of the second residual matrix is ​​less than the sum of the encoding costs of the first residual matrices of the target image block and the at least one first image block.

[0013] Fourthly, a video decoding apparatus is provided, comprising: a first acquisition module for acquiring encoded data corresponding to a target image block and at least one first image block, the encoded data including a second residual matrix corresponding to the target image block and at least one first image block, the encoded data being obtained based on the method of the first aspect; a second acquisition module for acquiring a preset rule corresponding to the target image block; a splitting module for splitting the second residual matrix based on the preset rule to obtain multiple first residual matrices; and a decoding module for decoding the encoded data into a target image block and at least one first image block based on the multiple first residual matrices.

[0014] Fifthly, an electronic device is provided, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to perform the methods of the first and second aspects described above by executing the executable instructions.

[0015] In a sixth aspect, a computer-readable storage medium is provided having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the methods of the first and second aspects.

[0016] The video coding method provided in this disclosure optimizes the internal correlation of the residual matrix by fusing the residual matrices of the target image block and the first image block conforming to preset rules to obtain a second residual matrix. It then compares the sum of the coding costs of the second residual matrix and the first residual matrices of the target image block and the first image block. If the coding cost of the second residual matrix is ​​lower, the coded data corresponding to the target image block and the first image block is obtained based on the second residual matrix. This improves the expressive efficiency of the coded data, significantly reduces the bitrate of the video quantization output, and overall improves video compression efficiency and video coding performance. Attached Figure Description

[0017] Figure 1 This diagram illustrates a flow chart of a video encoding method according to an embodiment of the present disclosure.

[0018] Figure 2 This illustration shows a flowchart of an embodiment of the present disclosure of traversing unencoded image blocks in a video frame to determine at least one first image block that conforms to a preset rule.

[0019] Figure 3 This illustration shows another flowchart in an embodiment of the present disclosure for traversing unencoded image blocks in a video frame to determine at least one first image block that conforms to a preset rule.

[0020] Figure 4 The diagram shows a flowchart of a video decoding method according to an embodiment of the present disclosure.

[0021] Figure 5A schematic diagram of the structure of a video encoding device according to an embodiment of the present disclosure is shown.

[0022] Figure 6 A schematic diagram of the structure of a video decoding device according to an embodiment of the present disclosure is shown.

[0023] Figure 7 A schematic diagram of the structure of an electronic device according to an embodiment of the present disclosure is shown. Detailed Implementation

[0024] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, they are provided so that this disclosure will be more comprehensive and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0025] Furthermore, the accompanying drawings are merely illustrative of this disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0026] Video coding technology can remove redundant information from the original video file through compression algorithms, reducing the file size and making the video file easier to transmit, store, and process. Related technologies divide each video frame into multiple image blocks. For each image block, intra-frame prediction or inter-frame prediction is used to predict the corresponding predicted image block. The difference between the predicted and predicted image blocks is then used to obtain the residual matrix of the image block. Next, frequency domain transformation, quantization, and entropy coding are performed on the residual matrix to obtain the encoded data of the image block. During this process, the frequency domain transformation aims to ensure that the elements of the residual matrix satisfy a certain distribution relationship to achieve better compression. However, in practical applications, the residual matrix often fails to meet this condition, thus compromising coding performance.

[0027] In view of this, the present disclosure provides a video coding method, comprising: acquiring a target image block in a video frame; traversing uncoded image blocks in the video frame to determine at least one first image block conforming to a preset rule; acquiring a first residual matrix for each of the target image block and the at least one first image block; fusing the first residual matrices of the target image block and the at least one first image block to obtain a second residual matrix; and, if the coding cost of the second residual matrix is ​​less than the sum of the coding costs of the first residual matrices of the target image block and the at least one first image block, obtaining coded data corresponding to the target image block and the at least one first image block based on the second residual matrix. This improves the expressive efficiency of the coded data, significantly reduces the bitrate of the video quantization output, and overall improves video compression efficiency and video coding performance.

[0028] This disclosure provides a video encoding method, a video decoding method, an apparatus, an electronic device, and a storage medium. The video encoding apparatus and / or video decoding apparatus can be integrated into an electronic device, which may be a smartphone, a camera, a surveillance camera, or a server.

[0029] It is understood that the video encoding and / or video decoding methods of the embodiments of this disclosure can be executed on a terminal, on a server, or jointly by a terminal and a server. This should not be construed as a limitation of this disclosure.

[0030] Furthermore, the video encoding and decoding methods of this disclosure can be applied to any video encoder and decoder, and are applicable to any video encoding standard such as H.265, H.264, AVS, and AV1. It is understood that the above examples are merely intended to clearly describe the specific implementation methods of the embodiments of this disclosure and should not be construed as limiting this disclosure.

[0031] The following will describe the exemplary implementation method in detail with reference to the accompanying drawings and embodiments.

[0032] First, this disclosure provides a video encoding method that can be executed by any electronic device with computing capabilities.

[0033] Figure 1 This diagram illustrates a flow chart of a video encoding method according to an embodiment of the present disclosure, as shown below. Figure 1 As shown in the embodiments of this disclosure, the video encoding method includes the following steps.

[0034] S101, acquire the target image block in the video frame.

[0035] Specifically, a video consists of a series of consecutive still images, and a video frame is a still image in the video. The encoder divides each video frame into multiple image blocks according to specific rules. The encoder can divide a video frame into multiple image blocks of the same size, or into multiple image blocks of slightly different sizes. Each image block includes the chroma and luminance information of a portion of the image in the video frame. For example, the encoder can first divide the video frame into multiple coding tree units of the same size according to a fixed size, and then further divide each coding tree unit into multiple image blocks according to specific rules.

[0036] In some embodiments, any image block can be selected as the target image block. After processing the current target image block, other unprocessed image blocks in the video frame are selected as new target image blocks. This ensures that the method of this embodiment can traverse all image blocks in the video frame.

[0037] S102, traverse the unencoded image blocks in the video frame and determine at least one first image block that conforms to the preset rules.

[0038] In some embodiments, the encoder traverses all unencoded image blocks in the video frame containing the target image block and determines at least one image block that conforms to a preset rule as the first image block. For example, the encoder starts searching from the unencoded image block with the closest number to the target image block according to the image block numbering order, and stops searching when an image block conforming to the preset rule is found. The image blocks in the same video frame can be numbered in a scanning order from left to right and from top to bottom, and each image block has a unique number.

[0039] For example, the preset rule may be a rule set for the area of ​​the image block; or, the preset rule may be a rule set for the length and / or width of the image block.

[0040] In some embodiments, if the encoder traverses all unencoded image blocks in the video frame where the target image block is located and does not find an image block that meets the preset rules, the target image block is encoded separately to obtain the encoded data corresponding to the target image block, and the processing of the current target image block ends.

[0041] S103, respectively obtain the first residual matrix of the target image block and at least one first image block.

[0042] During the encoding process, the encoder calculates residual information by comparing the original image block with its predicted image block, and stores this residual information in the encoded information, thereby reducing the data volume of the video bitstream and improving compression efficiency and video quality. The predicted image block is calculated by the encoder based on a prediction algorithm, and the residual information is typically represented as a matrix. Therefore, by performing the above processing on the target image block and at least one first image block respectively, the first residual matrix of each of the target image block and at least one first image block can be obtained.

[0043] S104, fuse the first residual matrices of the target image block and at least one first image block to obtain the second residual matrix.

[0044] In some embodiments, multiple first residual matrices or their transposes can be concatenated into a larger matrix to serve as the second residual matrix. The elements of the second residual matrix are the set of all elements from the multiple first residual matrices.

[0045] For example, if the first residual matrix consists of an 8×4 matrix A and a 4×8 matrix B, then by merging the two matrices, the transposes of A and B can be arranged horizontally to obtain an 8×8 second residual matrix (A|B). T ) 8×8 Alternatively, by transposing A and arranging B vertically, we can obtain an 8×8 second residual matrix.

[0046] For example, if the first residual matrix consists of a 4×4 matrix C and a 4×4 matrix D, then by merging the two matrices, C and D can be arranged horizontally to obtain a 4×8 second residual matrix (C|D). 4×8 Alternatively, C and D can be arranged vertically to obtain a second residual matrix of order 8×4.

[0047] The above combination method can combine multiple first residual matrices into a single second residual matrix. The combination method is simple, easy to restore, and does not lose the information contained in the original first residual matrix.

[0048] In some embodiments, the elements in the second residual matrix can be rearranged through matrix transformation, so that information with similar energy values ​​in the second residual matrix is ​​clustered together, thereby improving the energy concentration and making the second residual matrix exhibit better coding performance.

[0049] S105, if the encoding cost of the second residual matrix is ​​less than the sum of the encoding costs of the first residual matrices of the target image block and at least one first image block, the encoded data corresponding to the target image block and at least one first image block is obtained based on the second residual matrix.

[0050] In some embodiments, a frequency domain transformation is performed on the second residual matrix. The frequency domain transformation is an orthogonal transformation that transforms the image from the spatial domain to another domain (e.g., the frequency domain). The orthogonal transformation simplifies complex calculations in the spatial domain, resulting in the frequency domain coefficients of the second residual matrix. For example, the frequency domain transformation can be a Discrete Cosine Transform (DCT) or a Discrete Sine Transform (DST). The frequency domain transformation can compress most of the energy in the information into fewer coefficients, further improving energy concentration.

[0051] The frequency domain coefficients of the second residual matrix contain a large amount of information. To further compress this information, the frequency domain coefficients need to be quantized, categorizing the parameters into specific value ranges. For example, each parameter in the frequency domain coefficient can be divided by a scaling factor to obtain an integer or an integer with a remainder. The remainder can then be rounded off, ultimately quantizing the frequency domain coefficients into multiple integers. This process will result in a loss of some precision (i.e., image quality), but it removes redundant information and significantly improves coding efficiency, thus achieving the goal of compressing the information.

[0052] Entropy coding is performed on the quantized frequency domain coefficients to finally obtain the encoded data of the second residual matrix, which is the encoded data corresponding to the target image block and at least one first image block.

[0053] In some embodiments, although the transformed encoded data has high compression efficiency and low bit rate, the amount of information lost is too large, resulting in significant image distortion after decoding. Conversely, if too much information is retained, the image distortion is smaller, and the bit rate will increase accordingly. Therefore, encoding cost can be used to measure the relationship between image quality and encoding efficiency. Encoding cost characterizes the degree of distortion of different methods at a given bit rate; the lower the encoding cost of a method, the higher its encoding performance.

[0054] Therefore, when faced with two schemes—encoding multiple image blocks simultaneously based on the second residual matrix or encoding the target image block individually—the encoding costs of the first residual matrix of the target image block, the first residual matrix of the first image block, and the second residual matrix can be calculated separately. If the encoding cost of the second residual matrix is ​​less than the sum of the encoding costs of the first residual matrices of the target image block and the first image block, then encoding the target image block and the first image block based on the second residual matrix is ​​chosen to obtain the corresponding encoded data.

[0055] In some embodiments, one bit is used in the encoded data to record whether the current encoded data is encoded based on the second residual matrix, so that when decoding the video encoding later, the decoder can determine whether the second residual matrix needs to be split into multiple first residual matrices.

[0056] Through the above steps, the method of this embodiment can obtain a second residual matrix by fusing the residual matrices of the target image block and a first image block conforming to preset rules, thereby optimizing the internal correlation of the residual matrix. It then compares the encoding cost of the second residual matrix with the sum of the encoding costs of the first residual matrices of the target image block and the first image block. When the encoding cost of the second residual matrix is ​​lower, the encoded data corresponding to the target image block and the first image block is obtained based on the second residual matrix. This improves the expressive efficiency of the encoded data, significantly reduces the bitrate of the video quantization output, and overall improves video compression efficiency and video encoding performance.

[0057] In some embodiments, the rate-distortion cost of the residual matrix can be used as the coding cost. The rate-distortion cost is calculated based on the bitstream length of the encoded data for the target image patch and the degree of video distortion, and can be written as:

[0058] J=D+λR

[0059] Where J is the rate-distortion cost, R is the bitstream length of the encoded data, λ is the Lagrange coefficient, and D is the degree of distortion. The degree of distortion is obtained by inverse quantization and inverse transform of the encoded data, reconstructing it with the predicted image block, and then subtracting the reconstructed image block from the original target image block point by point. The smaller the rate-distortion cost of the residual matrix, the smaller the degree of distortion at a given bitrate, i.e., the lower the encoding cost.

[0060] Rate-distortion cost effectively reflects the relationship between coding cost, bitstream length, and distortion level; furthermore, by adjusting λ, the weights between bitstream length and distortion level can be flexibly adjusted. Therefore, using rate-distortion cost as the coding cost offers good performance.

[0061] In some embodiments, if the encoding cost of the second residual matrix is ​​greater than or equal to the sum of the encoding costs of the first residual matrices of the target image block and the first image block, then the target image block is encoded based on the first residual matrix of the target image block to obtain the encoded data corresponding to the target image block; simultaneously, the first image block is not encoded, and it participates in the subsequent encoding process as an unencoded image block. During encoding based on the first residual matrix of the target image block, frequency domain transformation and quantization of the first residual matrix are also required.

[0062] By following the steps above, it can be ensured that it will not be used as the first image block of a new target image block in subsequent processing, thus avoiding repeated calculations and processing, saving computing power and improving coding performance.

[0063] The following describes an implementation method in this disclosure for traversing unencoded image blocks in video frames and determining at least one first image block that conforms to preset rules. For example... Figure 2 As shown in the embodiments of this disclosure, the process of traversing unencoded image blocks in video frames and determining at least one first image block that conforms to preset rules includes the following steps.

[0064] S201, Traverse the uncoded image blocks in the video frames and determine the image block with the prediction mode of inter-frame prediction as the second image block.

[0065] As described above, the predicted image patch is calculated by the encoder based on a prediction algorithm. Specifically, during the encoding process, the encoder predicts the unencoded image patch based on the reference image with the highest similarity to the current unencoded image patch and certain algorithms (such as motion compensation techniques). Each unencoded image patch corresponds to one predicted image patch. Furthermore, depending on the location of the reference image, the above prediction process is divided into two prediction modes: intra-frame prediction and inter-frame prediction.

[0066] In intra-frame prediction mode, the brightness and chromaticity information of adjacent pixels in a video frame are relatively similar and change gradually without abrupt changes. Therefore, already encoded image blocks in the same video frame can be used as reference images to predict uncoded image blocks, thereby reducing spatial redundancy. In the actual encoding process, the encoded data of already encoded image blocks is decoded into the original image blocks, which are then used as reference images for uncoded image blocks.

[0067] Alternatively, in inter-frame prediction mode, the changes between consecutive video frames within the same video are small, and the temporal correlation of the video is high. Therefore, it is relatively easy to find an image block in a video frame other than the one containing the uncoded image block, which is very similar to the uncoded image block, as a reference image to predict the uncoded image block, thereby reducing temporal redundancy.

[0068] Since the reference image for an image block in the intra-frame prediction mode is an image block within the current video frame, when executing the method of this embodiment, it cannot be guaranteed that the reference image for the first image block conforming to the preset rules is an encoded image block. Therefore, an additional algorithm is needed to ensure the continued execution of the method of this embodiment. Thus, in some embodiments, image blocks from the inter-frame prediction mode can be selected first as second image blocks. Then, image blocks conforming to the preset rules are selected from the second image blocks as the first image block, ensuring the complete execution of the method of this embodiment without increasing the computational load excessively.

[0069] S202, traverse the second image blocks in the video frame, and determine the k second image blocks that have the same area as the target image block and are adjacent to the target image block as the first image blocks.

[0070] In some embodiments, a preset rule can be set to determine the k nearest second image blocks to the target image block, which have the same area as the target image block. Here, k is a positive integer, and the specific value of k can be set according to actual application requirements.

[0071] In some embodiments, the second image block's neighboring target image block can be understood as the one whose number is closest to the target image block, thereby ensuring that the method of this embodiment is carried out in an orderly manner and that too many image blocks that meet the preset rules are not searched out, so as to save computing power and improve encoding performance.

[0072] For example, Table 1 shows information about some image blocks in the video frame where the target image block is located, where the width and height are in pixels and the area is in square pixels. When k=1, the search can proceed backward based on the target image block number, and the first second image block with the same area as the target image block can be taken as the first image block.

[0073] First, using image block number 1 as the target image block, the remaining image blocks in the list are traversed. If no image block matching the preset rule (i.e., an area of ​​1024 square pixels) is found, the target image block is encoded separately. Then, using image block number 2 as the new target image block, the unencoded image blocks in the list are traversed. Image block number 4 is found and selected as the first image block, as its area is equal to that of the target image block and it is the closest to the target image block.

[0074] Table 1

[0075] Image block number width high area Predictive patterns 1 64 16 1024 Inter-frame prediction 2 16 32 512 Inter-frame prediction 3 16 8 128 Inter-frame prediction 4 16 32 512 Inter-frame prediction 5 16 16 256 Inter-frame prediction 6 8 16 128 Inter-frame prediction 7 32 16 512 Inter-frame prediction 8 8 8 64 Inter-frame prediction 9 16 16 256 Inter-frame prediction 10 16 8 128 Inter-frame prediction

[0076] For example, when k=3, the search can proceed backward based on the target image block number, and the first three second image blocks with the same area as the target image block can be used as the three first image blocks.

[0077] Through the above steps, the method of this embodiment can effectively determine at least one first image block using simple rules. Furthermore, these preset rules ensure that the first residual matrix of the target image block and the first image block can be easily combined into a second residual matrix, further improving coding performance.

[0078] The following describes another implementation method in this disclosure for traversing unencoded image blocks in video frames to determine at least one first image block that conforms to preset rules. For example... Figure 3As shown in the embodiments of this disclosure, the process of traversing unencoded image blocks in video frames and determining at least one first image block that conforms to preset rules includes the following steps.

[0079] S301, traverse the uncoded image blocks in the video frames and determine the image block with the prediction mode of inter-frame prediction as the third image block.

[0080] S302, traverse the third image blocks in the video frame, and determine at least one third image block whose area sums with the target image block is equal to the preset area and is adjacent to the target image block as the first image block.

[0081] In some embodiments, at least one second image block whose area sums with the target image block equals a preset area, and which is the closest to the target image block, can be used as the first image block. The specific value of the preset area can be set according to actual application requirements, but to ensure the correct execution of the subsequent frequency domain transformation process, the preset area needs to be set to 2. m ×2 n , where m and n are positive integers.

[0082] For example, continuing to refer to Table 1, during the encoding process, the preset area is first determined to be 1024 square pixels.

[0083] First, image block number 1 is selected as the target image block. Its area equals a preset area, and it is encoded separately. Next, image block number 2 is selected as the new target image block. The list of unencoded image blocks is traversed, and image block number 4 is found and selected as the first image block. Its area, combined with the area of ​​the target image block, is 1024 square pixels, and it is the closest to the target image block. Then, image block number 3 is selected as the new target image block. The list of unencoded image blocks is traversed again. If image block number 4 is already encoded, image blocks numbered 3, 5, 6, and 7 are selected as the four first image blocks. If image block number 4 is not encoded, image blocks numbered 3, 4, 5, and 6 are selected as the four first image blocks.

[0084] Through the above steps, the method of this embodiment can effectively determine at least one first image block, and the preset rule is more flexible. It can ensure that the first residual matrix of the target image block and the first image block can be easily combined into a second residual matrix, further improving coding performance.

[0085] In some embodiments, image blocks with inter-frame prediction mode can be selected first as target image blocks. As mentioned above, since image blocks with intra-frame prediction mode have mutual dependencies, the image blocks with inter-frame prediction mode are first selected as target image blocks, and the method of this embodiment is applied to them. After all image blocks with inter-frame prediction mode are encoded, the remaining image blocks are then encoded. In this way, it can be ensured that the reference image of each image block has been encoded, so that the method of this embodiment can be executed smoothly.

[0086] In practice, it was found that within a video frame, the number of image blocks in the intra-frame prediction mode is far less than the number of image blocks in the inter-frame prediction mode, resulting in a lower probability of finding a suitable first image block. Therefore, in some embodiments, image blocks with the intra-frame prediction mode can be encoded separately to save computing power. Although this sacrifices some video compression efficiency, it improves overall coding performance.

[0087] Based on the same inventive concept, this disclosure also provides a video decoding method, such as... Figure 4 As shown, the video decoding method provided in this embodiment includes the following steps.

[0088] S401, acquire the encoded data corresponding to the target image block and at least one first image block.

[0089] In some embodiments, the decoder can acquire the encoded data and restore the encoded data to a second residual matrix through inverse quantization and inverse frequency domain transformation.

[0090] S402, Obtain the preset rule corresponding to the target image block.

[0091] In some embodiments, a preset rule corresponding to the target image block can be obtained based on information in the encoded data or information preset in the decoder.

[0092] S403, based on preset rules, split the second residual matrix to obtain multiple first residual matrices.

[0093] During decoding, the decoder uses information from the encoded data to determine whether the first residual matrix of the target image block has been fused with the first residual matrices of other image blocks. If so, the second residual matrix can be split into multiple first residual matrices based on preset rules. Each of these multiple first residual matrices corresponds to the target image block and at least one first image block that conforms to the preset rules.

[0094] S404, based on multiple first residual matrices, decodes the encoded data into target image blocks and at least one first image block.

[0095] The first residual matrix is ​​the original residual data of the corresponding image patch. Therefore, the target image patch and the first image patch can be obtained based on the first residual matrix and the predicted image patch, respectively. The predicted image patch is obtained based on the reference image and certain algorithms.

[0096] Through the above steps, the method of this embodiment can restore the encoded data of the target image block to the target image block, completing the data-to-image conversion. Multiple images are combined using an algorithm to ultimately complete the data-to-video conversion.

[0097] Based on the same inventive concept, this disclosure also provides a video encoding device, as shown in the following embodiments. Since the principle by which this device embodiment solves the problem is similar to that of the above-described method embodiments, the implementation of this device embodiment can refer to the implementation of the above-described method embodiments, and repeated details will not be elaborated further.

[0098] Figure 5 This diagram illustrates the structure of a video encoding apparatus according to an embodiment of the present disclosure, such as... Figure 5 As shown, the video encoding device 500 includes: a first acquisition module 501, a determination module 502, a second acquisition module 503, a fusion module 504, and an encoding module 505.

[0099] Specifically, the first acquisition module 501 is used to acquire a target image block in a video frame. The determination module 502 is used to traverse the unencoded image blocks in the video frame and determine at least one first image block that conforms to a preset rule. The second acquisition module 503 is used to acquire the first residual matrix of the target image block and the at least one first image block respectively. The fusion module 504 is used to fuse the first residual matrices of the target image block and the at least one first image block to obtain a second residual matrix. The encoding module 505 is used to obtain the encoded data corresponding to the target image block and the at least one first image block based on the second residual matrix, provided that the encoding cost of the second residual matrix is ​​less than the sum of the encoding costs of the first residual matrices of the target image block and the at least one first image block.

[0100] In some embodiments, the determining module 502 is further configured to: traverse unencoded image blocks in a video frame, determine an image block whose prediction mode is inter-frame prediction as a second image block; traverse the second image blocks in a video frame, and determine k second image blocks with the same area as the target image block and adjacent to the target image block as first image blocks, where k is a positive integer.

[0101] In some embodiments, the determining module 502 is further configured to: traverse unencoded image blocks in a video frame and determine an image block whose prediction mode is inter-frame prediction as a third image block; traverse the third image blocks in a video frame and determine at least one third image block whose area sums with that of the target image block is equal to a preset area and which is adjacent to the target image block as a first image block.

[0102] In some embodiments, the encoding cost is the rate-distortion cost, which is calculated based on the bitstream length of the encoded data of the target image block and the degree of video distortion.

[0103] In some embodiments, the encoding module 505 is further configured to, when the encoding cost of the second residual matrix is ​​greater than or equal to the encoding cost of the first residual matrix of the target image block, obtain the encoded data corresponding to the target image block based on the first residual matrix of the target image block.

[0104] In some embodiments, the prediction mode for the target image patch is inter-frame prediction.

[0105] It should be noted that the video encoding device provided in the above embodiments is only illustrated by the division of the above functional modules when used for video encoding. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the video encoding device and the video encoding method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0106] Based on the same inventive concept, this disclosure also provides a video decoding device, as shown in the following embodiments. Since the principle by which this device embodiment solves the problem is similar to that of the above-described method embodiments, the implementation of this device embodiment can refer to the implementation of the above-described method embodiments, and repeated details will not be described again.

[0107] Figure 6 This diagram illustrates the structure of a video decoding device according to an embodiment of the present disclosure, as shown below. Figure 6 As shown, the video decoding device 600 includes: a first acquisition module 601, a second acquisition module 602, a splitting module 603, and a decoding module 604.

[0108] Specifically, the first acquisition module 601 is used to acquire encoded data corresponding to the target image block and at least one first image block. The encoded data includes a second residual matrix corresponding to the target image block and at least one first image block. The encoded data is obtained based on the method in the embodiments of this disclosure. The second acquisition module 602 is used to acquire a preset rule corresponding to the target image block. The splitting module 603 is used to split the second residual matrix based on the preset rule to obtain multiple first residual matrices. The decoding module 604 is used to decode the encoded data into the target image block and at least one first image block based on the multiple first residual matrices.

[0109] It should be noted that the video decoding device provided in the above embodiments is only used as an example to illustrate the division of the above functional modules when used for video decoding. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the video decoding device and the video decoding method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0110] Those skilled in the art will understand that various aspects of this disclosure can be implemented as a system, method, or program product. Therefore, various aspects of this disclosure can be specifically implemented in the following forms: a completely hardware implementation, a completely software implementation (including firmware, microcode, etc.), or a combination of hardware and software aspects, collectively referred to herein as a "circuit," "module," or "system."

[0111] The following reference Figure 7 To describe an electronic device 700 according to such an embodiment of the present disclosure. Figure 7 The electronic device 700 shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0112] like Figure 7 As shown, the electronic device 700 is manifested in the form of a general-purpose computing device. The components of the electronic device 700 may include, but are not limited to: at least one processing unit 710, at least one storage unit 720, and a bus 730 connecting different system components (including storage unit 720 and processing unit 710).

[0113] The storage unit stores program code, which can be executed by the processing unit 710, causing the processing unit 710 to perform the steps described in the "Exemplary Methods" section above according to various exemplary embodiments of this disclosure.

[0114] In some embodiments, the processing unit 710 may perform the following steps of the above method embodiments: acquiring a target image block in a video frame; traversing unencoded image blocks in the video frame to determine at least one first image block that conforms to a preset rule; acquiring the first residual matrix of the target image block and the at least one first image block respectively; fusing the first residual matrices of the target image block and the at least one first image block to obtain a second residual matrix; and, if the encoding cost of the second residual matrix is ​​less than the sum of the encoding costs of the first residual matrices of the target image block and the at least one first image block, obtaining the encoded data corresponding to the target image block and the at least one first image block based on the second residual matrix.

[0115] In some embodiments, the processing unit 710 may also perform the following steps of the above method embodiments: obtaining encoded data corresponding to a target image block and at least one first image block, wherein the encoded data includes a second residual matrix corresponding to the target image block and at least one first image block, and the encoded data is obtained based on the method in the embodiments of this disclosure; obtaining a preset rule corresponding to the target image block; splitting the second residual matrix based on the preset rule to obtain multiple first residual matrices; and decoding the encoded data into a target image block and at least one first image block based on the multiple first residual matrices.

[0116] Storage unit 720 may include a readable medium in the form of a volatile storage unit, such as random access memory (RAM) 7201 and / or cache memory 7202, and may further include a read-only memory (ROM) 7203.

[0117] The storage unit 720 may also include a program / utility 7204 having a set (at least one) program module 7205, such program module 7205 including but not limited to: an operating system, one or more application programs, other program modules and program data, each or some combination of these examples may include an implementation of a network environment.

[0118] Bus 730 can represent one or more of several types of bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the various bus structures.

[0119] Electronic device 700 can also communicate with one or more external devices 740 (e.g., keyboard, pointing device, Bluetooth device, etc.), and with one or more devices that enable a user to interact with electronic device 700, and / or with any device that enables electronic device 700 to communicate with one or more other computing devices (e.g., router, modem, etc.). This communication can be performed via input / output (I / O) interface 750. Furthermore, electronic device 700 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 760. As shown, network adapter 760 communicates with other modules of electronic device 700 via bus 730. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with electronic device 700, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0120] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, terminal device, or network device, etc.) to execute the methods according to the embodiments of this disclosure.

[0121] In exemplary embodiments of this disclosure, a computer-readable storage medium is also provided, which may be a readable signal medium or a readable storage medium. A program product capable of implementing the methods described above is stored thereon. In some possible implementations, various aspects of this disclosure may also be implemented as a program product including program code, which, when run on a terminal device, causes the terminal device to perform the steps of the various exemplary embodiments of this disclosure described in the method embodiments section of this specification.

[0122] More specific examples of computer-readable storage media in this disclosure may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0123] In this disclosure, a computer-readable storage medium may include a data signal propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium, capable of transmitting, propagating, or transmitting a program for use by or in connection with an instruction execution system, apparatus, or device.

[0124] Optionally, the program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0125] In practical implementation, program code for performing the operations of this disclosure can be written in any combination of one or more programming languages, including object-oriented programming languages. The program code can execute entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0126] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to embodiments of this disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0127] Furthermore, although the steps of the method in this disclosure are described in a specific order in the accompanying drawings, this does not require or imply that the steps must be performed in that specific order, or that all the steps shown must be performed to achieve the desired result. Additional or alternative steps may be omitted, multiple steps may be combined into one step, and / or a step may be broken down into multiple steps.

[0128] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, mobile terminal, or network device, etc.) to execute the methods according to the embodiments of this disclosure.

[0129] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the appended claims.

Claims

1. A video encoding method, characterized in that, include: Obtain the target image patch from the video frame; Traverse the unencoded image blocks in the video frame and determine at least one first image block that conforms to a preset rule; Obtain the first residual matrix of each of the target image block and the at least one first image block; The first residual matrices of the target image block and the at least one first image block are fused to obtain a second residual matrix; If the encoding cost of the second residual matrix is ​​less than the sum of the encoding costs of the first residual matrices of the target image block and the at least one first image block, the encoded data corresponding to the target image block and the at least one first image block are obtained based on the second residual matrix.

2. The method according to claim 1, characterized in that, The step of traversing the unencoded image blocks in the video frame and determining at least one first image block that conforms to a preset rule includes: Traverse the unencoded image blocks in the video frame and determine the image block with the prediction mode of inter-frame prediction as the second image block; Traverse the second image blocks in the video frame and determine the k second image blocks that have the same area as the target image block and are adjacent to the target image block as the first image block, where k is a positive integer.

3. The method according to claim 1, characterized in that, The step of traversing the unencoded image blocks in the video frame and determining at least one first image block that conforms to a preset rule includes: Traverse the unencoded image blocks in the video frame and determine the image block with the prediction mode of inter-frame prediction as the third image block; Traverse the third image blocks in the video frame and determine at least one third image block whose area sums with the target image block to a preset area and which is adjacent to the target image block as the first image block.

4. The method according to claim 1, characterized in that, The encoding cost is the rate-distortion cost, which is calculated based on the bitstream length of the encoded data of the target image block and the degree of video distortion.

5. The method according to claim 1, characterized in that, Also includes: If the encoding cost of the second residual matrix is ​​greater than or equal to the encoding cost of the first residual matrix of the target image block, the encoded data corresponding to the target image block is obtained based on the first residual matrix of the target image block.

6. The method according to claim 1, characterized in that, The prediction mode for the target image patch is inter-frame prediction.

7. A video decoding method, characterized in that, include: Obtain encoded data corresponding to a target image block and at least one first image block, wherein the encoded data includes a second residual matrix corresponding to the target image block and the at least one first image block, and the encoded data is obtained based on the method according to any one of claims 1 to 6; Obtain the preset rules corresponding to the target image block; Based on the preset rules, the second residual matrix is ​​split to obtain multiple first residual matrices; Based on the plurality of first residual matrices, the encoded data is decoded into the target image block and the at least one first image block.

8. A video encoding device, characterized in that, include: The first acquisition module is used to acquire target image blocks in video frames; The determination module is used to traverse the unencoded image blocks in the video frame and determine at least one first image block that conforms to a preset rule; The second acquisition module is used to acquire the first residual matrix of each of the target image block and the at least one first image block; The fusion module is used to fuse the first residual matrices of the target image block and the at least one first image block respectively to obtain a second residual matrix; The encoding module is configured to obtain encoded data corresponding to the target image block and the at least one first image block based on the second residual matrix, provided that the encoding cost of the second residual matrix is ​​less than the sum of the encoding costs of the first residual matrices of the target image block and the at least one first image block.

9. A video decoding device, characterized in that, include: The first acquisition module is configured to acquire encoded data corresponding to a target image block and at least one first image block, wherein the encoded data includes a second residual matrix corresponding to the target image block and the at least one first image block, and the encoded data is obtained based on the method described in any one of claims 1 to 6; The second acquisition module is used to acquire the preset rules corresponding to the target image block; The splitting module is used to split the second residual matrix based on the preset rules to obtain multiple first residual matrices; A decoding module is used to decode the encoded data into the target image block and the at least one first image block based on the plurality of first residual matrices.

10. An electronic device, characterized in that, include: processor; as well as Memory for storing the executable instructions of the processor; The processor is configured to execute the method of any one of claims 1 to 7 by executing the executable instructions.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method described in any one of claims 1 to 7.

Images