Utilizing distortion cost as loss function for deep learning

Abstract

The invention discloses utilization of distortion cost as a loss function for deep learning. An apparatus for encoding an image block includes a processor that presents the image block to a machine learning model, obtains a segmentation decision from the model for encoding the image block, and encodes the image block using the segmentation decision. The model is trained to output segmentation decisions for encoding an image block using training data of a plurality of training blocks as input, the training data including, for the training blocks, the segmentation decisions for encoding the training blocks and, for each segmentation decision, a rate distortion value resulting from encoding the training blocks using the segmentation decisions. The model is trained using a loss function that combines a segmentation loss function based on a relationship between a segmentation decision and a respective predicted segmentation and a rate distortion cost loss function based on a relationship between a rate distortion value and a respective predicted rate distortion value.

Description

Description of the case

[0001] This application is a divisional application of Chinese invention patent application No. 201980095978.1, filed on March 21, 2019. Technical Field

[0002] The present disclosure is directed to using rate-distortion cost as a loss function for deep learning. Background Art

[0003] A digital video stream may represent a video using a series of frames or still images. Digital video may be used for a variety of applications, including, for example, video conferencing, high-definition video entertainment, video advertising, or sharing of user-generated videos. A digital video stream may contain a large amount of data and consume a large amount of computing or communication resources of a computing device to process, transmit, or store the video data. Various methods have been proposed to reduce the amount of data in a video stream, including compression and other encoding techniques. Summary of the invention

[0004] One aspect of the disclosed embodiment is a method for encoding an image block using a first encoder. The method includes presenting the image block to a machine learning model of the first encoder, wherein the machine learning model is trained to output a segmentation decision for encoding the image block by using training data of a plurality of training blocks as input to train a machine learning module including the machine learning model, for a training block in the plurality of training blocks, the training data includes a plurality of segmentation decisions used by a second encoder to encode the training block, and for each segmentation decision in the plurality of segmentation decisions, the training data includes a rate-distortion value resulting from encoding the training block using the segmentation decision. The machine learning model is trained using a loss function that combines: a segmentation loss function based on a relationship between the segmentation decision and a corresponding predicted segmentation and a rate-distortion cost loss function based on a relationship between the rate-distortion value and the corresponding predicted rate-distortion value. The method also includes obtaining a segmentation decision for encoding the image block from the machine learning model, and encoding the image block using the segmentation decision in a compressed bitstream.

[0005] Another method described herein includes presenting an image block to a machine learning model of a first encoder, wherein the machine learning model is trained to output a segmentation decision for encoding the image block using training data for a plurality of training blocks and a loss function, for a training block in the plurality of training blocks, the training data includes a plurality of segmentation decisions used by a second encoder to encode the training data, and for each segmentation decision in the plurality of segmentation decisions, the training data includes a rate-distortion value resulting from encoding the training block using the segmentation decision, the loss function including a combination of a segmentation loss function based on a relationship between the segmentation decision and a corresponding predicted segmentation and a rate-distortion cost loss function based on a relationship between the rate-distortion value and the corresponding predicted rate-distortion value. The method also includes obtaining a segmentation decision for encoding the image block from the machine learning model, and encoding the image block using the segmentation decision in a compressed bitstream.

[0006] An apparatus according to the techniques herein includes a processor configured to receive training data, the training data including a plurality of training blocks, and for a training block in the plurality of training blocks, including a plurality of segmentation decisions used by an encoder to encode the training block, and for each segmentation decision in the plurality of segmentation decisions, including a rate-distortion value resulting from encoding the training block using the segmentation decision. The processor is configured to train a machine learning model to output the segmentation decision for encoding the image block by inputting the training data into a neural network using a loss function, the loss function including a combination of: a segmentation loss function based on a relationship between the segmentation decision and a corresponding predicted segmentation and a rate-distortion cost loss function based on a relationship between the rate-distortion value and the corresponding predicted rate-distortion value.

[0007] On the other hand is an apparatus for encoding an image block. The apparatus includes a processor that presents the image block to a machine learning model, obtains a segmentation decision for encoding the image block from the machine learning model, and encodes the image block using the segmentation decision in a compressed bitstream. The machine learning model is trained to output segmentation decisions for encoding the image block by using training data for a plurality of training blocks as input to train the machine learning model, for a training block in the plurality of training blocks, the training data includes a plurality of segmentation decisions used by an encoder to encode the training block, and for each segmentation decision in the plurality of segmentation decisions, the training data includes a rate-distortion value resulting from encoding the training block using the segmentation decision, wherein the machine learning model is trained using a loss function that combines: a segmentation loss function based on a relationship between the segmentation decisions and the corresponding predicted segmentation and a rate-distortion cost loss function based on a relationship between the rate-distortion value and the corresponding predicted rate-distortion value.

[0008] Another device described herein includes a processor that presents an image block to a machine learning model. The machine learning model is trained to output a segmentation decision for encoding the image block using training data for a plurality of training blocks, for a training block in the plurality of training blocks, the training data includes a plurality of segmentation decisions used by an encoder to encode the training block, and for each segmentation decision in the plurality of segmentation decisions, the training data includes a rate-distortion value generated from encoding the training block using the segmentation decision. The machine learning model is also trained using a combination of a segmentation loss function based on a relationship between the segmentation decision and a corresponding predicted segmentation and a rate-distortion cost loss function based on a relationship between the rate-distortion value and the corresponding predicted rate-distortion value. The processor also obtains a segmentation decision for encoding the image block from the machine learning model, and encodes the image block using the segmentation decision in a compressed bitstream.

[0009] On the other hand, an apparatus for encoding an image block includes a machine learning model, an encoder, and a processor. The encoder generates training data by encoding each of a plurality of training blocks multiple times, each time encoding the corresponding training block using a corresponding segmentation decision in a plurality of segmentation decisions. The processor trains the machine learning model to output a segmentation decision for encoding the image block using the training data as input to train the machine training model, for a training block in the plurality of training blocks, the training data includes a plurality of segmentation decisions used by the encoder to encode the training block, and for each segmentation decision in the plurality of segmentation decisions, the training data includes a rate-distortion value resulting from encoding the training block using the segmentation decision. The machine learning model is trained using a loss function that combines a segmentation loss function based on a relationship between the segmentation decisions and the corresponding predicted segmentation and a rate-distortion cost loss function based on a relationship between the rate-distortion value and the corresponding predicted rate-distortion value.

[0010] Each aspect may optionally include one or more of the following features. The loss function may include a non-uniform weighting of a segmentation loss function and a rate-distortion loss function. The segmentation loss function may include a binary cross-entropy loss function. The binary cross-entropy loss function may apply a corresponding weight to a segmentation decision, the weight applied to a segmentation decision in a plurality of segmentation decisions being equal to the rate-distortion value of the segmentation decision normalized by the lowest rate-distortion value resulting from the plurality of segmentation decisions. The rate-distortion cost loss function may exponentially penalize a predicted segmentation that does not match a selected segmentation decision of a training block resulting from an exhaustive segmentation search of a second encoder. The machine learning module may include a rate-distortion cost estimator that receives a predicted segmentation from a machine learning model and estimates a predicted rate-distortion value of a loss function, and the machine learning model is a convolutional neural network model. For each segmentation decision in a plurality of segmentation decisions, the rate-distortion value resulting from encoding a training block using the segmentation decision may include a rate-distortion cost for encoding the training block without further segmenting the training block and a cost resulting from splitting the training block. The plurality of segmentation decisions may include a quadtree segmentation decision. The plurality of segmentation decisions may include a segmentation decision that results in at least one non-square segmentation of the training block. The second encoder may include a block-based encoder. For each of the plurality of segmentation decisions, the rate-distortion cost loss function may be an exponential function including the difference between a rate-distortion value resulting from encoding the training block using the segmentation decision and a predicted rate-distortion value. For each of the plurality of segmentation decisions, the corresponding predicted rate-distortion value may be normalized using the lowest rate-distortion value of the plurality of segmentation decisions in the rate-distortion cost loss function. The rate-distortion cost loss function may include: determining the predicted rate-distortion value using the segmentation decision in the plurality of segmentation decisions and the rate-distortion value resulting from encoding the training block using the segmentation decision in the plurality of segmentation decisions, wherein the segmentation decision includes a combination of a split decision and a non-split decision of the training block, and the rate-distortion value includes a combination of rate-distortion values ​​resulting from encoding the training block using the combination of the split decision and the non-split decision. The predicted rate-distortion value may include a first term representing the rate-distortion cost of the non-split decision of the training block and a second term representing the rate-distortion cost of the split decision of the training block that divides the training block into a plurality of sub-blocks. The rate-distortion cost of a splitting decision of a training block to divide the training block into a plurality of sub-blocks may include a corresponding rate-distortion cost of a splitting decision for each of the plurality of sub-blocks; the rate-distortion cost of the first item may be weighted by a lowest rate-distortion value of the plurality of splitting decisions used to encode the training block; and the rate-distortion cost of the second item may include weighting each of the corresponding rate-distortion costs for splitting each of the plurality of sub-blocks by a lowest rate-distortion value of the plurality of splitting decisions used to encode the training block.The machine learning model can be trained using a machine learning module that includes a machine learning model and a rate-distortion cost estimator that receives a predicted segmentation from the machine learning model and estimates a predicted rate-distortion value of a loss function. The machine learning model can be trained using a loss function by applying a weight that is larger than the rate-distortion cost loss function to the segmentation loss function. The machine learning module can include a rate-distortion cost estimator that receives a predicted segmentation from the machine learning model and estimates a predicted rate-distortion value of a loss function.

[0011] These and other aspects of the present disclosure are disclosed in the following detailed description of the embodiments, the appended claims and the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The description herein refers to the drawings, wherein like reference numerals refer to similar parts throughout the views.

[0013] Figure 1 It is a schematic diagram of a video encoding and decoding system.

[0014] Figure 2 is a block diagram of an example of a computing device that may implement a sending station or a receiving station.

[0015] Figure 3 is a schematic diagram of a video stream to be encoded and subsequently decoded.

[0016] Figure 4 is a block diagram of an encoder according to an embodiment of the present disclosure.

[0017] Figure 5 is a block diagram of a decoder according to an embodiment of the present disclosure.

[0018] Figure 6 is a block diagram of a representation of a portion of a frame according to an embodiment of the present disclosure.

[0019] Figure 7 is a block diagram of an example of a quadtree representation of blocks according to an embodiment of the present disclosure.

[0020] Figure 8 is a flow chart of a process for searching for an optimal mode for coding a block.

[0021] Fig. 9 is a simplified representation of a convolutional neural network that can be used to implement the teachings herein.

[0022] Fig. 10A is an example of a segmentation decision for a b×b block with relatively uniform content.

[0023] Fig. 10B is Fig. 10AAn example of a segmentation decision for a b×b block in more detail.

[0024] Fig.11 is a schematic diagram illustrating a block partitioning pattern and recursive partitioning.

[0025] Fig.12 is a schematic diagram illustrating the training of a machine learning / deep learning model.

[0026] Fig.13 Is to use from according to Fig.12 Flowchart encoding the process of obtaining inferences from a trained machine learning / deep learning model. DETAILED DESCRIPTION

[0027] Coding techniques can be designed to maximize coding efficiency. Coding efficiency can refer to encoding a video at the lowest possible bit rate while minimizing distortion (e.g., while maintaining a certain level of video quality). Coding efficiency is usually measured in terms of both rate and distortion. Rate refers to the number of bits required for encoding (such as encoding a block, frame, etc.). Distortion measures the quality loss between, for example, a source video block and a reconstructed version of the source video block. For example, distortion can be calculated as the mean square error between the pixel values ​​of the source block and the pixel values ​​of the reconstructed block. By performing a rate-distortion optimization process, a video codec optimizes the amount of distortion for the rate required to encode the video.

[0028] Modern video codecs (e.g., H.264, also known as MPEG-4 AVC; VP9, ​​H.265, also known as HEVC; AVS2 and AV1) define and use a large number of tools and configurations (e.g., parameters) to improve coding efficiency. A video codec can use mode decisions to examine (e.g., test, evaluate, etc.) at least some valid parameter combinations to select a combination that results in a relatively low rate-distortion value. An example of a mode decision is an intra-prediction mode decision that determines the best prediction intra-mode for coding a block. Another example of a mode decision is a partitioning decision that determines the best partitioning of a coding unit (also referred to as a coding unit or CU). Another example of a mode decision includes a decision on a transform type to be used when converting a block (such as a residual block or an image block) from a pixel domain to a frequency domain to form a transform block including transform coefficients.

[0029] To evaluate whether one combination is better than another, a metric may be calculated for each examined combination and the corresponding metrics compared. In an example, the metric may combine the above-mentioned rate and distortion to produce a rate-distortion (RD) value or cost. The RD value or cost may be a single scalar value.

[0030] As described above, the best mode can be selected from many possible combinations. For example, the RD cost associated with a particular mode (or a particular combination of tools) can be determined by performing at least a subset of the encoding steps of the encoder. Depending on the mode for which the RD cost is to be determined, the subset of encoding steps may include at least one of the following: determining a prediction block, determining a residual block, determining a transform type, determining an interpolation filter, quantizing a transform block, entropy encoding, etc. Note that these encoding steps are neither intended to be an exhaustive list of encoding steps that a typical encoder may perform, nor are they presented in any particular order (i.e., the encoder does not necessarily perform these steps in the order listed). As the number of possible tools and parameters increases, the number of combinations also increases, which in turn increases the time required to determine the best mode.

[0031] In addition to the exhaustive search, the encoder can terminate the pattern search when it finds a pattern with an RD cost less than a set threshold. However, this means that if the encoder continues with the pattern search, it may find a better pattern later. In some cases, an exhaustive search may or may not be performed, but the entire RD cost calculation is replaced by a rough estimate. This can further reduce the decisions made by the encoder.

[0032] Techniques such as machine learning can be utilized to reduce the time required to determine the best mode, such as the segmentation mode. For example, instead of performing all encoding steps (i.e., brute force or exhaustive methods) to determine the rate and distortion of various segmentation modes to compare these modes and select the best mode, a machine learning model can be used to estimate the best mode.

[0033] Machine learning models can be implemented using standard coding techniques such as Figure 4 and Figures 6 to 8 The described techniques are trained using a large amount of training data obtained by the encoder. More specifically, during the learning phase of machine learning, the training data can be used to derive (e.g., learn, infer, etc.) a machine learning model, which is (e.g., define, construct, etc.) a mapping from input data (e.g., block data) to output.

[0034] Once the machine learning model is trained, the model computes output as a deterministic function of its input. In an example, the machine learning model may be a neural network model, which may be a convolutional neural network (CNN). Further details of neural network models (including CNN) are discussed below.

[0035] In general, training of machine learning models occurs using a loss function. The loss function can be characterized as measuring how well the predicted values ​​(also referred to as predictions) of the model match the expected results. Common loss functions include mean squared error (MSE), binary cross entropy, or cross entropy. Using the rate and distortion loss functions described in this article can result in better training models than using such loss functions alone, thereby improving coding efficiency. The details of this machine learning model are first described in this article with reference to a block-based codec that can be incorporated into the teachings. Although block-based codecs are described as examples, other codecs can also be used with the present teachings, including feature-based codecs.

[0036] Figure 1 1 is a schematic diagram of a video encoding and decoding system 100. The transmitting station 102 may be, for example, a computer having an internal configuration of hardware, such as Figure 2 However, other suitable implementations of the sending station 102 are possible. For example, the processing of the sending station 102 can be distributed among multiple devices.

[0037] The network 104 can connect the sending station 102 and the receiving station 106 to encode and decode the video stream. Specifically, the video stream can be encoded in the sending station 102, and the encoded video stream can be decoded in the receiving station 106. The network 104 can be, for example, the Internet. The network 104 can also be a local area network (LAN), a wide area network (WAN), a virtual private network (VPN), a cellular phone network, or any other means of transmitting the video stream from the sending station 102 to the receiving station 106 in this example.

[0038] In one example, the receiving station 106 may be a computer having an internal configuration of hardware, such as Figure 2 However, other suitable implementations of receiving station 106 are possible. For example, the processing of receiving station 106 may be distributed among multiple devices.

[0039] Other embodiments of the video encoding and decoding system 100 are possible. For example, an embodiment may omit the network 104. In another embodiment, the video stream may be encoded and then stored for transmission at a later time to the receiving station 106 or any other device with memory. In one embodiment, the receiving station 106 receives (e.g., via the network 104, a computer bus, and / or some communication path) the encoded video stream and stores the video stream for later decoding. In an example embodiment, the real-time transport protocol (RTP) is used to transmit the encoded video over the network 104. In another embodiment, a transport protocol other than RTP may be used, such as a video streaming protocol based on the hypertext transfer protocol (HTTP).

[0040] When used in a video conferencing system, for example, the sending station 102 and / or the receiving station 106 may include the capabilities for encoding and decoding video streams as described below. For example, the receiving station 106 may be a video conference participant that receives an encoded video bitstream from a video conference server (e.g., the sending station 102) for decoding and viewing, and further encodes its own video bitstream and transmits it to the video conference server for decoding and viewing by other participants.

[0041] Figure 2 is a block diagram of an example of a computing device 200 that may implement a sending station or a receiving station. For example, the computing device 200 may implement Figure 1 The computing device 200 may be in the form of a computing system including multiple computing devices or in the form of a single computing device, such as a mobile phone, a tablet computer, a laptop computer, a notebook computer, a desktop computer, etc.

[0042] The CPU 202 in the computing device 200 may be a central processing unit. Alternatively, the CPU 202 may be any other type of device or devices, now or later developed, that are capable of manipulating or processing information. Although the disclosed embodiments may be practiced with a single processor (e.g., CPU 202) as shown, advantages in speed and efficiency may be achieved by using more than one processor.

[0043] In an embodiment, the memory 204 in the computing device 200 may be a read-only memory (ROM) device or a random access memory (RAM) device. Any other suitable type of storage device may be used as the memory 204. The memory 204 may include code and data 206 accessed by the processor 202 using the bus 212. The memory 204 may further include an operating system 208 and an application 210, which includes at least one program that allows the CPU 202 to perform the methods described herein. For example, the application 210 may include applications 1 to N, which further include a video coding application that performs the methods described herein. The computing device 200 may also include a secondary storage 214, which may be, for example, a memory card used with the mobile computing device 200. Because a video communication session may contain a large amount of information, they may be stored in full or in part in the secondary storage 214 and loaded into the memory 204 as needed for processing.

[0044] The computing device 200 may also include one or more output devices, such as a display 218. In one example, the display 218 may be a touch-sensitive display that combines a display with a touch-sensitive element operable to sense touch input. The display 218 may be coupled to the CPU 202 via the bus 212. In addition to or as an alternative to the display 218, other output devices that allow a user to program or otherwise use the computing device 200 may be provided. When the output device is or includes a display, the display may be implemented in various ways, including as a liquid crystal display (LCD), a cathode ray tube (CRT) display, or a light emitting diode (LED) display, such as an organic LED (OLED) display.

[0045] The computing device 200 may also include or communicate with an image sensing device 220, such as a camera or any other image sensing device now known or later developed that can sense an image, such as an image of a user operating the computing device 200. The image sensing device 220 may be positioned so that it is pointed toward the user operating the computing device 200. In an example, the position and optical axis of the image sensing device 220 may be configured so that the field of view includes an area directly adjacent to the display 218 and from which the display 218 is visible.

[0046] The computing device 200 may also include, or communicate with, a sound sensing device 222, such as a microphone or any other sound sensing device now known or later developed that can sense sounds near the computing device 200. The sound sensing device 222 may be positioned so that it is directed toward a user operating the computing device 200, and may be configured to receive sounds, such as speech or other utterances uttered by the user while operating the computing device 200.

[0047] although Figure 2 The CPU 202 and memory 204 of the computing device 200 are described as being integrated into a single unit, but other configurations may also be utilized. The operation of the CPU 202 may be distributed across multiple machines (each having one or more processors), which may be coupled directly or across a local area network or other network. The memory 204 may be distributed across multiple machines such as network-based memory or memory in multiple machines that perform the operation of the computing device 200. Although depicted here as a single bus, the bus 212 of the computing device 200 may be composed of multiple buses. Further, the secondary storage 214 may be directly coupled to other components of the computing device 200 or may be accessed via a network, and may include a single integrated unit, such as a memory card, or multiple units, such as multiple memory cards. Therefore, the computing device 200 may be implemented in various configurations.

[0048] Figure 3300 is a schematic diagram of an example of a video stream 300 to be encoded and subsequently decoded. The video stream 300 includes a video sequence 302. At the next level, the video sequence 302 includes a plurality of adjacent frames 304. Although three frames are depicted as adjacent frames 304, the video sequence 302 may include any number of adjacent frames 304. The adjacent frames 304 may then be further subdivided into individual frames, such as frame 306. At the next level, the frame 306 may be divided into a series of segments 308 or planes. For example, the segment 308 may be a subset of a frame that allows for parallel processing. The segment 308 may also be a subset of a frame that may separate video data into individual colors. For example, a frame 306 of color video data may include a luminance plane and two chrominance planes. The segment 308 may be sampled at different resolutions.

[0049] Regardless of whether the frame 306 is divided into segments 308, the frame 306 can be further subdivided into blocks 310, which can contain data corresponding to, for example, 16×16 pixels in the frame 306. The blocks 310 can also be arranged to include data from one or more segments 308 of pixel data. The blocks 310 can also have any other suitable size, such as 4×4 pixels, 8×8 pixels, 16×8 pixels, 8×16 pixels, 16×16 pixels, or larger.

[0050] Figure 4 is a block diagram of an encoder 400 according to an embodiment of the present disclosure. The encoder 400 may be implemented in the transmission station 102 as described above, such as by providing a computer software program stored in a memory, such as the memory 204. The computer software program may include machine instructions that, when executed by a processor, such as the CPU 202, cause the transmission station 102 to encode video data in the manner described herein. The encoder 400 may also be implemented as dedicated hardware that is included, for example, in the transmission station 102. The encoder 400 has the following stages to perform various functions in a forward path (represented by solid connecting lines) using the video stream 300 as input to produce an encoded or compressed bitstream 420: an intra / inter prediction stage 402, a transform stage 404, a quantization stage 406, and an entropy encoding stage 408. The encoder 400 may also include a reconstruction path (represented by a dashed connecting line) for reconstructing frames to encode future blocks. In Figure 4 In FIG. 4 , the encoder 400 has the following stages to perform various functions in the reconstruction path: a dequantization stage 410 , an inverse transform stage 412 , a reconstruction stage 414 , and a loop filter stage 416 . Other structural variations of the encoder 400 may be used to encode the video stream 300 .

[0051] When the video stream 300 is presented for encoding, the frames 306 may be processed in units of blocks. In the intra / inter prediction stage 402, the blocks may be encoded using intra prediction (also referred to as intra prediction) or inter prediction (also referred to as inter prediction), or a combination of both. In either case, a prediction block may be formed. In the case of intra prediction, all or part of the prediction block may be formed from samples in the current frame that have been previously encoded and reconstructed. In the case of inter prediction, all or part of the prediction block may be formed from samples in one or more previously constructed reference frames determined using motion vectors.

[0052] Next, still refer to Figure 4 , the prediction block can be subtracted from the current block in the intra / inter prediction stage 402 to produce a residual block (also referred to as a residual). The transform stage 404 uses a block-based transform to convert the residual into a transform coefficient in, for example, the frequency domain. For example, such block-based transforms (i.e., transform types) include discrete cosine transforms (DCT) and asymmetric discrete sine transforms (ADST). Other block-based transforms are possible. Further, combinations of different transforms can be applied to a single residual. In one example of a transform application, the DCT transforms the residual block into the frequency domain, where the transform coefficient values ​​are based on spatial frequencies. The lowest frequency (DC) coefficient is at the top left of the matrix, and the highest frequency coefficient is at the bottom right of the matrix. It is worth noting that the size of the prediction block may be different from the size of the transform block, and therefore, the resulting residual block may be different from the size of the transform block. For example, the prediction block can be divided into smaller blocks, and separate transforms are applied to these smaller blocks.

[0053] The quantization stage 406 uses a quantizer value or quantization level to convert the transform coefficients into discrete quantum values, which are called quantized transform coefficients. For example, the transform coefficients can be divided by the quantizer value and truncated. The quantized transform coefficients are then entropy encoded by the entropy encoding stage 408. Entropy coding can be performed using any number of techniques, including tokens and binary trees. The entropy coded coefficients and other information for decoding the block (for example, the prediction type, transform type, motion vector, and quantizer value used may be included) are then output to the compressed bitstream 420. The information for decoding the block can be entropy coded into blocks, frames, slices, and / or section headers in the compressed bitstream 420. The compressed bitstream 420 may also be referred to as a coded video stream or a coded video bitstream; these terms will be used interchangeably in this article.

[0054] Figure 4The reconstruction path in (represented by a dashed connecting line) can be used to ensure that the encoder 400 and the decoder 500 (described below) use the same reference frames and blocks to decode the compressed bitstream 420. The reconstruction path performs functions similar to those that occur during the decoding process and are discussed in more detail below, including dequantizing the quantized transform coefficients at the dequantization stage 410 and inverse transforming the dequantized transform coefficients at the inverse transform stage 412 to produce a derivative residual block (also referred to as a derivative residual). In the reconstruction stage 414, the prediction block predicted in the intra / inter prediction stage 402 can be added to the derivative residual to create a reconstructed block. A loop filtering stage 416 can be applied to the reconstructed block to reduce distortion, such as blocking artifacts.

[0055] Other variations of the encoder 400 may be used to encode the compressed bitstream 420. For example, a non-transform based encoder 400 may directly quantize the residual signal without requiring a transform stage 404 for certain blocks or frames. In another embodiment, the encoder 400 may combine the quantization stage 406 and the dequantization stage 410 into a single stage.

[0056] Figure 5 2 is a block diagram of a decoder 500 according to an embodiment of the present disclosure. The decoder 500 may be implemented in the receiving station 106, for example, by providing a computer software program stored in the memory 204. The computer software program may include machine instructions that, when executed by a processor, such as the CPU 202, cause the receiving station 106 to decode the video data in the manner described below. The decoder 500 may also be implemented in hardware that is included in, for example, the sending station 102 or the receiving station 106.

[0057] Similar to the reconstruction path of the encoder 400 described above, in one example, the decoder 500 includes the following stages to perform various functions to produce an output video stream 516 from the compressed bitstream 420: an entropy decoding stage 502, a dequantization stage 504, an inverse transform stage 506, an intra / inter prediction stage 508, a reconstruction stage 510, a loop filtering stage 512, and a post-filtering stage 514. Other structural variations of the decoder 500 may be used to decode the compressed bitstream 420.

[0058] When the compressed bitstream 420 is presented for decoding, the data elements in the compressed bitstream 420 can be decoded by the entropy decoding stage 502 to produce a set of quantized transform coefficients. The dequantization stage 504 dequantizes the quantized transform coefficients (e.g., by multiplying the quantized transform coefficients by a quantizer value), and the inverse transform stage 506 inversely transforms the dequantized transform coefficients using the selected transform type to produce a derivative residual, which can be the same as the reciprocal residual created by the inverse transform stage 412 in the encoder 400. Using the header information decoded from the compressed bitstream 420, the decoder 500 can use the intra / inter prediction stage 508 to create a prediction block that is the same as the prediction block created in the encoder 400, e.g., in the intra / inter prediction stage 402. In the reconstruction stage 510, the prediction block can be added to the derivative residual to create a reconstructed block. The loop filtering stage 512 can be applied to the reconstructed block to reduce blocking artifacts. Other filtering can be applied to the reconstructed block. In an example, a post-filtering stage 514 is applied to the reconstructed blocks to reduce block distortion, and the result is output as an output video stream 516. The output video stream 516 may also be referred to as a decoded video stream; these terms will be used interchangeably herein.

[0059] Other variations of the decoder 500 may be used to decode the compressed bitstream 420. For example, the decoder 500 may produce an output video stream 516 without the post-filtering stage 514. In some embodiments of the decoder 500, the post-filtering stage 514 is applied after the loop filtering stage 512. The loop filtering stage 512 may include an optional deblocking filtering stage. Optionally or alternatively, the encoder 400 includes an optional deblocking filtering stage in the loop filtering stage 416.

[0060] Figure 6 is a frame according to an embodiment of the present disclosure—such as Figure 3 600 of a frame 306. As shown, the portion 600 of the frame includes four 64×64 blocks 610, which may be referred to as super blocks, in two rows and two columns in a matrix or Cartesian plane. The super blocks may be larger or smaller in size. Although Figure 6 The description is made with respect to superblocks of size 64×64, but the description is easily extended to larger (eg, 128×128) or smaller superblock sizes.

[0061] In an example, and without loss of versatility, a super block may be a basic or maximum coding unit (CU). Each super block may include four 32×32 blocks 620. Each 32×32 block 620 may include four 16×16 blocks 630. Each 16×16 block 630 may include four 8×8 blocks 640. Each 8×8 block 640 may include four 4×4 blocks 650. Each 4×4 block 650 may include 16 pixels, which may be represented in four rows and four columns in a Cartesian plane or matrix. The pixel may include information representing an image captured in a frame, such as brightness information, color information, and position information. In an example, a block, such as a 16×16 pixel block as shown, may include a luma block 660, which may include a luma pixel 662 and two chroma blocks 670 / 680, such as a U or Cb chroma block 670 and a V or Cr chroma block 680. The chroma block 670 / 680 may include a chroma pixel 690. For example, luma block 660 includes 16×16 luma pixels 662, and each chroma block 670 / 680 includes 8×8 chroma pixels 690 as shown. Although one arrangement of blocks is shown, any arrangement may be used. Figure 6 N×N blocks are shown, but in some embodiments, N×M can be used, where N≠M. For example, 32×64 blocks, 64×32 blocks, 16×32 blocks, 32×16 blocks, or any other size blocks can be used. In some embodiments, N×2N blocks, 2N×N blocks, or a combination thereof can be used.

[0062] In some embodiments, video coding may include ordered block-level coding. Ordered block-level coding may include coding blocks of a frame in a certain order, such as a raster scan order, where blocks can be identified and processed starting with a block in the upper left corner of the frame or portion of the frame, and proceeding along rows from left to right and from the top row to the bottom row, identifying each block in turn for processing. For example, the super block in the top row and left column of the frame can be the first block to be coded, and the super block immediately to the right of the first block can be the second block to be coded. The second row from the top can be the second row to be coded, so that the super block in the left column of the second row can be coded after the super block in the rightmost column of the first row.

[0063] In an example, coding the block may include using quadtree coding, which may include coding smaller block units in a raster scan order. Figure 6The 64×64 block shown in the lower left corner of the portion of the frame shown in is coded, in which quadtree coding, the 32×32 block at the upper left may be coded, then the 32×32 block at the upper right may be coded, then the 32×32 block at the lower left may be coded, and finally the 32×32 block at the lower right may be coded. Each 32×32 block may be coded using quadtree coding, in which the 16×16 block at the upper left may be coded, then the 16×16 block at the upper right may be coded, then the 16×16 block at the lower left may be coded, and finally the 16×16 block at the lower right may be coded. Each 16×16 block may be coded using quadtree coding, in which the upper left 8×8 block may be coded, then the upper right 8×8 block may be coded, then the lower left 8×8 block may be coded, and then the lower right 8×8 block may be coded. Each 8×8 block may be coded using quadtree coding, in which the upper left 4×4 block may be coded, then the upper right 4×4 block may be coded, then the lower left 4×4 block may be coded, and then the lower right 4×4 block may be coded. In some embodiments, for 16×16 blocks, the 8×8 block may be omitted, and the 16×16 block may be coded using quadtree coding, in which the upper left 4×4 block may be coded, and then the other 4×4 blocks in the 16×16 block may be coded in raster scan order.

[0064] In an example, video coding may include compressing information included in an original or input frame by omitting some information in the original frame from the corresponding encoded frame. For example, coding may include reducing spectral redundancy, reducing spatial redundancy, reducing temporal redundancy, or a combination thereof.

[0065] In an example, reducing spectral redundancy may include using a color model based on a luma component (Y) and two chroma components (U and V or Cb and Cr), which may be referred to as a YUV or YCbCr color model or color space. Using the YUV color model may include using a relatively large amount of information to represent the luma component of a portion of a frame, and using a relatively small amount of information to represent each corresponding chroma component of the portion of the frame. For example, a portion of a frame may be represented by a high-resolution luma component, which may include a 16×16 pixel block, and represented by two lower-resolution chroma components, each of which represents a portion of the frame as an 8×8 pixel block. A pixel may indicate a value (e.g., a value ranging from 0 to 255) and may be stored or transmitted using, for example, 8 bits. Although the present disclosure is described with reference to the YUV color model, any color model may be used.

[0066] Reducing spatial redundancy may include transforming the blocks into the frequency domain, as described above. For example, a unit of an encoder such as Figure 4 The entropy coding unit 408 may perform DCT using the transform coefficient values ​​based on spatial frequencies.

[0067] Reducing temporal redundancy can include using similarities between frames to encode frames using a relatively small amount of data based on one or more reference frames, which can be previously encoded, decoded, and reconstructed frames of a video stream. For example, blocks or pixels of a current frame can be similar to spatially corresponding blocks and pixels of a reference frame. Blocks or pixels of a current frame can be similar to blocks or pixels of a reference frame at a different spatial location. Thus, reducing temporal redundancy can include generating motion information indicating spatial differences (e.g., translations between the positions of blocks or pixels in the current frame and corresponding positions of blocks or pixels in the reference frame).

[0068] Reducing temporal redundancy may include identifying a block or pixel in a reference frame or a portion of a reference frame that corresponds to a current block or pixel of a current frame. For example, a reference frame or a portion of a reference frame, which may be stored in a memory, may be searched for an optimal block or pixel for encoding the current block or pixel of the current frame. For example, the search may identify a block of the reference frame for which the difference in pixel values ​​between the reference block and the current block is minimized, and may be referred to as a motion search. The portion of the reference frame that is searched may be limited. For example, the portion of the reference frame that is searched, which may be referred to as a search area, may include a limited number of rows of the reference frame. In an example, identifying a reference block may include calculating a cost function, such as a sum of absolute differences (SAD), between pixels of a block in the search area and pixels of the current block.

[0069] The spatial difference between the position of the reference block in the reference frame and the position of the current block in the current frame can be represented as a motion vector. The difference in pixel values ​​between the reference block and the current block can be referred to as differential data, residual data, or as a residual block. In some embodiments, generating a motion vector can be referred to as motion estimation, and the pixels of the current block can be represented using Cartesian coordinates, such as f x,y - based on position. Similarly, the pixels of the search area of ​​the reference frame can be indicated using Cartesian coordinates - r x,y - Indicated based on position. The motion vector (MV) of the current block may be determined based on, for example, the SAD between a pixel of the current frame and a corresponding pixel of a reference frame.

[0070] Although other partitions are possible, as described above for Figure 6 described, but a CU or block can use Figure 7 The example shows a quadtree partitioning of block 700. However, block 700 may be partitioned in a different manner, such as by an encoder (e.g., Figure 4 encoder 400) or a machine learning model.

[0071] Block 700 is partitioned into four blocks, namely, blocks 700-1, 700-2, 700-3, and 700-4. Block 700-2 is further partitioned into blocks 702-1, 702-2, 702-3, and 702-4. Thus, if, for example, the size of block 700 is N×N (e.g., 128×128), blocks 700-1, 700-2, 700-3, and 700-4 are each of size N / 2×N / 2 (e.g., 64×64), and blocks 702-1, 702-2, 702-3, and 702-4 are each of size N / 4×N / 4 (e.g., 32×32). If the block is partitioned, it is partitioned into four equally sized non-overlapping square sub-blocks.

[0072] The quadtree data representation is used to describe how to segment a block 700 into sub-blocks, such as blocks 700-1, 700-2, 700-3, 700-4, 702-1, 702-2, 702-3, and 702-4. The segmented quadtree 704 of block 700 is shown. If the node is further split into four child nodes, each node of the quadtree 704 is assigned a flag "1", and if the node is not split, a flag "0" is assigned. The flag may be referred to as a split bit (e.g., 1) or a stop bit (e.g., 0), and is coded in a compressed bit stream. In a quadtree, a node either has four child nodes or no child nodes. A node without a child node corresponds to a block that has not been further split. Each child node of a split block corresponds to a sub-block.

[0073] In quadtree 704, each node corresponds to a sub-block of block 700. The corresponding sub-blocks are shown between brackets. For example, node 704-1 with a value of 0 corresponds to block 700-1.

[0074] Root node 704-0 corresponds to block 700. When block 700 is split into four sub-blocks, the value of root node 704-0 is a split bit (e.g., 1). At the intermediate level, the flag indicates whether the sub-block of block 700 is further split into four sub-sub-blocks. In this case, node 704-2 includes a flag "1" because block 700-2 is split into blocks 702-1, 702-2, 702-3, and 702-4. Each of nodes 704-1, 704-3, and 704-4 includes a flag "0" because the corresponding blocks are not split. When nodes 704-5, 704-6, 704-7, and 704-8 are at the bottom level of the quadtree, these nodes do not need flags "0" and "1". Blocks 702-5, 702-6, 702-7, and 702-8 are not further split, which can be inferred from the lack of additional flags corresponding to these blocks. In this example, the smallest sub-block is 32x32 pixels, but further partitioning is possible.

[0075] The quadtree data of the quadtree 704 can be represented by binary data "10100", where each bit represents a node of the quadtree 704. The binary data indicates the partitioning of the block 700 to the encoder and decoder. Figure 5 In the case of decoder 500, the encoder may compress the bit stream such as Figure 4 The binary data is encoded in a compressed bit stream 420.

[0076] The blocks corresponding to the leaf nodes of the quadtree 704 may be used as the basis for prediction. That is, prediction may be performed on each of the blocks 700-1, 702-1, 702-2, 702-3, 702-4, 700-3, and 700-4, referred to herein as coded blocks. Figure 6 As mentioned, the coded block can be a luma block or a chroma block. Note that in the example, the block partitioning can be determined with respect to the luma block. The same partitioning or a different partitioning can be used with the chroma block.

[0077] The prediction type (e.g., intra or inter prediction) is determined at the coded block. That is, the coded block is the decision point for the prediction.

[0078] A mode decision process (e.g., a partition decision process) determines the partitioning of a coded block, such as block 700. The partition decision process calculates the RD cost for different combinations of coding parameters. That is, for example, different combinations of prediction blocks and predictions (e.g., intra prediction, inter prediction, etc.) are examined to determine the best partitioning.

[0079] As will be appreciated by those skilled in the art, many mode decision processes may be performed by an encoder.

[0080] Figure 8 800 is a flow chart of a process 800 for searching for an optimal mode for coding a block. Process 800 is an illustrative high-level process for a mode decision process for determining an optimal mode. For ease of description, process 800 is described with respect to selecting an intra prediction mode for encoding a prediction block. Other examples of optimal modes that may be determined by a process similar to process 800 include determining a transform type and determining a transform size. Process 800 may be performed by an encoder, such as Figure 4 Encoder 400 - implemented using a brute force approach to mode decision making.

[0081] At 802, process 800 receives an image block. Since process 800 is described with respect to determining an intra prediction mode, the image block may be a prediction unit. Figure 7 As described, each leaf node coding block (e.g., block 700-1, 702-1, 702-2, 702-3, 702-4, 700-3, or 700-4) may be split into one or more prediction units until a minimum prediction unit / block size is reached so that further splitting is not possible. An image block may be one such prediction unit.

[0082] At 804, process 800 determines (e.g., selects, calculates, chooses, etc.) a pattern list. The pattern list may include K nodes, where K is an integer. The pattern list may be represented as {m1, m2, ..., m k}. The encoder may have a list of available intra-frame prediction modes. For example, the list of available intra-frame prediction modes may be {DC_PRED, V_PRED, H_PRED, D45_PRED, D135_PRED, D117_PRED, D153_PRED, D207_PRED, D63_PRED, SMOOTH_PRED, SMOOTH_V_PRED and SMOOTH_H_PRED, PAETH_PRED}. The description of these intra-frame prediction modes is omitted because the description is not relevant to the understanding of the present disclosure. The list of modes determined at 804 may be any subset of the list of available intra-frame prediction modes.

[0083] In block 806 , process 800 initializes the BEST_COST variable to a high value (eg, INT_MAX, which may be equal to 2,147,483,647) and initializes a loop variable i to 1, which corresponds to the first pattern to be checked.

[0084] In 808, process 800 is mode i Calculate (e.g. calculate) RD_COST i At 810, process 800 tests the current mode mode in the check mode i RD cost RD_COST i Is it less than the current best cost BEST_COST. If the test is positive, then at 812, process 800 updates the best cost to the cost of the current mode (ie, BEST_COST = RD_COST i ), and sets the current best mode index (BEST_MODE) to loop variable i (BEST_MODE=i). Process 800 then continues to 814, incrementing loop variable i (i.e., i=i+1) in preparation for checking the next mode (if any). If the test is negative, process 800 continues to 814.

[0085] At 816, if there are more modes to check, the process 800 continues back to 808; otherwise, the process 800 continues to 816. At 816, the process 800 outputs the index of the best mode BEST_MODE. Outputting the best mode may mean returning the best mode to the caller of the process 800. Outputting the best mode may mean encoding the image using the best mode. Outputting the best mode may have other semantics. The process 800 terminates at 820.

[0086] Figure 8 The brute force method of mode decision is illustrated to be largely a serial process, which essentially encodes the image block X using candidate modes to determine the mode with the best cost. Machine learning can be used to reduce the computational complexity of mode decision. That is, the mode decision of the encoder can be implemented using deep learning. Deep learning refers to a machine learning model in which the network learns from unstructured or unlabeled data, and the deep learning model can be referred to as a deep neural network.

[0087] At a high level, and without loss of generality, a machine learning model, such as a classification deep learning model, consists of two main parts: a feature extraction part and a classification part. The feature extraction part detects features of the model. The classification part attempts to classify the detected features into an expected response. Each part can include one or more layers and / or one or more operations.

[0088] As mentioned above, CNN is an example of a machine learning model. A simplified CNN 900 is shown in Fig. 9 As shown. In the CNN 900, the feature extraction portion 902 includes a set of convolution operations, which can be a series of filters set in convolution layers 904, 906, 908, which are used to filter the input image based on filters (usually squares of size k, without loss of generality). For example, and in the context of machine vision, these filters can be used to find features in the input image. For example, features can include edges, corners, endpoints, etc. As the number of stacked convolution operations increases, later convolution operations can find more advanced features.

[0089] In CNN 900, several fully connected (FC) layers 912, 914 can look at all input features of an image to generate a high-level classifier. Several stages (e.g., a series) of high-level classifiers ultimately generate the desired classification output. Collectively, these classifiers form the classification portion 910.

[0090] As can be seen from this description, the CNN network is typically composed of several convolution operations (e.g., convolution layers 904, 906, 908 of the feature extraction part 902) and several FC layers 912, 914 forming the classification part. The number of operations of each type and the corresponding size are usually determined in the training phase of machine learning. As those skilled in the art will recognize, additional layers and / or operations may be included in each part. For example, pooling, maximum pooling, exit, activation, normalization, batch normalization, and other operations may be grouped with convolution operations (i.e., in the feature extraction part 902) and / or fully connected operations (i.e., in the classification part 910). FC layers 912, 914 may be referred to as dense operations. As those skilled in the art will recognize, the convolution operations of any convolution layer 904, 906, 908 may use separable convolution 2D or convolution 2D operations.

[0091] As used in the present disclosure, a convolutional layer may be a set of operations starting with a convolution 2D or a separable convolution 2D, followed by zero or more operations (e.g., pooling, dropout, activation, normalization, batch normalization, other operations, or a combination thereof) until reaching another convolutional layer, a dense operation, or the output of the CNN 900. Similarly, a dense layer may be a set of operations or layers starting with a dense operation (i.e., FC layers 912, 914), followed by zero or more operations (e.g., pooling, dropout, activation, normalization, batch normalization, other operations, or a combination thereof) until reaching another convolutional layer, another dense layer, or the output of the CNN 900. The boundary between the feature extraction portion 902 based on the convolutional network and the feature classification portion 910 using dense operations may be marked by a flattening operation that flattens a multi-dimensional matrix from feature extraction into a vector.

[0092] In CNN 900, each convolution layer 904, 906, 908 can be composed of a set of filters. When the filter is applied to a subset of the input data each time, the filter is applied to the entire input, such as by sweeping the input. The operation performed by the convolution layer is usually linear / matrix multiplication. The output of the convolution filter can be further filtered using an activation function. The activation function can be a linear function or a nonlinear function (e.g., a sigmoid function, an arcTan function, a tanH function, a ReLu function, etc.).

[0093] Each FC layer 912, 914 is a linear operation, where each input is connected to each output through a weight. The linear operation of the FC layer is usually followed by a nonlinear activation function to generate the output of the layer. When there is a flattening operation (e.g., a flattening layer) between the last convolutional layer (e.g., convolutional layer 908) and the first FC layer (e.g., FC layer 912), the number of parameters of the entire neural network is determined by the number of parameters at this interface.

[0094] Note that details of machine learning, convolutional neural networks, and / or details known to those skilled in the art are omitted here. For example, those skilled in the art recognize that during the training phase, the values ​​of the convolutional filters and the weights of the connections between the nodes (i.e., neurons) in the CNN 900 are determined by the CNN 900. Therefore, these are not discussed in detail here.

[0095] In about Figure 7 and Figure 8 One problem with training a model with the segmentation decisions described is that the segmentation decisions rely on very nonlinear operations. For example, and as mentioned above about Figure 4As described, the block residual is the (linear) difference between the image block and the predicted block (e.g., in the intra / inter prediction stage 402), but the residual is then subjected to non-linear operations, including computing a transform of the residual (e.g., in the transform stage 404) and applying quantization to the transformed residual (e.g., in the quantization stage 406). Further, a reconstructed version of the image block (e.g., by Figure 4 The number of bits (rate R) required to encode the image block rate and the distortion D between the image blocks in the reconstruction path performed) to obtain the RD cost as a scalar value D + λR also includes nonlinear operations. For example, the distortion can be the mean square error, the sum of absolute difference errors, or some other error measure between the pixel values ​​of the image block and the reconstructed block.

[0096] The Lagrangian parameter or the Lagrangian multiplier λ itself may have a nonlinear relationship with the quantization parameter QP used to quantize the transformed residual. For example, a codec implementing the H.264 or HEVC standard may be based on 0.85×2 (QP－12) / 3 Derived from the multiplier, a codec implementing the H.263 standard can use 0.85 · QP 2 Derived from the multiplier, a codec implementing the VP9 standard can use 88 · QP 2 / 24 derives the multiplier, and a codec implementing the AVI standard can use 0.12 · QP 2 / 256 derives the multiplier.

[0097] To approximate the segmentation problem with a machine learning or deep learning model, the labeled segments (e.g., Figure 7 The exhaustive encoder segmentation described above) is compared to the predicted segmentation. As described above, for example, the loss function measures how well the model's predicted value p matches the expected result y, in this example how well the predicted segmentation of the block matches the labeled segmentation of the block from the training data. The loss function can be an MSE loss function, a binary cross entropy loss function, or a cross entropy loss function.

[0098] The formula for the MSE loss function is: , Where [n] represents sample n, and N represents the total number of samples. The formula for the binary cross entropy function is: .

[0099] The cross entropy function is a general case of the binary cross entropy function, where there are only two choices. The decision process can be represented by a predictor variable that takes values ​​from a finite alphabet A. The decision process can be shown in the formula of the cross entropy function below: , where i denotes the index of the choice of the predictor variable in A. In the binary case, for example, i is either 0 or 1. In the case where the predictor variable can take values ​​from the alphabet {0, 1, ...A-1} (e.g., A>2), i can be 0, ...A-1, ... In other words, the function iterates over the number of available prediction classes using index i. For two-class problems (e.g., quadtree splitting or not splitting), the cross entropy function can be simplified to the binary cross entropy function described above.

[0100] As mentioned above Figure 6 and Figure 7 As described, the split decision can be a quadtree decision process. In this process, the non-split decision of a block of size b×b (also referred to as block b×b in this article) means that the size The sub-blocks (also referred to as blocks in this article) ) does not need to be evaluated. In this example, assuming the maximum block size is 64×64 pixels and the minimum block size is 8×8 pixels, the modified binary entropy loss function that can be used is as follows: (1)

[0101] On top, y 64 and p 64 Corresponding to the marked segmentation and predicted segmentation (or prediction) of 64×64 blocks respectively. Similarly, and corresponds to the labeled segmentation and prediction of 32×32 blocks, respectively, and and Corresponding to the labeled segmentation and prediction of 16×16 blocks, respectively. When all corresponding blocks are split, an 8×8 segmentation is assumed. This is an example of a modified loss function. Additional or fewer terms may be included. For example, additional terms may be added where the maximum block size is 128×128 pixels.

[0102] One problem with the above technique is that it gives equal weights to different segmentation decisions. Including a weighting function for different segmentation decisions may lead to additional benefits. This modifies the loss function of Eq. (1) to: (2)

[0103] The weighting function may weight the segmentation decisions proportionally, such as by applying a scaling factor to the rate and distortion costs. The scaling factor of the weighting function may be developed by considering the rate and distortion costs incurred due to the segmentation decision. At each segmentation decision, for example, the encoder may perform a comparison as follows: (3)

[0104] Equation (3) determines the rate and distortion of a block of size b × b (also called block b × b), and recursively computes the best decision for each sub-block i, where is the RD cost for not splitting block b × b, RDC b×b is the optimal RD cost between the splitting decision and the non-split decision for block b × b, The size is The best RD cost between the split decision and the non-split decision of the sub-block, and Δ b×b is the additional cost incurred when splitting block b × b. In other words, Equation (3) compares the cost of not splitting block b × b with the cost of splitting block b × b into four The costs of the sub-blocks are compared and the lower cost is chosen to make the split decision.

[0105] By identifying whether the segmentation decision is a binary decision, Equation (3) can be converted into a loss function for training the deep learning model. Assume that the variable p b represents the decision of the block, then the variable p b May belong to the set of 0 and 1 (p b ∈ [0,1]). For labeled training data, in this example, the variable p b can have a value of 0 or 1. Therefore, when p b =1, the RD cost is , and when p b =0, the RD cost is More generally, multiply the non-splitting decision by the variable p b , and multiply the split decision by the expression (1- p b ), resulting in the following loss function: (4)

[0106] Making the same assumptions as those made about (1) and (2), (4) can be extended to consider all splits of the recursive split decision, leading to the following loss function: , in, The calculation of the cost of whether to further split the i-th 32×32 block can be expressed as follows:

[0107] Therefore, RDC(p) can be expressed as follows: (5)

[0108] Assume that the encoder performs an exhaustive search for the lowest (ie, optimal) RD cost, which can be referred to as RDC MIN The weight of a segmentation decision applied to a segmentation decision may be equal to the rate-distortion value of the segmentation decision normalized by the lowest rate-distortion value produced by the plurality of segmentation decisions. That is, the scaling factor may be 1 / RDC MIN This can be done by dividing RDC(p) in equation (5) by RDC MIN As can be seen, the results are as follows: (6)

[0109] Among them, the constant (weight) and constant . In this example, the predicted rate-distortion value includes a first term representing a rate-distortion cost of a non-split decision for the block and a second term representing a rate-distortion cost of a split decision for the block to be split into a plurality of sub-blocks. The rate-distortion cost of the split decision for the block to be split into a plurality of sub-blocks may include a corresponding rate-distortion cost of the split decision for each of the plurality of sub-blocks. As can be seen from equation (6), the rate-distortion cost of the first term is weighted by the lowest rate-distortion value used to encode the training block. Similarly, the rate-distortion cost of the second term includes each corresponding rate-distortion cost of splitting each of the plurality of sub-blocks weighted by the lowest rate-distortion value used to encode the training block.

[0110] It is worth noting that in formula (6) This is because is the minimum RD cost of the segmentation of the 64×64 block resulting from the exhaustive search, and the prediction will not produce an RD cost lower than this minimum RD cost. From this knowledge, Equation (6) can develop a rate-distortion cost loss function Loss(RDC), which is based on the relationship between the rate-distortion value and the corresponding predicted rate-distortion value. In this example, it is the relationship between the labeled data y and the predicted data p of the corresponding segmentation decision, where the segmentation decision can include a combination of splitting decisions and non-split decisions of the block. The rate-distortion cost loss function can be an MSE function or an exponential function. An example of an exponential function is as follows: (7) Among them, RDC(y) = RDC(1)=RDC MIN

[0111] The rate-distortion cost loss function of equation (7) penalizes bad decisions exponentially. A bad decision is a decision that is worse than optimal, e.g., the predicted RD cost of the segmentation is higher than the RD cost of the labeled data used for training. In other words, the rate-distortion cost loss function penalizes exponentially predicted segmentations that do not match the selected segmentation decisions of the training blocks resulting from an exhaustive segmentation search (e.g., of a block-based encoder). Further, the complexity of this determination is low. This low complexity is due to normalizing the RD cost by dividing it by the RD cost of the labeled data, since the RD cost of the best segmentation (e.g., RDM MIN ) is known.

[0112] Further benefits can be derived from recalling that the encoder can select the best segmentation by computing and comparing the RD costs of all segmentation types according to Equations (1) and (2). For this reason, the segmentation determined by the encoder is a good target for the machine learning model. The combined loss function that includes the segmentation loss function and the RDC-based cost function of Equation (7) - which penalizes bad decisions - is as follows: Loss=αLoss(partition) + βLoss(RDC) (8)

[0113] In formula (8), the segmentation loss function Loss (partition) can be a relationship between a segmentation decision based on a training block and a corresponding predicted segmentation. The segmentation loss function can be an MSE loss function, a binary cross entropy loss function, or a cross entropy loss function. For example, the hierarchical binary cross entropy loss function of formula (1) or the weighted loss function of formula (2) can be used. The loss function of formula (8) can weight the segmentation loss function and the rate-distortion cost loss function differently to determine a combined loss function. The weighting can be implemented using variables α and β. The variables α and β are multipliers for balancing the weights of these functions and can be predefined or adjusted during training. For example, the function Loss (RDC) can first be used to train the model, and then the function Loss (partition) can be used to fine-tune the model. In the example, α + β = 1, where 1> α, β> 0. The variable α can be 10%, and the variable β can be 90%. Other values ​​of the variables α and β may be used to weight the segmentation loss function and the rate-distortion cost loss function. In some implementations, a machine learning model can be trained using a loss function by applying a ratio-distortion cost loss function with a large weight to the segmentation loss function.

[0114] Comparing Eq. (6) used to develop Eq. (7) and (8) with Eq. (2), we can see that by allowing the weight w to be adjusted for each block for different segmentation decisions b , Equation (6) improves the use of weighting function. In contrast, Equation (2) uses weight wb fixed value.

[0115] Reference Fig. 10A and Fig. 10B , it can be seen that the weight w of the weighting function is adjusted b The desirability of the value. Fig. 10A illustrates the segmentation decisions for a b×b block 1000 having relatively uniform content, and Fig. 10B The segmentation decision of a b×b block 1002 with more detailed content (e.g., Fig. 10A The pixel values ​​have greater differences than relatively uniform content).

[0116] The segmentation decision performed by the (eg, software) encoder can be used to extract the segmentation y b , and the segmentation prediction obtained from the machine learning / deep learning model is identified as p b .exist Fig. 10A Due to the relatively uniform content, the encoder is likely to convert y b is set to a value so that the b×b block 1000 should not be split (i.e., not divided into b / 2×b / 2 blocks as indicated by the dashed lines) because the block 1000 can be represented by a single (intensity and color) component. In the example, y b =1. If the segmentation prediction p b is a wrong decision, then when Loss (partition) is a weighted loss function, the segmentation loss cost will be at most w b In some cases, the RD costs of the non-splitting decision and the splitting decision may be very similar. Therefore, the RD costs may be almost the same, resulting in a small penalty. That is, Loss(RDC) is small.

[0117] exist Fig. 10B In the case of b is set to a value such that the b×b block 1000 should be split (i.e., divided into b / 2×b / 2 blocks represented by the solid lines) because the additional detail makes the representation of the individual components undesirable. For example, for block 1002, y b = 0. If a bad decision is made by Fig. 10B The model is made in (for example, without splitting the b×b block into 1000), if the w in equation (2) is b By weighting the splits, at most w b However, Fig. 10A In contrast, the RD cost difference may be much higher. This difference can be represented by an exponential penalty represented by Loss(RDC) in Eq. (8).

[0118] Therefore, and in general, the rate in the RD cost is used to take into account how the encoder processes a given content. In other words, the rate provides an estimate of the complexity of a given encoder's block. This complexity aspect cannot be captured in the distortion.

[0119] Although the examples and formulas described so far are expressed only using quadtree partitioning, many codecs support non-quadtree (e.g., non-square) partitioning. The following formula extends the teachings of this article to non-quadtree partitioning by including it in equation (5) and by including it in the calculation of Loss(RDC): (9) and (10)

[0120] In equations (9) and (10), i is the index of the non-quadtree partition to be considered for the current block size b×b. Furthermore, k is the index of the sub-partition in the current partition i, so that is the RD cost of the current sub-partition k of the current non-quadtree partition i. For example, and referring to Fig.11 , each block size up to the minimum block size (here b / 8×b / 8) includes a decision whether to perform a vertical split, a horizontal split, a split split, or no split. At a particular block size level, the decision as to whether to split and which split is best can be based on an error value, such as an RD cost. If no split is made at the block size level, no further splitting occurs. Likewise, in this example, if a vertical or horizontal split is selected, no further splitting into child nodes occurs. If a split split is selected, further splitting is possible.

[0121] In this example, when the largest CU is a b×b block, a split or no split decision is made. No split results in a final block size of b×b pixels. Otherwise, the vertical split of the b×b block consists of two splits k of b / 2×b pixels, the horizontal split of the b×b block consists of two splits k of b×b / 2 pixels, or the split split of the b×b block each consists of four splits of b / 2×b / 2. When considering whether the split split mode is the best mode, consider the recursive splits of each b / 2×b / 2 block (given by Fig.11). As each b / 2×b / 2 block is considered as part of this recursive partitioning, another partition or no partition decision is made. No partitioning of the b / 2×b / 2 block results in a final block size of b / 2×b / 2 pixels. When the b / 2×b / 2 block is partitioned, the vertical partitioning consists of two partitions k of b / 4×b / 2 pixels, the horizontal partitioning consists of two partitions k of b / 2×b / 4 pixels, or the split partitioning each consists of four partitions of b / 4×b / 4. Similarly, when a split partitioning of the b / 2×b / 2 block occurs, another partition or no partition decision is considered for each b / 4×b / 4 block. No partitioning of the b / 4×b / 4 block results in a final block size of b / 4×b / 4 pixels. The vertical partition of the b / 4×b / 4 block includes two partitions k of b / 8×b / 8 pixels, the horizontal partition of the b / 4×b / 4 block includes two partitions k of b / 4×b / 8 pixels, and the split partitions of the b / 4×b / 4 block each include four partitions of b / 8×b / 8 pixels.

[0122] If you can Fig.11 As can be seen, and given that Figure 7 , the partition can be represented by a tree. The tree can be represented by a vector. If P represents the set of all valid partitions (or, equivalently, the corresponding representation of the partition), then the CNN can be trained to infer a mapping to the set P. Training the CNN to infer the partition comprises defining an appropriate set P and using appropriate training data for each index i, k of the set P. For example, in the case where the codec uses a coding unit size of 64×64 and four basic partition types, for quadtree partitioning, there are only 21 decisions corresponding to one 64×64 partition decision, four 32×32 partition decisions, and sixteen 16×16 partition decisions. In the case where the CNN is also used to determine non-quadtree partitions, there are 21*4=84 possible decisions, of which 21 correspond to quadtree partitions and 4 correspond to Fig.11 Corresponding to the four basic segmentation types.

[0123] Additional partition types may result from modifications to the vertical partition type and the horizontal partition type. For example, a non-square partition may correspond to an internal node of a quadtree having a number of child nodes greater than or equal to two child nodes. A partition type may split a b×b block into two horizontally adjacent square blocks, each of size b / 2×b / 2, and a rectangular prediction unit of size b×b / 2. A partition type may split a b×b block into a rectangular prediction unit of size b×b / 2 and two horizontally adjacent square blocks, each of size b / 2×b / 2. A partition type may split a b×b block into two vertically adjacent square blocks—each of size b / 2×b / 2—and a rectangular prediction unit of size b / 2×b. A partition type may split a b×b block into a rectangular prediction unit of size b / 2×b and two vertically adjacent square blocks, each of size b / 2×b / 2. A partition type may split a b×b block into a rectangular prediction unit of size b / 2×b and two vertically adjacent square blocks, each of size b / 2×b / 2. A partition type may split a b×b block into vertically adjacent rectangular blocks, each of size b×b / 4. The partition type can split the b×b block into four horizontally adjacent rectangular blocks, each of which is of size b / 4×b. This adds another six partition types, which can be referred to as extended partition types.

[0124] In cases where additional partition types are added, the number of decisions increases rapidly. In the example where the codec uses a maximum coding unit size of 128×128 pixels, four basic partition types, and six extended partition types (10 partition types in total), for quadtree partitioning, there are only 85 decisions per partition type (corresponding to one 128×128 partition decision, four 64×64 partition decisions, sixteen 32×32 partition decisions, and forty-six partition decisions). In cases where the model is also used to determine non-quadtree partitions, there can be 850 decisions (corresponding to 85 decisions times 10 partition types).

[0125] Fig.11 It also helps to understand the development of Equation (5), and thus understand Loss (RDC). More specifically, and as Fig.11 As shown by the shaded block in , the split decision can include recursive splits, where the divisive split pattern needs to consider the available splits for each resulting block. More generally, to determine whether a split or no split of block b×b is better, the rate and distortion incurred by not splitting the block can be compared to the total rate and distortion incurred by subsequent split decisions. Fig.11 In equation (5), only the split segmentation can lead to further segmentation decisions. However, in other embodiments, further segmentation decisions of other segmentation modes are possible.

[0126] To incorporate these techniques into machine learning / deep learning models, one can use Fig.12The structure of Fig.12 In the example, the model 1200 may be a Fig. 9 The model 1200 is trained using training data 1204 to infer mode decisions for image blocks. The image block can be a block of an image (e.g., a video frame) to be encoded using inter-frame or intra-frame prediction. In this example, the mode decision is a segmentation decision for the image block. The segmentation decision can be a quadtree segmentation decision for the image block. In another example, the segmentation decision can be a decision including information about Fig.11 Segmentation of the described segmentation.

[0127] To train the model 1200, an RD cost estimator 1202 is attached to the model 1200, but without any learnable parameters. For ease of explanation, this combination of the model 1200 and the RD cost estimator 1202 may be referred to herein as a machine learning module. The model 1200 may be trained using training data 1204 of a plurality of video blocks (also referred to as training blocks) as input to train the machine learning model. The plurality of training blocks are encoded using an encoder through a conventional encoding method (such as Figure 4 and Figures 6 to 8 The encoding method described herein is encoded. In this document, an encoder that performs an exhaustive search and an encoder that uses model 1200 may be referred to as a first encoder or a second encoder, which is only to distinguish each other and has nothing to do with the order in which the encoders are used. For a training block in a plurality of training blocks, the training data 1204 may include a segmentation decision determined by an encoder that implements a traditional encoding method, for example, all segmentations of all segmentation patterns used in the exhaustive search of the encoder. For a training block in a plurality of training blocks, the training data may include a rate-distortion value resulting from encoding the training block using the segmentation decision. For each segmentation decision, the rate-distortion value resulting from encoding the training block using the segmentation decision may include a rate-distortion cost for encoding the training block without further segmenting the training block and a cost incurred when splitting the training block. For example, the rate-distortion value may be an RDC corresponding to the segmentation decision. b and Δ b (and the corresponding RDC MIN ) corresponding values ​​of ω and ο. Many indications (e.g., representations) of the segmentation decisions are possible. In an example, a vector (e.g., sequence) of binary flags as described with respect to the quadtree 704 may be used for the corresponding values ​​of ω and ο for each segmentation decision.

[0128] A subset 1206 of the training data 1204 is input into the RD cost estimator 1202 to estimate the RD cost of the segmentation output from the model 1200. The rate-distortion cost estimator can receive the predicted segmentation from the machine learning model, optionally with the rate-distortion value corresponding to the segmentation decision (e.g., with RDCb and Δ b The corresponding values ​​of ω and ο) are combined and the predicted rate-distortion value of the loss function is estimated. For example, during the training phase, the parameters of the model 1200 are generated so that the model 1200 infers at least some split / no split decisions (predicted split p b ), such as Figure 7 The quadtree segmentation decision described is used, and the predicted segmentation is used to calculate the Loss (RDC) according to Equation (7). b exist Fig.12 In the example, this is represented by the arrows labeled 64, 32, and 8 / 16, which represent the quadtree decisions for blocks of size 64×64, 32×32, and 16×16, respectively. b and the labeled segmentation y of the segmentation decision of the training data 1204 b It can be used to determine Loss(partition) according to, for example, equation (1) or equation (2). The values ​​Loss(partition) and Loss(RDC) can be combined to minimize the value Loss in the training of the model 1200.

[0129] It is noteworthy that this technique for training the model 1200 uses the values ​​of ω and ο extracted from the exhaustive analysis of the encoder to estimate the segmentation decision, which is then used to estimate the cost of the segmentation. An alternative technique is to estimate the values ​​of ω and ο directly as part of the training of the model 1200. The RD cost can then be incorporated directly into the model 1200 to produce the segmentation, rather than using a separate RD cost estimator 1202. However, this latter technique is less desirable because estimating ω and ο is significantly more difficult.

[0130] During training, the model 1200 learns (e.g., trains, builds, derives, etc.) a mapping (i.e., a function) that accepts a block as input and outputs a segmentation of the block. During training, and in order to make the learned function as useful as possible, the model 1200 is preferably trained using a large number of input training blocks that each lead to a different segmentation decision, so that each segmentation decision is associated with multiple input training blocks.

[0131] Fig.13 Is to use from according to Fig.12 Flowchart of process 1300 for encoding inferences obtained by training machine learning / deep learning model 1200. Model 1200 can be used in the inference stage while encoding the current block. That is, during the inference process, RD cost estimator 1202 is removed from its attachment to model 1200, because the main goal of the network is to obtain segmentation of the image block.

[0132] At 1302, an input is presented to the model 1200. The input is an image block to be encoded. The image block may be a color plane block. Thus, the block may be a luma block. The block may be the largest CU of the encoder. Although blocks of size 64×64 are described in the development of the above formula, the blocks may be of any size. For example, the blocks may be 128×128 pixels, 32×32 pixels, or any size, the partitioning of which is typically determined by the encoder. As described above, the encoder may be any block-based encoder.

[0133] At 1304, the model 1200 infers or obtains a partitioning decision for the block. For example, the model 1200 may provide an output (referred to as a partitioning decision) indicating a (e.g., quadtree) partitioning of the block. The partitioning decision may include one or more decisions about whether to split the block into different partitions until a minimum partition size is reached—such as a decision about whether to split the block into different partitions until a minimum partition size is reached. Figure 7 or Fig.11 Decisions described. The split decision may include a single decision, i.e., the block should not be split. The split decision may include a combination of split decisions and no-split decisions for the block. Model 1200 may be used by an encoder where the smallest possible block split is an 8×8 split. Thus, in this example, for blocks of 16×16 or larger (i.e., sub-blocks of the largest CU), the decision or determination as to whether to split the block is made by model 1200. Model 1200 may be used by an encoder where the smallest possible block split is a 4×4 split. Thus, in this example, for blocks of 8×8 or larger (i.e., sub-blocks of the largest CU), the decision or determination as to whether to split the block is made by model 1200.

[0134] At 1306, the image block is encoded using the segmentation decision. Model 1200 provides input to an encoder, such as encoder 400, or may be incorporated into an encoder. For example, the segmentation decision is defined in Figure 4 The block size may be provided to the intra / inter prediction stage 402, or the model 1200 may be incorporated into the intra / inter prediction stage 402. The remainder of the encoding steps for each block may be based on Figure 4 The description of , finally completes entropy coding of each block, as described with respect to the entropy coding stage 408, so that in the compressed bit stream (such as Figure 4 The segmentation decision may also be encoded as a compressed bitstream.

[0135] Decoders such as Figure 5 The decoder 500 may decode the image block using the segmentation decision received in the compressed bit stream. That is, the process of decoding the image block may include receiving the segmentation decision received in the compressed bit stream, such as Figure 5 The method comprises receiving an indication of partitioning an image block into one or more sub-blocks from a compressed bitstream 420; and decoding the image block using the indication of the partitioning by decoding the one or more sub-blocks.

[0136] For ease of explanation, the processes herein are each depicted and described as a series of frames, steps or operations. However, frames, steps or operations according to the present disclosure may occur in various orders and / or simultaneously. Additionally, other steps or operations not presented and described herein may be used. In addition, all steps or operations shown may not be required to implement the technology according to the disclosed subject matter.

[0137] The above encoding and decoding aspects illustrate some encoding and decoding techniques. However, it is to be understood that "encoding" and "decoding", as those terms used in the claims, may refer to compression, decompression, transformation or any other processing or change of data.

[0138] The words "example" or "implementation" are used herein to refer to serving as examples, instances or illustrations. Any aspect or design described as an "example" or "implementation" in this article is not necessarily considered to be preferred or advantageous over other aspects or designs. Specifically, the use of the words "example" or "implementation" is intended to present concepts in a specific way. As used in this application, the term "or" is intended to refer to an inclusive "or", rather than an exclusive "or". That is, unless otherwise specified or explicitly stated in the context, "X includes A or B" is intended to refer to any natural inclusive arrangement thereof. That is, if X includes A, X includes B or X includes A and B, "X includes A or B" is satisfied in any of the above cases. In addition, the articles "one" and "an" used in this application and the appended claims should generally be deemed to refer to "one or more", unless the context explicitly states otherwise that it refers to a singular form. In addition, unless so described, the term "implementation" or "an implementation" is not intended to refer to the same embodiment or implementation from beginning to end.

[0139] The sending station 102 and / or the receiving station 106 (and therefore the algorithms, methods, instructions, etc. stored thereon and / or executed thereby (including by the encoder 400 and the decoder 500)) can be implemented in hardware, software, or any combination thereof. For example, the hardware may include a computer, an intellectual property (IP) core, an application specific integrated circuit (ASIC), a programmable logic array, an optical processor, a programmable logic controller, microcode, a microcontroller, a server, a microprocessor, a digital signal processor, or any other suitable circuit. In the claims, the term "processor" should be understood to include any of the above hardware, whether alone or in combination. The terms "signal" and "data" are used interchangeably. Further, parts of the sending station 102 and the receiving station 106 are not necessarily implemented in the same manner.

[0140] Further, in one aspect, for example, the sending station 102 or the receiving station 106 can be implemented using a general-purpose computer or a general-purpose processor having a computer program that, when executed, performs any corresponding method, algorithm, and / or instruction described herein. Additionally or alternatively, for example, a special-purpose computer / processor can be utilized that can include other hardware for executing any method, algorithm, or instruction described herein.

[0141] For example, sending station 102 and receiver 106 can be implemented on the computer in the video conference system. Alternatively, sending station 102 can be implemented on a server, and receiving station 106 can be implemented on the equipment separated from the server, such as a handheld communication device. In this case, using encoder 400, sending station 102 can be encoded as a coded video signal, and the coded video signal is sent to the communication device. In turn, the communication device can use decoder 500 to decode the coded video signal then. Alternatively, the communication device can decode the content stored locally on the communication device, for example, the content not sent by sending station 102. Other sending stations 102 and receiving station 106 embodiments are also available. For example, receiving station 106 can be a substantially fixed personal computer, rather than portable communication device and / or the equipment comprising encoder 400 can also include decoder 500.

[0142] Further, all or part of the embodiments of the present disclosure may take the form of a computer program product that is accessible from, for example, a tangible computer-usable or computer-readable medium. A computer-usable or computer-readable medium may be any device that, for example, may tangibly contain, store, convey, or transmit a program for use by or in conjunction with any processor. For example, the medium may be an electronic, magnetic, optical, electromagnetic, or semiconductor device. Other suitable media may also be available.

[0143] The above embodiments, implementations and aspects have been described to allow for ease of understanding of the present disclosure and not to limit the present disclosure. On the contrary, the present disclosure is intended to cover various modifications and equivalent arrangements included within the scope of the appended claims, which scope should be given the broadest interpretation to cover all such modifications and equivalent structures permitted by law.

Claims

1. A method for decoding an image block, comprising: Receiving a coded image block from a compressed bitstream, wherein the coded image block is encoded using segmentation decisions obtained from a neural network of a first encoder by: Presenting the image block to the neural network of the first encoder, wherein the neural network is trained using a loss function to output a segmentation decision for encoding the image block, and using: Training data of a plurality of training image blocks, wherein the training data for each training image block in the plurality of training image blocks comprises: a plurality of segmentation decisions, the plurality of segmentation decisions being used by a second encoder to encode the training image blocks; and for each segmentation decision of the plurality of segmentation decisions, a rate-distortion value resulting from encoding the training image block using the segmentation decision; and A loss function, which includes a weighted combination of: a segmentation loss function based on the relationship between the segmentation decisions and the corresponding predicted segmentations; and a rate-distortion cost loss function based on a relationship between the rate-distortion values ​​and corresponding predicted rate-distortion values ​​estimated by a rate-distortion cost estimator coupled to the neural network; and The image block is decoded by decoding sub-blocks of the image block resulting from the segmentation decision.

2. The method according to claim 1, wherein: The loss function combines the segmentation loss function and the rate-distortion cost loss function using different multipliers for the segmentation loss function and the rate-distortion cost loss function.

3. The method according to claim 1, wherein: The segmentation loss function comprises a binary cross entropy function that applies corresponding weights to the segmentation decisions, the weights applied to a segmentation decision among the plurality of segmentation decisions being equal to the rate-distortion value of the segmentation decision normalized by a lowest rate-distortion value produced from the plurality of segmentation decisions.

4. The method according to claim 1, wherein: The rate-distortion cost estimator receives the predicted segmentation from the neural network and estimates a predicted rate-distortion value for the loss function, and the neural network is a convolutional neural network.

5. The method according to claim 1, wherein: For each segmentation decision of the plurality of segmentation decisions, the rate-distortion value resulting from encoding the training image block using the segmentation decision comprises a rate-distortion cost for encoding the training image block without further segmenting the training image block and a cost resulting from splitting the training image block.

6. The method according to any one of claims 1 to 5, wherein: The plurality of segmentation decisions include at least one of: a) a quadtree segmentation decision, and b) a segmentation decision resulting in at least one non-square segmentation of the training image block.

7. An apparatus for decoding an image block, comprising: A processor, the processor being configured to: A coded image block is received from a compressed bitstream, wherein the coded image block is encoded using a segmentation decision determined by: An encoder, the encoder: presenting the image patch to a neural network, wherein the neural network is trained using a loss function to output the segmentation decision for encoding the image patch, and using: Training data of a plurality of training image blocks, wherein the training data for each training image block in the plurality of training image blocks comprises: a plurality of segmentation decisions used by an encoder to encode the training image blocks; and for each segmentation decision of the plurality of segmentation decisions, a rate-distortion value resulting from encoding the training image block using the segmentation decision; and A loss function, which includes a weighted combination of: a segmentation loss function based on the relationship between the segmentation decisions and the corresponding predicted segmentations; and a rate-distortion cost loss function based on a relationship between the rate-distortion values ​​and corresponding predicted rate-distortion values ​​estimated by a rate-distortion cost estimator coupled to the neural network; and The image block is decoded by decoding sub-blocks of the image block resulting from the segmentation decision.

8. The device according to claim 7, wherein: For each segmentation decision of the plurality of segmentation decisions, the rate-distortion cost loss function is an exponential function comprising a difference between the rate-distortion value resulting from encoding the training image block using the segmentation decision and a predicted rate-distortion value.

9. The device according to claim 7, wherein: For each segmentation decision of the plurality of segmentation decisions, a corresponding predicted rate-distortion value is normalized using a lowest rate-distortion value of the plurality of segmentation decisions in the rate-distortion cost loss function.

10. The device according to claim 7, wherein: The rate-distortion cost loss function includes: The predicted rate-distortion value is determined using the segmentation decision of the plurality of segmentation decisions and the rate-distortion value results from encoding the training image block using the segmentation decision of the plurality of segmentation decisions, wherein the segmentation decision comprises a combination of split decisions and non-split decisions for the training image block and the rate-distortion value comprises a combination of rate-distortion values ​​resulting from encoding the training image block using the combination of split decisions and non-split decisions.

11. The device according to claim 10, wherein: The predicted rate-distortion value includes a first term representing a rate-distortion cost of a non-splitting decision of the training image block and a second term representing a rate-distortion cost of a splitting decision of the training image block to split the training image block into a plurality of sub-blocks.

12. The device according to claim 11, wherein: the rate-distortion cost of the splitting decision of the training image block to split the training image block into the plurality of sub-blocks comprises a respective rate-distortion cost of the splitting decision of each of the plurality of sub-blocks; The rate-distortion cost of the first term is weighted by a lowest rate-distortion value of the plurality of segmentation decisions used to encode the training image block; and The rate-distortion cost of the second term comprises weighting each of the respective rate-distortion costs for segmenting each of the plurality of sub-blocks by the lowest rate-distortion value of the plurality of segmentation decisions used to encode the training image block.

13. The device according to any one of claims 7 to 12, wherein: The neural network is trained using the rate-distortion cost estimator, which receives the predicted segmentation from the neural network and estimates a predicted rate-distortion value for the loss function.

14. A non-transitory computer-readable storage medium for storing a compressed bitstream comprising a coded image block, wherein: The encoded image block is encoded using segmentation decisions obtained from a neural network of a first encoder by presenting the image block to the neural network of the first encoder, wherein the neural network is trained using a loss function to output the segmentation decision for encoding the image block, and using training data of a plurality of training image blocks, the training data comprising, for each of the plurality of training image blocks: a plurality of segmentation decisions, the plurality of segmentation decisions being used by a second encoder to encode the training image block; and for each of the plurality of segmentation decisions, a rate-distortion value resulting from encoding the training image block using the segmentation decision; and a loss function comprising a weighted combination of a segmentation loss function based on a relationship between the segmentation decisions and a corresponding predicted segmentation; and a rate-distortion cost loss function based on a relationship between the rate-distortion value and a corresponding predicted rate-distortion value estimated by a rate-distortion cost estimator coupled to the neural network.

Images