Encoder and method of encoding a sequence of frames

Abstract

To provide an encoder capable of encoding a sequence of frames that better meets at least one of the following criteria: high compression, good image quality, and moderate cost.SOLUTION: A video encoder (100) is configured to, in encoding a sequence of frames (video image stream 113), apply a linear transform (liner transform assembly 101), such as a wavelet transform (wavelet transform module 104), to the sequences of frames on a frame-by-frame basis. At least one portion of a linearly transformed frame thus obtained is encoded in the inter mode on the basis of a differential representation of the linearly transformed frame. The differential representation corresponds with a difference between, on the one hand, the portion of the linearly transformed frame and, on the other hand, a representation of a corresponding portion of at least one other linearly transformed frame. Data compression (quantization modules 106, 107) is then applied to the differential representation of the portion of the linearly transformed frame.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] An aspect of the invention relates to an encoder adapted to encode a sequence of frames. The encoder may, for example, encode a video stream in a manner that allows efficient transmission or storage or both. This can be used to encode the data for efficient processing. The present invention relates to a method for encoding a series of frames, a computer program for an encoder, a decoder, and and a method for decoding a sequence of encoded frames. [Background technology]

[0002] U.S. Pat. No. 9,332,258 states that compression of digital images typically involves decorrelation transformations. de-correlative transform, entropy coding and The decorrelation transform is performed in three steps: The entropy of the transformed image decreases as the probability of occurrence of the coefficients decreases. It can be lower because it increases in a small subset. The transformation can be a color transformation, an inter / intra prediction, a DCT transformation, or a wavelet transformation. Entropy coding reduces the data size of the converted image, resulting in less data Finally, the rate allocation allows you to achieve the desired compression ratio. To obtain this, data is selected that will be part of the output stream of compressed images.

[0003] U.S. Pat. No. 9,332,258 minimizes the complexity of the encoder and decoder. compressing an input data stream into an output data stream with acceptable compression efficiency while This paper describes a technique for compressing a sequence of m-bit words in an input data stream. Group into groups of n words of m bits, where n is greater than or equal to 2 For each group, find the maximum coding line index (GCLI) value. GCLI is the most significant bit among any bits of the words in the group, excluding the sign bit. is the index of the large non-zero bit. The output data stream is A group of n words of GCLI bits corresponding to n words of m bits in the group The GCLI bits in each word correspond to the The least significant GCLI bit of the word and the value of the GCLI. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] U.S. Patent No. 9,332,258 Summary of the Invention [Problem to be solved by the invention]

[0005] It is better suited to at least one of the following criteria: high compression, high image quality, and moderate cost. What is needed is an improved solution that can encode a series of overlapping frames. [Means for solving the problem]

[0006] According to an aspect of the invention as defined in claim 1, a coding system adapted to code a sequence of frames is provided. An encoder is provided, the encoder comprising: A series of linear transforms with less entropy than the frames in the series being coded Apply a linear transformation to a series of frames frame by frame to obtain a frame that matches the a linear translation assembly adapted to to obtain a sequence of coded frames with less data than the linearly transformed frames. a coding assembly adapted to code the sequence of linearly transformed frames; Equipped with The encoding assembly transmits at least one portion of the linearly transformed frame to the encoding assembly. The assembly is at least one part of a linearly transformed frame coded in inter mode. The differential representation is adapted to be encoded in an inter mode that provides a differential representation of the image, while At least one portion of the linearly transformed frame, and at least one other portion of the linearly transformed frame, corresponding to the difference between the representation of at least one portion of the linearly transformed frame and the encoding assembly is adapted to obtain at least one portion of the encoded frame. , applying data compression to the differential representation of at least one portion of the linearly transformed frame; A decoder is provided.

[0007] According to further aspects of the invention as set forth in claims 12, 13 and 14, a series of frames is A method for encoding, a computer program for an encoder and a decoder are respectively provided.

[0008] The main difference with traditional coding schemes is that in inter mode, linearly transformed frames are The differential representation of at least one part is a representation of the original rather than a differential representation of a frame or part thereof. This means that the data is processed in the form of hardware or software or Both allow for implementation with reduced complexity, resulting in lower cost and power consumption. Furthermore, by applying inter-mode coding, High compression can be achieved without significant degradation of image quality.

[0009] Even if we encode the difference between linearly transformed frames rather than the difference between the original frames, In principle, this does not adversely affect image quality. This is because the difference calculation after linear transformation is The difference expression obtained by the difference operation after the linear transformation is equivalent to the linear transformation. Applying the inverse results in the difference representation obtained by the difference operation before the linear transformation. Conversely, when a linear transformation is applied to the differential expression obtained by the differential operation before the linear transformation, the linear The difference expression obtained by the difference operation after the transformation will be obtained.

[0010] In the above-mentioned encoder, the encoding assembly performs any transformation that may be decorrelated. It should be noted that in such an encoder, it is not necessary to perform The transformed frame does not need to undergo any further transformations that may reduce entropy. Rather, the data compression applied to the differential representation as described above allows the differential representation to be represented with less data. To achieve this, entropy coding is applied directly to the differential representation.

[0011] For purposes of illustration, certain embodiments of the present invention will now be described in detail with reference to the accompanying drawings. In this description we present additional features, some of which are set out in the dependent claims. And the benefits will be clear. [Brief explanation of the drawings]

[0012] [Figure 1] FIG. 1 is a block diagram of a video encoder. [Figure 2]FIG. 2 is a frequency diagram illustrating the subbands defined by the video encoder. [Figure 3] FIG. 3 is a diagram of an image of a linearly transformed frame generated in a video encoder. [Figure 4] FIG. 4 is a block diagram of a coding mode selection module in a video encoder. [Figure 5] FIG. 5 is a conceptual diagram illustrating GCLI value extraction from a group of four samples in the coding mode selection module. [Figure 6] FIG. 6 is a diagram of an image of a linearly transformed frame showing parts coded in intra mode and other parts coded in inter mode. [Figure 7] FIG. 7 is a block diagram of a reference frame buffer assembly in a video encoder. [Figure 8] FIG. 8 is a block diagram of a video decoder. DETAILED DESCRIPTION OF THE INVENTION

[0013] FIG. 1 is a schematic diagram of a video encoder 100. 1 provides a block diagram of a video encoder 100. The video encoder 100 may, for example, capture video. It can record or store video images or both, for example, a smartphone. The video encoder 100 can be installed in any communication device. and efficiently transmitting the image to another communication device over a communication path that may be provided by a network. This can be done.

[0014] The video encoder 100 includes a linear transform assembly 101 and an encoding assembly 102. The linear transformation assembly 101 includes a reversible color transformation module 103 and a wavelet The encoding assembly 102 may include a conversion module 104. module 105, quantization module 106, entropy coding module 107, Data Packaging Module 108, Rate Allocation Module 109, and Inverse Quantization module 110, reconstruction module 111, and reference frame buffer assembly 11 2.

[0015] The video encoder 100 basically operates as follows: , receives a video stream 113, which is a series of frames In response, the video encoder 100 outputs an encoded video stream 114. However, the video stream 114 has a data rate higher than that of the received video stream 113. can be significantly smaller.

[0016] A frame of the received video stream 113 can be considered as a pixel matrix, The pixel matrix includes pixel rows and pixel columns. A pixel includes three components: The three components can be combined to provide luminance and color information for a pixel. The three components are expressed in a color coordinate system, for example, the color coordinate system known as RGB. It is expressed as

[0017] The reversible color conversion module 103 converts the three components of a pixel into three components of another color coordinate system. This reversible color transformation can reduce entropy. In response, the reversible color transformation module provides a reversibly color transformed frame, The entropy of the frame may be reduced compared to the original form. This is useful for image compression. tsu.

[0018] For convenience and simplicity, we will use the lossless color transformed frame as a pixel matrix. By this, a pixel is assumed to have only one component, for example, only the luminance component (Y). That is, the reversibly color-transformed frame is a single matrix of samples showing only a single component. On the other hand, the lossless color-transformed frame is actually a sample of three matrices. can be thought of as a set of risks, each of which represents a particular component. For example, The transformed frame is in fact a matrix of samples representing the luminance (Y) component, the first color component, Another matrix of samples representing the first color component (Cr) and another matrix of samples representing the second color component (Cb) These matrices of samples may be used in conjunction with individual samples, as described below. The samples can be processed separately. The samples can be put into the form of binary words that represent the values ​​of the components. This can be done.

[0019] The wavelet transform module 104 converts the wavelet transform into a reversible color transformed frame. The wavelet transform requires filtering and subsampling. These processes are necessary to obtain the linearly transformed frame. A set of subband frames is generated. A subband frame is a set of subband frames in a specific spectral band. The spectral components of the frame are reversibly color transformed using the spectral band. The wavelet transform module 104 performs the wavelet transformation. The Lett transform is a powerful alternative to existing image coding schemes, such as the one known as JPEG-XS. The transformations can be similar to those in wavelet-based image coding schemes. Cut.

[0020] FIG. 2 shows the subbands 201 to 207 defined by the wavelet transform module 104. Each subband 201 to 207 is shown in a two-dimensional frequency diagram. The frequency diagram shows the horizontal frequency F H The horizontal axis indicates the frequency F V Show The wavelet transform module 104 therefore calculates each of these sub-axes. Each subband frame is provided for each subband 201 to 207. The subband coefficients for the reversible color transformed frame can be considered as a matrix of subband coefficients. The size of the matrix of subband coefficients is determined by the wavelet transform module 104. It depends on the sub-sampling factor applied to obtain the band frame.

[0021] FIG. 3 shows subband frames 301 to 302 that jointly constitute a linearly transformed frame. Each of the subband frames 301 to 307 is an image diagram. A subband frame is associated with a particular subband as shown in FIG. FIG. 3 shows a specific subband with a reference signature. gn) to this particular subband, the subband frame is associated. indicates the specific subband selected.

[0022] The linearly transformed frame is divided into four relatively small, low frequency subband frames 30 1 to 304 and three relatively large, high-frequency subband frames 305 to 307. The three high frequency subband frames 305 to 307 are subsampled. The ringing factor is 2. For the four low frequency subband frames 301 to 304, The subsampling factor is 4. Four low frequency subband frames 301 to 304 In the image, the lossless color-transformed frame is processed by low-pass filtering and downscaled. There is a corresponding lowest subband frame 301. Filtering is equivalent to subsampling.

[0023] As can be seen from Figure 3, the subband frame contains both spectral and spatial information. This property is demonstrated in the video coder 100 shown in FIG. That is, a specific part of the subband frame is converted into a reversible color transformed frame. The image can be associated with a similarly positioned portion of the frame being coded.

[0024] Referring to FIG. 1, the encoding assembly 102 encodes each of the subband frames 301 to 307. However, when combining compressed subband frames, The coding mode selection module can be used to select the data package. module 105, quantization module 106, dequantization module 110, and entropy The encoding module 107 processes each subband frame 301-307 individually in succession. In another embodiment, the encoding assembly 102 may be configured to process the data in a parallel manner. In one such embodiment, the encoding assembly 102 is operable to a coding mode selection module, a quantization module, and a dequantization module; Each subband filter may include a filter and an entropy coding module. One such module for a frame can be included in the linearly transformed frame. We will now describe the coding of a single subband frame for simplicity and convenience. Sub-band frames are coded in a similar manner.

[0025] To encode a subband frame, the reference frame buffer assembly 112 A corresponding subframe associated with another frame in the previously encoded video stream 113 The term corresponding includes the representation of both sub-band frames. This indicates that the previously coded frames are associated with the same subband. The representation of the corresponding subband frame in relation to the frame will be hereafter referred to as We call the linearly transformed frame the corresponding subband frame. To encode, the reference frame buffer assembly 112 receives a previously encoded reference frame from another Both of the above linearly transformed frames may include a representation of a linearly transformed frame of It includes the subband frames 301 to 307 shown in FIG.

[0026] The encoding assembly 102 can encode the subband frame in parts. The portion of the subband frame to be coded consists of groups of 32 samples on the same line. The subsampling factor, which in this embodiment can be 2 or 4, allows for subbanding. Such a portion of the frame is divided into two samples and then split into six parts. 4 blocks and 4 samples correspond to 128 blocks. The portion of the subband frame to be coded is hereinafter referred to as the subband frame for the sake of brevity and simplicity. This is called the frame part.

[0027] To encode the subband frame portion, the encoding mode selection module 105 From the reference frame buffer assembly 112, the corresponding previous subband frame In this embodiment, the corresponding part is searched for in the subband frame to be coded. The position of the corresponding subband portion in the corresponding previous subband frame is the same as the position of the corresponding previous subband portion in the corresponding previous subband frame. Corresponding portions of subband frames will hereafter be referred to as corresponding subband frames for the sake of brevity and simplicity. This is called the subband frame portion.

[0028] FIG. 4 is a schematic diagram illustrating the coding mode selection module 105 in more detail. 4 provides a block diagram of the coding mode selection module 105. The module 105 includes a comparator 401 and two maximum coding line indices (g Realest coded line index) extractor 402, 403, a data buffer 404, a coding mode decision unit 405, and a multiplexer 406; One of the two maximum coding line index extractors is hereafter referred to as Therefore, it is called the Intermode GCLI Extractor 402. The other one is the intra-mode GCLI extractor. It's called 403.

[0029] The coding mode selection module 105 basically operates as follows: 401 provides a differential representation of the subband frame portion to be coded. the difference between the subband frame portion to be coded and the corresponding previous subband frame portion The difference corresponds to the frame buffer assembly. The multiplier 406 divides the subband frame portion to be coded at one input and the subband frame portion to be coded at another input. and receiving a differential representation between the subband frame portion of itself.

[0030] The encoding mode selection module 105 selects whether the encoding assembly 102 is an intra-modulation encoding mode. The subband frame portions are coded in either intra-mode or inter-mode. In this mode, the multiplexer 406 multiplexes the subband frames present in the linearly transformed frame. The part is passed to the quantization module 106 shown in FIG. The subband frame portion present in the linearly transformed frame is coded, data compression by the quantization module 106 and the entropy coding module 107 In inter mode, the multiplexer 406 receives the subband frame portion The differential representation is passed to the quantizer. In inter mode, the subband frame part is The differential representation of is coded, which also involves a quantization module 106 and an entropy Data compression by the encoding module 107 is required.

[0031] The main difference with traditional coding schemes is that in inter mode, linearly transformed frames are The differential representation of at least one part is a representation of the original rather than a differential representation of a frame or part thereof. Accordingly, the coding scheme according to the present disclosure is a linear transformation. whereas there is no need to apply a linear inverse transform to at least one part of the transformed frame. Conventional coding schemes require the application of a linear inverse transform. Such coding schemes may be used to encode data related to hardware or software or both. This allows for less complex implementations, which in turn allows for lower cost and power consumption implementations. This becomes possible.

[0032] Rather than encoding the differences between the original frames, it is better to encode the differences between the transformed frames. In principle, this has no adverse effect on image quality. This is because the bit transformations are linear. , which suggests that the difference operations are equivalent to those before these transformations. Applying the inverse of these transformations to the difference representation obtained by performing the Conversely, if the difference is Applying the transformations associated with the difference representations obtained by performing the operations allows these transformations A difference expression is then obtained by performing a difference operation.

[0033] The coding mode selection module 105 selects the intra-mode or inter-mode as follows: Determines whether to encode the subband frame portion in the inter-mode GCLI error. The subtractor 402 extracts the four samples contained in the differential representation of the subband frame portion. We provide a set of eight GCLI values ​​associated with eight groups. These GCLI values ​​are For simplicity, this will be referred to as the inter-mode GCLI value. The tractor 403 also calculates the four samples included in the subband frame portion to be coded. We provide a set of eight GCLI values ​​related to eight groups of For simplicity, this will be referred to as the intra-mode GCLI value. The extractor 402 and the intra-mode GCLI extractor 403 are, respectively, It can be operated as described in Patent No. 9,332,258. The GCLI extract is briefly outlined below.

[0034] Figure 5 shows a schematic diagram of the extraction of GCLI values ​​from a group of four samples 501-504. FIG. 5 shows this extraction, which represents four samples 501-504 in the form of a binary word. In this example, the binary words are layered multiples that define a value. Each binary word may further include a sign bit. The sign bit is not shown in FIG. 5 for simplicity.

[0035] In the layered multiple bits, each bit is M, ... 5, 4, 3, 2, 1 M is the number of bits included in the layered bits. The bit with the highest hierarchy level M is usually referred to as the most significant bit. The bit with the lowest hierarchy level 1 is usually referred to as the least significant bit.

[0036] In a group of four samples, bits with the same hierarchical position are called bits A group of four samples constitutes a plane. Each bit plane contains as many bits as there are in the bit plane. The most significant bit has a hierarchical rank corresponding to the hierarchical rank of the bits contained in the lane. The most significant bit plane is the highest hierarchical level M, while , the least significant bit constitutes the least significant bit plane, and the least significant bit plane is It is at hierarchy level 1.

[0037] The extraction of GCLI values ​​is performed by dividing the bit planes into the most significant bit plane and the least significant bit plane. The purpose of GC is to determine whether a GC contains at least one non-zero bit towards the top plane. The LI value indicates the hierarchical level of this bit plane. For example, the GCLI value is an integer value of 1, which indicates the lowest level. The integer value M indicates the hierarchical order indicating the most significant bit plane, and the integer value M indicates the hierarchical order indicating the most significant bit plane. We can accommodate this.

[0038] In the example shown in FIG. 5, bit planes M through 5 each contain only zero bits. Such bit planes contain only zero bits and will henceforth be referred to as , referred to as the zero bit-plane. In the embodiment shown in FIG. 5, bit-plane 4 is at least Both are the highest hierarchical bit planes containing one non-zero bit. The value can therefore be 4.

[0039] The buffer stores the intermodulation data based on a differential representation of the subband frame portion to be coded. The buffer temporarily stores the eight GCLI values ​​provided by the GCLI extractor 402. The subband frame to be coded by the intra-mode GCLI extractor is also Temporarily store eight GCLI values ​​to provide based on the part.

[0040] The coding mode decision unit 405 determines whether the inter-mode GCLI extractor 402 Calculate the sum of the eight GCLI values ​​provided. This sum is hereafter referred to as the Intermode GCL The coding mode decision unit 405 also determines the intra-mode GCLI output. Calculate the sum of the eight GCLI values ​​provided by the constructor 403. This sum is then used as the index. This is called the intra-mode GCLI total value.

[0041] The total intra-mode GCLI value is the sum of the subband frame parts coded in intra mode. The intermode GCLI sum indicates the data compression factor that can be achieved when The data compression factor that can be achieved when the band frame portion is coded in inter mode is In fact, each of the GCLI sums above represents an estimate of the data compression factor.

[0042] The coding mode selection module 105 selects whether the intra-mode GCLI total value is greater than the inter-mode GCLI total value. If the sum of the GCLI values ​​is less than or equal to the subband frame rate, the For that purpose, the coding mode decision unit 405 encodes the multi-frame portion. The multiplier 406 divides the subband frame portion to be coded into the coding mode selection module 1. 05 to the output of the multiplexer 406. In this mode, the subband frame portion is processed by the quantization module 106, the entropy coding module The data is further processed by the data processing module 107 and the data packaging module 108.

[0043] The coding mode selection module 105 selects whether the inter-mode GCLI total value is greater than the intra-mode GCLI total value. If the total GCLI value is smaller than the subband frequency, the The intra mode can be coded by the coding mode decision unit 4. 05, the corresponding subband frame portion of a predetermined number K is all coded in inter mode. If it is determined that the sub-band is encrypted, it is selected. K represents an integer value. The end frame part is forced to be coded in intra mode, while the will be coded in inter mode. The determination unit 405 determines at least the number of subbands in the corresponding subband frame portion of a series of K+1. It ensures that at least one subband frame portion is coded in intra mode. Therefore, for K+1, which is greater than 1, the corresponding subband frame part is coded in intra mode. This can be taken to represent the minimum frequency that can be

[0044] The coding mode selection module 105 selects whether the inter-mode GCLI total value is greater than the intra-mode GCLI total value. The corresponding previous subband frame is smaller than the total GCLI value and is less than K. If the subband frame is coded in the inter mode, the subband frame portion is coded in the inter mode. In this case, the coding mode decision unit 405 determines whether the multiplexer 406 is The differential representation of the band frame part is passed to the coding mode selection module 105. In inter mode, the differential representation is controlled by the quantization model. module 106, entropy coding module 107 and data packaging module The data is further processed by filter 108.

[0045] The coding mode selection module 105 therefore determines the inter-mode GCLI sum. The indicated data compression factor is greater than the compression factor indicated by the intra-mode GCLI total value. If the above conditions are applied, the subband frame part is evaluated as follows. The coding mode selection module 105 also selects the inter-mode. Regardless of the frequency, the intra-mode coding is applied to guarantee a certain minimum frequency. It may be decided to force the coding of a subband frame part in intra mode. If the above conditions do not apply, the subband frame portion is coded in intra mode. will be done.

[0046] The encoding mode selection module 105 thus allows encoding of several types of output data. The coding mode selection module 105 provides the subband frame portion to be compressed. The subband frame data is subjected to linear transformation. a subband frame portion or a difference table of subband frame portions present in the frame; Either the intra mode or the inter mode is selected. Furthermore, the coding mode selection module 105 determines whether the coding mode flag The flag indicates whether the subband frame portion is an intra subband frame that provides an inter mode. The GCLI coding mode selection module 105 indicates the coding mode. further provides a GCLI encoding value, which may be an intra-mode GCLI value or If the subband frame portion is one of the intermode GCLI values The difference depends on whether the image is coded in standard or intermode, respectively.

[0047] FIG. 6 illustrates a schematic representation of the same linearly transformed frame as illustrated in FIG. In this figure, a portion of the subband frame is coded in intra mode, and other subbands are coded in intra mode. The band frame portion is coded in inter mode. The subband frame portion is shown by a rectangle with a darker color tone. The portions of the subband frame coded in the code are indicated by rectangles with a lighter shade. There are.

[0048] Figure 6 clearly illustrates that the subband frames are coded separately. , the first subband frame portion of the lowest subband frame 301 is while another low frequency subband frame 302, 3 The first subband frame portion of 03, 304 can be coded in intra mode. Cut.

[0049] As mentioned above, the coding mode decision unit 405 performs the coding on a series of linearly transformed frames. In this case, the corresponding subband frame portions are periodically coded in intra mode. This can be seen as a refresh mechanism, which ensures that the decryption On the other hand, a relatively good image quality can be achieved quickly. The video encoder 100 shown in the figure provides an encoded video stream 114, which is called This also allows for more stable automatic tracking. If a part of the encoded video stream 114 is ,Even if lost due to a transmission error, the affected frame area can be recovered relatively quickly. It can be done.

[0050] In a series of corresponding subband frames that all relate to the same subband, In other words, a part of the subband frame that is coded in intra mode is forced to be coded in intra mode. The position may differ from that of a portion of the preceding subband frame coded in the same code. For example, referring to FIG. 6, the first subband frame portion of the lowest subband frame Assume that minute 301 is forced to be coded in intra mode. In that case, the following minimum In the subband frame of the region, a second subband frame portion is arranged after the first subband frame portion. The subband frame portion of the following subband frame may be forced to be coded in intra mode. In the lowest subband frame, it is placed after the second subband frame portion. A third subband frame portion may be forced to be coded in intra mode, etc. This is because subband frame parts are periodically coded in intra mode. Such a scheme can be considered as a shift or phase shift scheme. The video decoder can then quickly generate the encoded video stream 114. It helps to maintain automatic tracking.

[0051] The minimum frequency at which a subband frame part is forced to be coded in intra mode is These subband frame portions are determined by the subband frame to which they belong. , the minimum frequency is the frequency for the high frequency subband frames 305 to 307. The frequency for the low frequency subband frames 301 to 304 is higher than the frequency That is, the number K+1 above, which represents the reciprocal of the minimum frequency, may be The higher frequency sub-band frame 30 is 5-307 may be higher. This may affect the effectiveness of the compression. A low-resolution video representation of acceptable quality on the decoding side and a high-resolution representation a little later without significant loss of quality. For example, in one embodiment, a low-resolution representation of the motion image can be obtained quickly. is obtained after 4 frames, while a high-resolution representation is obtained after 32 frames. can be obtained.

[0052] Referring again to FIG. 1, the quantization module 106 is connected to the coding mode selection module 10. 5 provides the sub-band frame data. Quantization Module 106 provides quantized subband frame data. applies a quantization coefficient when quantizing subband frame data. If relatively high, the coarseness provides relatively high data compression for the subband frame data. Conversely, if the quantization coefficients are relatively low, the subband frame data is Fine quantization is performed, which provides relatively low data compression. The quantization factor is the rate at which the The video stream 11 is coded by the matching module 109 while optimizing the image quality. 4 can be adjusted to have the desired data rate.

[0053] The entropy coding module 107 performs the entropy coding process on the quantized sub-pixels. The entropy coding process is applied to the band frame data, e.g., as already mentioned above. This may include the process described in U.S. Pat. No. 9,332,258. The P coding module 107 converts the GCLI coded value into the coding mode selection module described above. The GCLI coded value is received from 105 in the quantized subband frame data. The entropy coding module 107 then divides these In practice, the zero bit planes are removed from the sub-pixels. The GCLI coded values ​​and positions that indicate where these bit planes are located in the band frame data. The entropy coding module 107 also performs entropy coding on the GC Apply LI encoding to values ​​so that these values ​​are represented using as little data as possible. Patent No. 9,332,258 describes an entropy code that can be used to that effect. This document describes encryption techniques.

[0054] In addition to removing zero bit planes, the entropy coding module 107 Also, truncating samples in the quantized subband frame data Such truncation can involve removing one or more least significant bitplanes. This is also described in U.S. Pat. No. 9,332,258. The rate allocation module 109 adjusts for truncation and allocates one or more Determine whether to remove least significant bitplanes, and if so, how many The rate allocation module 109 allocates the coded video stream 114 to the To achieve the desired data rate while optimizing the quality, truncation is performed. You can adjust the brightness.

[0055] The data packaging module 108 packages some type of data into an encoded motion image. The image stream 114 is packaged in the entropy coding module 10. 7 includes the entropy-encoded subband frame data. Further, for example, the entropy-encoded GCLI value, the encoding mode flag, and the quantization and the quantization coefficients applied by the quantization module 106.

[0056] The rate allocation module 109 performs the quantization and entropy coding. The module 107 is configured to select the GCLI coding value provided by the coding mode selection module 105. The GCLI coding value allows you to control only the zero bit planes. The data compression factor obtained by removing the The module 109 applies this data compression factor to the desired encoded video stream 114. This can be compared to the desired data compression factor, which represents the data rate of the If the difference between the aforementioned data compression factors is relatively large, the rate allocation module 109 Should a relatively large quantization factor be applied or should a relatively large truncation be applied? It can be decided whether to apply either the standard or both.

[0057] As mentioned above, the reference frame buffer assembly 112 stores previously encoded This representation may include a linearly transformed representation of the frame that is now being encoded. Forming a basis for inter-mode coding of a subband frame portion of a frame The representation of the previously linearly transformed frame is obtained as follows:

[0058] The inverse quantization module 110 uses the quantized sub-quantizers provided by the quantization module 106. Inverse quantization is applied to the end frame data. Accordingly, the inverse quantization module 110 , a coarser representation of the subband frame data generated by the coding mode selection module 105. provides dequantized subband frame data that can be considered as a representation. In the case of linear mode coding, the dequantized subband frame data is linearly transformed. It is a more coarse representation of the portion of a subband frame that exists in a frame. In the case of encoding, the dequantized subband frame data is This is a more coarse representation of the difference.

[0059] When encoding in inter mode, the reconstruction module 111 reconstructs the dequantized sub-bands. The reconstruction module 111 processes the subframe data. The frame buffer assembly 112 retrieves the corresponding frame. The destination subband frame is dequantized after the dequantized subband frame data is generated. It is used to generate a differential representation of a subband frame portion based on the The synthesis module 111 then converts the corresponding previous subband frame portion into a dequantized The subband frame data is then added to the encoded subband frame data. The reconstruction module 111 obtains a reconstructed version of the subband frame part. The reconstructed version is applied to the reference frame buffer assembly 112 .

[0060] The reconstruction module 111, when encoding in intra mode, The frame buffer assembly 112 directly passes the frame data to the reference frame buffer assembly 112. In this case, the dequantized subband frame data is used to reconstruct the subband frame part. Compose the edition.

[0061] Thus, during the encoding process, the reference frame buffer assembly 112 is reconstructed. a plurality of reconstructed subband frame portions that can jointly form a reconstructed subband frame; More generally, the reference frame buffer assembly 112 receives the lines The reconstructed subband frames jointly form a reconstructed version of the transformed frame. Ideally, the linearly transformed reconstructed frames are This corresponds to the decoded version of the linearly transformed frame in the video decoder of the embodiment. In general, the standard for encoding data is to ideally ensure that the same data can be decoded. Therefore, the criteria used to decrypt the data should correspond to the criteria used to decrypt the data. If the criterion consists of previously decoded data, the criterion used to encode that data is It should be a decoded version of previously encoded data rather than the data in its original form.

[0062] FIG. 7 illustrates a schematic of the reference frame buffer assembly 112. 1 provides a block diagram of the reference frame buffer assembly 112. The base assembly 112 includes a base entropy coder 701, a buffer memory 702, a base It includes an entropy decoder 703 .

[0063] The reference frame buffer assembly 112 basically operates as follows: The tropy coder 701 applies GCLI extraction to the reconstructed version of the subband frame portion. This GCLI extraction can be similar to that described above in connection with Figure 5. The entropy coder 701 removes zero bit planes in each group of samples. The base entropy coder 701 further encodes one or more least significant bit planes as The sample can be truncated by removing the zeros. There may be one or more most significant bits missing and one or more truncated least significant bits missing. A compressed sample is obtained.

[0064] The basic entropy coder 701 generates a compressed reconstruction of the subband frame portion. The compressed reconstruction includes compressed samples. It may further include GCLI values ​​obtained by GCLI extraction, as discussed above in this specification. The compressed reconstruction may include the GCLI values ​​in entropy-encoded form.

[0065] The buffer memory 702 stores the compressed and reconstructed version of the subband frame portion. In response, during the encoding process, the buffer memory 702 stores the compressed and reconstructed sub-bands. The compressed and reconstructed subband frame portions can be concatenated to form a single subband frame. More generally, the reference frame buffer assembly 112 stores the linearly transformed compressed and reconstructed subband frames that jointly form a compressed and reconstructed version of the converted frame. The programs are memorized consecutively.

[0066] The basic entropy decoder 703 restores the compressed and reconstructed subband frame portion. , the subband frame portion is intermodulated with the corresponding subsequent subband frame portion. In this way, the corresponding subband frame may be required to be coded. The above expression of the game is obtained.

[0067] The video encoder 100 shown in FIG. 1 therefore applies frame buffer compression. Correspondingly, subband frames, and more generally, inter-mode coding, A relatively small amount of memory is sufficient to store the linearly transformed frames required for Furthermore, frame buffer compression allows the reconstructed subband frames to be The bandwidth requirements associated with transmitting and receiving data relating to the portion to the buffer memory 702 are reduced. This means that the buffer memory 702 is in the form of an external memory circuit, while the image Embodiments where the other entities of the encoder 100 are integrated into a single processing circuit In this and other embodiments, the mitigation A limited bandwidth requirement generally lends itself to relatively low power consumption.

[0068] However, compressing the frame buffer may affect image quality to some extent. is the corresponding previous subband frame that is available for coding in inter mode. The representation of each subband frame is a coarser version of the corresponding subband frame itself. Nevertheless, in many embodiments, the loss of image quality is barely noticeable visually. It is possible to configure frame buffer compression so that it is imperceptible or completely visually unnoticeable. That is, the frame buffer compression can be visually lossless. Moreover, in principle, frame buffer compression only reaches sufficient image quality after decoding begins. This does not affect the delay between

[0069] FIG. 8 shows an outline of a video decoder, which is shown in block diagram form. The video decoder uses the video generated by the video encoder 100 described above in connection with FIGS. The encoded video stream 114 can be received and decoded. The device includes an unpacking module 801, an entropy decoding and dequantization module 802, and a decoding module 803. Signal reconstruction module 803, decoding buffer memory 804, inverse wavelet transform and color transform It includes a module 805.

[0070] The video decoder basically operates as follows: An unpacking module 801 It performs the inverse of the process performed by the pack module described above. The packing module 801 extracts from the coded video stream 114 a Encoded subband frame data, entropy-encoded GCLI values ​​and codes It retrieves various types of data, such as encryption mode flags.

[0071] The entropy decoding and dequantization module 802 is the same as the entropy coding module described above. For example, the entropy decoding and inverse quantization module 107 performs the inverse process of the process performed by the module 107. Module 802 searches for the GCLI coded value from the entropy coded GCLI value. The entropy decoding and dequantization module 802 then removes The zero bit planes can be restored based on these GCLI coded values. The tropy decoding and dequantization module 802 further adds dummy bits to dequantize the transcoded data. The least significant bits removed as a result of the application can be replaced accordingly. The quantized sub-band frame data is reconstructed and then subjected to an inverse quantization process. The subprocess is based on the quantized coefficients packed in the coded video stream 114. In response, the entropy decoding and dequantization module 106 Provides frame data.

[0072] The decoding reconstruction module 803 reconstructs the decoded subbands based on the coding mode flag. Processes frame data. If the encoding mode flag indicates intra-mode encoding, The decoded sub-band frame data can be further processed into a linearly transformed frame. is a decoded version of the subband frame portion that can be used directly to reconstruct the subband frame without In that case, the decoding reconstruction module 803 temporarily stores the decoded version of the subband frame portion. The decoding buffer memory 804 is used for storing the decoded version of the sub-band frame portion. is applied to the inverse wavelet transform and color transform module 805.

[0073] If the coding mode flag indicates inter-mode coding, the decoded subband frame is The frame data is a decoded version of the differential representation of the subband frame portion. The construction module 803 constructs a corresponding previously decoded subband frame portion in a decoding buffer. The decoding and reconstruction module 803 then retrieves the corresponding previously decoded The subband frame portion is added to the decoded version of the differential representation of the subband frame portion. In response, a decoded version of the subband frame portion is obtained and stored in the decoding buffer memory 804 and and the inverse wavelet transform and color transform module 805.

[0074] Therefore, contrary to the conventional video decoder, the decoding buffer memory 804 is in the spatial domain. Subbands that are part of a linearly transformed frame, rather than a fully decoded frame. That is, in the moving image decoder 800 shown in FIG. , the linearly transformed frame is more likely to be an inter-mode frame than a fully decoded frame. It serves as a reference for decoding.

[0075] The inverse wavelet transform and color transform module 805 performs the inverse and reversible wavelet transform. The inverse of the color transform is applied to the decoded version of the subband frame portion. The inverse linear transform is applied to the decoded version of the subband frame part. By applying it to successively decoded versions, the linear inverse transform is is applied to the decoded version of the subband frame that constitutes the decoded version of A decoded version of the frame encoded by the video encoder 100 is obtained.

[0076] Things to keep in mind The embodiments described above with reference to the drawings are presented for illustrative purposes. To illustrate this, some alternatives are briefly shown.

[0077] The present invention relates to many types of products related to video compression and / or video restoration. For example, the present invention can be applied to at least one method other than video compression. This is applicable to multi-function integrated circuits that can perform the functions of:

[0078] There are many different ways to implement a video encoder and video decoder in accordance with the present invention. Any of the modules in the presented embodiments may be dedicated or programmable. by an electrical circuit or by a suitably programmed processor, The computer program may be implemented by a video encoder. The video encoder can define one or more processes to be executed by the following: 7. Similarly, the computer program may be 8. The video encoder may define one or more processes to be performed by the video encoder. In this regard, the block diagrams of Figures 1, 4, 7 and 8 are respectively and, at least in part, flowchart diagrams of such computer programs, It is considered to represent the method that a processor can perform when executing a computer program. For example, the reversible color transformation module shown in FIG. Similarly, other modules can be thought of as representing steps in a method. It's okay to do so.

[0079] There are many different ways to implement linear transformations in products or methods according to the present invention. In some embodiments, the linear transformation includes a reversible color transformation. In other embodiments, this transformation can be omitted. In the embodiment presented, the linear transform comprises a wavelet transform. In other embodiments, the linear transform may be, for example, a discrete cosine-based transform.

[0080] There are many different ways to implement the encoding of compressed data in the products or methods of the present invention. In the presented embodiment, such encoding includes GCLI extraction, zero bit planes, In other embodiments, other entropy coding methods are used. A scheme may be used.

[0081] Implementing a reference frame buffer assembly in a product according to the invention or a method according to the invention There are many different ways to implement this. In the embodiment presented, the reference frame buffer assembly In other embodiments, the reference frame buffer adapter uses compression of the frame buffer. The assembly must be equipped with a video encoder for data compression and a video decoder for data compression. Frame buffer compression may be omitted so that

[0082] To encode in inter mode, we use a representation of the corresponding part of another linearly transformed frame. In the embodiment presented, the corresponding parts are first linearly transformed. The parts that have the same position in the frames are the same, i.e., no motion estimation or compensation is used. In other embodiments, motion estimation may be used to identify corresponding portions. The corresponding part used for encoding in inter mode is not necessarily linearly transformed first. It is not necessary that the corresponding parts are based on a single frame. The frame rate may be based on a combination of parts in two or more frames that have been converted.

[0083] In general, there are many different ways to implement the invention, and the different implementations may involve different methods. In any given topology, a single entity may have several or several entities can jointly perform a single function. In this respect the drawing is very schematic.

[0084] The embodiments described in connection with the drawings are intended to illustrate rather than limit the invention. The invention may be embodied in many alternative ways within the scope of the appended claims. All changes that come within the meaning and range of equivalency of the claims are to be embraced within their scope. Any reference sign in a claim shall not be construed as limiting that claim. The verb "comprise" in the claims should not be interpreted as meaning that The same does not exclude the presence of other elements or steps than those listed in a claim. , and similar verbs such as "include" and "contain." A reference to a singular element in a product claim does not imply that the product is a plural of that element. Similarly, the use of the singular "a" in a method claim does not exclude the inclusion of such elements. The reference to a step does not exclude that the method may include a plurality of such steps. The mere fact that each dependent claim defines additional features does not necessarily mean that the features reflected in that claim are not included in the invention. This does not exclude combinations of additional features other than the above.

Claims

1. An encoder (100) adapted to encode a sequence of frames, the encoding The generator is A series of linear frames with less entropy than the frames in the series to be coded. applying a linear transformation to the series of frames frame by frame to obtain a transformed frame; a linear translation assembly (101) adapted to A series of encoded frames having less data than the linearly transformed frames are obtained. a coding assembly adapted to code the series of linearly transformed frames so as to and a casing (102), The encoding assembly is adapted to convert the linearly transformed frame to obtain an encoded frame. and encoding at least one portion of the frame in inter mode, and and adapted to encode at least one other part of the stream in intra mode, Thus, in the inter mode, the encoding assembly providing a differential representation of a portion of the linearly transformed frame, the differential representation being and, on the other hand, a representation of the corresponding portion of at least one other linearly transformed frame. and the encoding assembly performs data compression on the linearly transformed frames. and applying the differential representation of the portion of the frame to the coding algorithm in the intra mode. The assembly applies the data compression directly to another portion of the same linearly transformed frame. An encoder that does this.

2. The encoding assembly (102) performs the following for the linearly transformed portion of the frame: Condition: if the linearly transformed frame portion is coded in the inter mode, and encoding the linearly transformed frame in intra mode. A coded version of the linearly transformed portion of the frame is obtained that contains less data than the coded version. and the encoding assembly is adapted to evaluate whether the condition that If the above condition is met, the linearly transformed frame portion is coded in the inter mode. If the condition is not met, the linearly transformed frame portion in the intra mode is converted to the 2. The encoder of claim 1 adapted to encode minutes.

3. The encoding assembly (102) encodes the at least one other linearly transformed frame. at least one encoded frame obtained by encoding the corresponding portion of the frame; applying the inverse of said data compression to corresponding portions of said frames to generate said at least one and adapted to obtain a representation of said corresponding portion of another linearly transformed frame of 3. An encoder according to any one of claims 1 and 2.

4. The encoding assembly (102) encodes each linearly transformed frame portion into the intra-module. An encoder according to any one of claims 1 to 3, adapted to encode periodically in a mode.

5. The encoding assembly (102) encodes each linear variable having the same position with a minimum frequency. The converted frame portion is periodically coded in the intra mode, and the same minimum frequency , but the above-mentioned linearly transformed frame portions have different equivalent positions. Each linearly transformed frame portion is phase-shifted relative to the encoding of the linear mode.

5. The encoder of claim 4, adapted to encode periodically in a linear mode.

6. The encoding assembly (102) encodes a group of linearly transformed frames with minimum frequencies. Each linearly transformed frame portion belonging to a loop is periodically encoded in the intra mode. , each linear belonging to another group of linearly transformed frame parts with different minimum frequencies A request adapted to periodically encode the transformed frame portions in said intra mode. The encoder according to any one of claims 4 and 5.

7. The linear transformation assembly (101) is adapted to define each subband, Therefore, the linearly transformed frame portions of one group and the other group are different.

7. The encoder of claim 6 associated with a subband.

8. The data compression applied by the encoding assembly (102) is An encoder according to any one of claims 1 to 7, comprising a DEX encoding process.

9. The data compression applied by the encoding assembly (102) includes a quantization process. Item 9. The encoder according to any one of items 1 to 8.

10. The encoding assembly (102) encodes the signal to obtain a decoded linearly transformed frame. and the encoding assembly is adapted to decode the decoded frames. and compressing the decoded linearly transformed frame to obtain a compressed version of the linearly transformed frame. and the encoding assembly is adapted to apply compression to the decoded linear transform frame. adapted to store a compressed version of said code in a frame buffer memory (702); The encoding assembly includes at least one linearly transformed frame to be encoded in the inter mode. At least one linearly transformed frame is used to provide a differential representation of the corresponding version. so as to obtain a reconstructed version of the decoded linearly transformed frame from which a representation of one part is obtained. , applying the inverse of the data storage compression to a compressed version of the decoded linearly transformed frame. An encoder according to any one of claims 1 to 9, adapted to:

11. The encoding assembly (102) applies the decoded linear transform to the decoded linear transform frame.

11. The method of claim 10, wherein the compression of the data storage comprises a maximum coding line index coding process. encoder.

12. 1. A method of encoding a sequence of frames, the method comprising: a series of linear transforms with lower entropy than a frame in the series of frames to be encoded applying a linear transformation to the sequence of frames frame by frame to obtain a set of frames a linear transformation step; The series of linearly transformed frames are then converted into a series of linearly transformed frames each having a larger amount of data than the series of linearly transformed frames. an encoding step for encoding to obtain a reduced sequence of encoded frames; In the encoding step, the linearly transformed frame is obtained to obtain an encoded frame. The same linear coding method is used for coding at least one part of a frame in inter mode. At least one other portion of the transformed frame is coded in inter mode, to obtain, in the inter mode, a differential representation of the linearly transformed portion of the frame, The differential representation comprises, on the one hand, the linearly transformed portion of the frame and, on the other hand, at least one corresponding to the difference between the representation of the corresponding portion of another linearly transformed frame, , applying data compression to the differential representation of the linearly transformed portion of the frame, thereby In the intra mode, the data compression is performed on different parts of the same linearly transformed frame. Directly applied encoding method.

13. 13. Instructions for causing the encoder to perform the method of claim 12. A computer program for an encoder (100) comprising a set of options.

14. The sequence of encoded frames obtained by the encoder according to any one of claims 1 to 11. A decoder (800) adapted to decode a frame, the decoder comprising a linear transformation and encoding the inter-mode signal to obtain at least one decoded portion of the frame. and decoding at least a portion of the encoded frame to obtain the same linearly transformed frame. and obtaining at least one other decoded portion of the program in the same intra mode. a decoder adapted to decode at least one other portion of the encoded frame ( ８００）。

Images