Method, bitstream, apparatus, computer program and computer readable medium

Abstract

The present invention provides a method 1200. A configuration block 1210 including a configuration value 1220 is obtained. The configuration block 1210 may be used to reconstruct a first representation of an image at a first quality level using a second representation of the image at the first quality level in a hierarchical hierarchy having a plurality of quality levels. The second representation is based on a representation of the image at a second lower quality level in the hierarchical hierarchy. An extended configuration block 1240 including an extended configuration value 1250 is obtained. The configuration value 1220 and / or the extended configuration value 1250 may be used by a decoder to reconstruct the first representation of the image at the first quality level.

Description

Technical Field

[0001] The present invention relates to methods, apparatus, computer programs and computer readable media. In particular, but not exclusively, the present invention relates to methods, apparatus, computer programs and computer readable media for use in the processing of configuration information relating to residual data that can be used to reconstruct a representation of an image at a relatively high quality level. Background Art

[0002] Compression and decompression of signals is a consideration in many known systems. Many types of signals, such as video, can be compressed and encoded for transmission, such as via a data communications network. When decoding such a signal, it may be desirable to increase the quality level of the signal and / or to recover as much information as possible contained in the original signal.

[0003] Some known systems employ scalable coding techniques. Scalable coding involves encoding a signal together with information to allow reconstruction of the signal at one or more different quality levels, depending on, for example, the capabilities of the decoder and the available bandwidth.

[0004] There are several considerations regarding the reconstruction of signals in scalable coding systems. One such consideration is the amount of information stored, used, and / or transmitted. The amount of information may vary, for example, depending on the desired quality level of the reconstructed signal, the nature of the information used in the reconstruction, and / or how such information is configured.

[0005] Another consideration is the ability of the decoder to accurately and / or reliably process the received information. One factor that may affect the reliability of the decoder is the ability of the decoder to handle errors and / or process received information that was not anticipated, modified, and / or not recognized by the decoder. Another factor in the accuracy and / or reliability with which the decoder is able to reconstruct involves providing the decoder with up-to-date information indicating how to reconstruct the signal.

[0006] Another consideration is the ability of the encoder and / or decoder to efficiently process information. The efficiency with which the encoder and / or decoder processes information can be a factor in the performance level of the encoder and / or decoder. Summary of the invention

[0007] Various aspects of the invention are set out in the accompanying claims.

[0008] Other features and advantages will become apparent from the following description of preferred embodiments given by way of example only with reference to the accompanying drawings.

[0009] According to one aspect of the present invention, there is provided a method comprising:

[0010] Obtaining a configuration block including configuration values, the configuration block operable to reconstruct a first representation of an image at a first quality level in a hierarchical structure having a plurality of quality levels using a second representation of the image at the first quality level, the second representation being based on a representation of the image at a second, lower quality level in the hierarchical structure; and

[0011] Get an extended configuration block containing extended configuration values,

[0012] Wherein the extended configuration value and optionally the extended configuration value can be used by a decoder to reconstruct the first representation of the image at the first quality level.

[0013] The decoder (which may be referred to as an extended decoder) may be configured to process an extended configuration block. In some cases, by processing the extended configuration block, the decoder updates the configuration (indicated by the configuration value) or performs another predetermined action to determine the "extended" (or "extended") configuration. The decoder may decode according to the extended configuration. In some instances, the extended decoder processes the extended configuration block to cover and / or exceed the configuration of the configuration block. In this way, the extended configuration block may be used to perform one or more predetermined actions related to the configuration signaled by the configuration block. Examples of such predetermined actions include, but are not limited to, updating, modifying, adjusting, using, covering, and ignoring the configuration signaled by the configuration block. As used herein, the term "update" does not mean modifying the configuration; updating the configuration may result in the same or different configurations.

[0014] According to another aspect, a bitstream is provided that includes a configuration block and an extended configuration block, wherein the configuration block and the extended configuration block are associated with a common configuration. The configuration block includes configuration data and the extended configuration block includes extended configuration data. The bitstream includes data, such as video data, that when decoded is suitable for display on a screen (e.g., displaying a video or image).

[0015] Preferably, the extended configuration block may be configured to adjust and / or modify the configuration signaled by the configuration block.

[0016] Preferably, the extended configuration block is separated from the configuration block in the bitstream. In some examples, the extended configuration block follows at least one configuration block in the bitstream, preferably, the extended configuration block is at the end of the bitstream.

[0017] According to another aspect of the present invention, there is provided a method comprising:

[0018] Obtaining a configuration block including configuration values, the configuration block operable to reconstruct a first representation of an image at a first quality level in a hierarchical structure having a plurality of quality levels using a second representation of the image at the first quality level, the second representation being based on a representation of the image at a second, lower quality level in the hierarchical structure; and

[0019] Get an extended configuration block containing extended configuration values,

[0020] The configuration value is updated based on the extended configuration value, wherein the extended configuration value and the updated configuration value are usable by a decoder to reconstruct the first representation of the image at the first quality level.

[0021] According to another aspect, there is provided an encoder, the decoder being configured to: generate a configuration block comprising a configuration value, wherein the configuration value indicates a configuration to be selected by the decoder; and

[0022] An extended configuration block is generated that includes an extended configuration value, wherein the extended configuration value indicates one or more predetermined actions to be performed by the decoder related to the selected configuration.

[0023] According to another aspect, there is provided a decoder configured to:

[0024] Gets a configuration block containing configuration values

[0025] Selecting a configuration from a plurality of configurations based on the configuration value;

[0026] Get an extended configuration block including extended configuration values;

[0027] performing one or more actions related to the selected configuration based on the extended configuration value;

[0028] Using the configuration block, reconstructing a first representation of an image at a first quality level in a hierarchical structure having a plurality of quality levels using a second representation of the image at the first quality level, the second representation being based on a representation of the image at a second, lower quality level in the hierarchical structure;

[0029] The decoder is configured to reconstruct the first representation of the image at the first quality level using the extended configuration value. According to another aspect, a decoder is provided, the decoder being configured to:

[0030] Get a configuration block containing configuration values;

[0031] obtaining an extended configuration block including extended configuration values; and

[0032] Decoding is performed based on the extended configuration block and optionally based on the configuration block. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 A schematic block diagram showing an example of a signal processing system according to an embodiment of the present invention;

[0034] Figure 2A and 2B A schematic block diagram showing another example of a signal processing system according to an embodiment of the present invention;

[0035] Figure 3 A schematic diagram showing an example of a data processing technique according to an embodiment of the present invention;

[0036] Figure 4 A schematic diagram showing an example of entry point data according to an embodiment of the present invention;

[0037] Figure 5 A schematic diagram showing an example of a configuration message according to an embodiment of the present invention;

[0038] Figure 6 is a flow chart depicting an example of a method according to an embodiment of the present invention;

[0039] Figure 7 A schematic diagram showing another example of a configuration message according to an embodiment of the present invention;

[0040] Figure 8 A schematic diagram showing another example of a configuration message according to an embodiment of the present invention;

[0041] Fig. 9 A schematic diagram showing another example of a configuration message according to an embodiment of the present invention;

[0042] Fig.10 is a flow chart depicting an example of a method according to an embodiment of the present invention;

[0043] Fig.11 is a flowchart depicting another example of a method according to an embodiment of the present invention;

[0044] Fig.12 A schematic diagram showing another example of a data processing technique according to an embodiment of the present invention; and

[0045] Fig.13 A schematic block diagram showing an example of an apparatus according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0046] refer to Figure 1, shows an example of a signal processing system 100. The signal processing system 100 is used to process signals. Examples of various types of signals include, but are not limited to, video signals, image signals, audio signals, volumetric signals such as those used in medical, scientific or holographic imaging, or other multi-dimensional signals.

[0047] The signal processing system 100 includes a first device 102 and a second device 104. The first device 102 and the second device 104 can have a client-server relationship, wherein the first device 102 performs the function of a server device and the second device 104 performs the function of a client device. The signal processing system 100 may include at least one additional device (not shown). The first device 102 and / or the second device 104 may include one or more components. One or more components may be implemented in hardware and / or software. In the signal processing system 100, one or more components may be co-located or may be located away from each other. Examples of various types of devices include, but are not limited to, computerized devices, handheld or laptop computers, tablet computers, mobile devices, game consoles, smart TVs, set-top boxes, augmented and / or virtual reality headsets, etc.

[0048] The first device 102 is communicatively coupled to the second device 104 via a data communication network 106. Examples of the data communication network 106 include, but are not limited to, the Internet, a local area network (LAN), and a wide area network (WAN). The first device 102 and / or the second device 104 may have a wired and / or wireless connection to the data communication network 106.

[0049] The first device 102 includes an encoder 108. The encoder 108 is configured to encode data included in the signal, which is referred to as "signal data" hereinafter. For example, in the case where the signal is a video signal, the encoder 108 is configured to encode the video data. The video data includes a sequence of multiple images or frames. In addition to encoding the signal data, the encoder 108 can also perform one or more additional functions. The encoder 108 can be implemented in a variety of different ways. For example, the encoder 108 can be embodied in hardware and / or software.

[0050] Although the first device 102 includes the encoder 108 in this example, in other examples the first device 102 is separate from the encoder 108. In such examples, the first device 102 is communicatively coupled to the encoder 108. The first device 102 may be embodied as one or more software functions and / or hardware modules.

[0051] The second device 104 includes a decoder 110. The decoder 110 is configured to decode the signal data. In addition to decoding the signal data, the decoder 110 can also perform one or more additional functions. The decoder 110 can be embodied in a variety of different ways. For example, the decoder 110 can be embodied in hardware and / or software.

[0052] Although the second device 104 includes the decoder 110 in this example, in other examples the second device 104 is separate from the decoder 110. In such examples, the second device 104 is communicatively coupled to the decoder 110. The second device 104 may be embodied as one or more software functions and / or hardware modules.

[0053] The encoder 108 encodes the signal data and transmits the encoded signal data to the decoder 110 via the data communication network 106. The decoder 110 decodes the received encoded signal data and generates decoded signal data. The decoder 110 can output the decoded signal data or data derived using the decoded signal data. For example, the decoder 110 can output such data for display on one or more display devices associated with the second device 104.

[0054] In some examples described herein, the encoder 108 transmits to the decoder 110 a representation of a signal at a given quality level and information that the decoder 110 can use to reconstruct the representation of the signal at one or more higher quality levels. Such information can be referred to as "reconstruction data". In some examples, the "reconstruction" of a representation involves obtaining a representation that is not an exact replica of the original representation. The extent to which the representation is the same as the original representation can depend on various factors, including but not limited to the quantization level. The representation of a signal at a given quality level can be considered to be a reproduction, version, or description of the data included in the signal at the given quality level. In some examples, the reconstruction data is included in the signal data encoded by the encoder 108 and transmitted to the decoder 110. For example, the reconstruction data can be in the form of metadata. In some examples, the reconstruction data is encoded and transmitted separately from the signal data.

[0055] The information used by the decoder 110 to reconstruct the representation of the signal at one or more higher quality levels may include residual data, as described in more detail below. Residual data is an example of reconstruction data. The information used by the decoder 110 to reconstruct the representation of the signal at one or more higher quality levels may also include configuration data related to the processing of the residual data. The configuration data may indicate how the residual data has been processed by the encoder 108 and / or how the residual data is to be processed by the decoder 110. The configuration data may be signaled to the decoder 110, for example, in the form of metadata.

[0056] refer to Figure 2A and2B , schematically illustrates an example of a signal processing system 200. The signal processing system 200 includes: a first device 202 including an encoder and a second device 204 including a decoder. In each of the first device 202 and the second device 204, items are shown on two logical levels. The two levels are separated by a dotted line. The items on the first highest level relate to data at a relatively high-quality level. The items on the second lowest level relate to data at a relatively low-quality level. The relatively high-quality level and the relatively low-quality level relate to a hierarchical structure with multiple quality levels. In some instances, the hierarchical structure includes more than two quality levels. In such instances, the first device 202 and the second device 204 may include more than two different levels. There may be Figure 2A and 2B One or more additional levels above and / or below the level depicted in .

[0057] First reference Figure 2A , the first device 202 obtains a first representation 206 of an image at a relatively high quality level. The representation of a given image is a representation of the data included in the image. The image may be a given frame of a video. The first representation 206 of the image at a relatively high quality level will be referred to as "input data" hereinafter because, in this example, it is data provided as input to an encoder in the first device 202. The first device 202 may receive the input data 206. For example, the first device 202 may receive the input data 206 from at least one other device. The first device 202 may be configured to receive consecutive portions of the input data 206 (e.g., consecutive frames of a video) and perform the operations described herein on each consecutive frame. For example, a video may include frame F 1 、F 2 ,……F T , and the first device 202 can process each of these frames in sequence.

[0058] The first device 202 derives data 212 based on the input data 206. In this example, the data 212 based on the input data 206 is a representation 212 of an image at a relatively low quality level. In this example, the data 212 is derived by performing a downsampling operation on the input data 206, and will therefore be referred to as "downsampled data" hereinafter. In other examples, the data 212 is derived by performing an operation other than a downsampling operation on the input data 206.

[0059] In this example, the downsampled data 212 is processed to generate processed data 213 at a relatively low quality level. In other examples, the downsampled data 212 is not processed at the relatively low quality level. Thus, the first device 202 can generate data at a relatively low quality level, wherein the data at the relatively low quality level includes the downsampled data 212 or the processed data 213.

[0060] In some instances, generating the processed data 213 involves encoding the downsampled data 212. Encoding the downsampled data 212 produces an encoded image at a relatively low quality level. The first device 202 can output the encoded image, for example, for transmission to the second device 204. A series of encoded images (e.g., forming an encoded video) as an output for transmission to the second device 204 can be referred to as a "basic" stream. The encoded image can be generated by an encoding device separate from the first device 202, rather than generated in the first device 202. The encoded image can be part of an H.264 encoded video. Generating the processed data 213 can, for example, include generating continuous frames of a video as output by a separate encoding device such as an H.264 video encoder. The intermediate data set used to generate the processed data 213 can include the output of such an encoder, as opposed to any intermediate data generated by a separate encoding device.

[0061] Generating the processed data 213 at a relatively low quality level may further involve decoding the coded image at the relatively low quality level. A decoding operation may be performed to simulate a decoding operation at the second device 204, as will become apparent below. Decoding the coded image produces a decoded image at a relatively low quality level. In some instances, the first device 202 decodes the coded image at a relatively low quality level to produce a decoded image at a relatively low quality level. In other instances, the first device 202 receives a decoded image at a relatively low quality level, for example, from an encoding and / or decoding device separate from the first device 202. The coded image may be decoded using an H.264 decoder. The decoding performed by a separate decoding device may include: inputting a coded video (such as a coded data stream configured for transmission to a remote decoding device) into a separate black box decoder implemented with the first device 202 to generate continuous decoded frames of the video. The processed data 213 may therefore include frames of video data generated via a complex non-linear encoding and decoding process, wherein the encoding and decoding process may involve modeling spatiotemporal correlations according to a specific coding standard such as H.264. However, because the output of any encoder is fed into a corresponding decoder, this complexity is effectively hidden from the first device 202.

[0062] In an example, generating the processed data 213 at a relatively low quality level further involves obtaining correction data based on a comparison between the downsampled data 212 and a decoded image obtained by the first device 202 (e.g., based on a difference between the downsampled data 212 and the decoded image). The correction data can be used to correct errors introduced when encoding and decoding the downsampled data 212. In some examples, the first device 202 outputs the correction data (e.g., for transmission to the second device 204) as well as the encoded signal. This allows the recipient to correct errors introduced when encoding and decoding the downsampled data 212. The correction data may also be referred to as a "first enhancement" stream. Since the correction data may be based on the difference between the downsampled data 212 and the decoded image, it may be considered as a form of residual data (e.g., different from another residual data set described later below).

[0063] In some instances, generating the processed data 213 at a relatively low quality level further involves using the correction data to correct the decoded image. For example, the correction data for transmission as an output can be placed in a form suitable for combination with the decoded image and then added to the decoded image. This can be done on a frame-by-frame basis. In other instances, the first device 202 uses the downsampled data 212 instead of using the correction data to correct the decoded image. For example, in some cases, only the encoded data can be used and then the decoded data, while in other cases, the encoding and decoding can be replaced by other processes.

[0064] In some examples, generating processed data 213 involves performing one or more operations in addition to the encoding, decoding, obtaining, and correcting actions described above.

[0065] The first device 202 obtains data 214 based on the data at the relatively low quality level. As described above, the data at the relatively low quality level may include processed data 213 or downsampled data 212, where the downsampled data 212 is not processed at a lower level. As described above, in some cases, the processed data 213 may include a reconstructed video stream (e.g., from an encoding-decoding operation) corrected using correction data. Figure 2A and 2BIn the example of , data 214 is a second representation of an image at a relatively high quality level, and the first representation of the image at the relatively high quality level is input data 206. The second representation at the relatively high quality level can be considered as a preliminary or predicted representation of the image at the relatively high quality level. In this example, the first device 202 derives data 214 by performing an upsampling operation on the data at the relatively low quality level. Data 214 will be referred to as "upsampled data" hereinafter. However, in other examples, one or more other operations may be used to derive data 214, for example, where data 212 is not derived by downsampling input data 206. It should be noted that references to relatively high quality levels and relatively low quality levels may correspond to references to a first quality level and a second quality level, where the second quality level is higher than the first quality level. As described herein, in some cases, quality levels may correspond to different spatial resolutions.

[0066] The input data 206 and the upsampled data 214 are used to obtain residual data 216. The residual data 216 is associated with an image. The residual data 216 may be in the form of a residual element set. The residual elements in the residual element set 216 may be associated with corresponding image elements in the input data 206. An example of an image element is a pixel.

[0067] In this example, a given residual element is obtained by subtracting the value of an image element in upsampled data 214 from the value of a corresponding image element in input data 206. Thus, residual data 216 may be used in combination with upsampled data 214 to reconstruct input data 206. Residual data 216 may also be referred to as "reconstructed data" or "enhancement data." In one case, residual data 216 may form part of a "second enhancement" stream.

[0068] The first device 202 obtains configuration data related to the processing of the residual data 216. The configuration data indicates how the residual data 216 has been processed and / or generated by the first device 202 and / or how the residual data 216 is to be processed by the second device 204. The configuration data may include a set of configuration parameters. The configuration data may be used to control how the second device 204 processes data and / or how to use the residual data 216 to reconstruct the input data 206. The configuration data may relate to one or more characteristics of the residual data 216. Different configuration data may result in different processing of the residual data 216 and / or different processing using the residual data. The configuration data may therefore be used to reconstruct the input data 206 using the residual data 216.

[0069] In this example, the first device 202 transmits data based on the downsampled data 212 , data based on the residual data 216 , and configuration data to the second device 204 to enable the second device 204 to reconstruct the input data 206 .

[0070] Now turn to Figure 2B , the second device 204 receives data 220 based on the downsampled data 212 (e.g., derived from the downsampled data). The second device 204 also receives data based on the residual data 216. For example, the second device 204 may receive a "base" stream (data 220), a "first enhancement stream" (any correction data), and a "second enhancement stream" (residual data 216). The second device 204 also receives configuration data related to the processing of the residual data 216. The data 220 based on the downsampled data 212 may be the downsampled data 212 itself, the processed data 213, or data derived from the downsampled data 212 or the processed data 213. The data based on the residual data 216 may be the residual data 216 itself or data derived from the residual data 216.

[0071] In some instances, the received data 220 includes processed data 213, which may include coded images and / or corrected data at a relatively low quality level. In some instances, for example, in the case where the first device 202 has processed the downsampled data 212 to generate the processed data 213, the second device 204 processes the received data 220 to generate processed data 222. Such processing by the second device 204 may include decoding a coded image (e.g., forming part of a "basic" coded video stream) to produce a decoded image at a relatively low quality level. In some instances, the processing performed by the second device 204 includes correcting the decoded image using the obtained corrected data. Therefore, the processed data 222 may include a frame of corrected data at a first quality level or a relatively low quality level. In some instances, the coded image at a relatively low quality level is decoded by a decoding device separate from the second device 204. The coded image at a relatively low quality level can be decoded using an H.264 decoder.

[0072] In other instances, received data 220 includes downsampled data 212 and does not include processed data 213. In some such instances, second device 204 does not process received data 220 to generate processed data 222.

[0073] The second device 204 uses the data at the relatively low quality level to derive the upsampled data 214. As described above, the data at the relatively low quality level may include the processed data 222 or the received data 220, wherein the second device 204 does not process the received data 220 at the relatively low quality level. The upsampled data 214 is a preliminary representation of the image at the relatively high quality level. The upsampled data 214 may be derived by performing an upsampling operation on the data at the relatively low quality level.

[0074] The second device 204 obtains residual data 216. The residual data 216 may be used together with the upsampled data 214 to reconstruct the input data 206. The residual data 216 indicates a comparison between the input data 206 and the upsampled data 214.

[0075] The second device 204 also obtains configuration data related to the processing of the residual data 216. The configuration data can be used by the second device 204 to reconstruct the input data 206. For example, the configuration data can indicate characteristics or properties related to the residual data 216 that affect how the residual data 216 is to be used and / or processed, or whether the residual data 216 is to be used in any way. In some examples, the configuration data includes the residual data 216.

[0076] There are several considerations associated with the use of such configuration data. One such consideration is the amount of information generated, stored, transmitted and / or processed. The more information that is used, the greater the amount of resources that may be involved in processing such information. Examples of such resources include transmission resources, storage resources, and processing resources. Compared to some known techniques, the examples described herein allow for the use of relatively small amounts of information. This can reduce the amount of data transmitted via the data communications network 106. This savings can be particularly important in situations where the data relates to high-quality video data, where the amount of information transmitted in known systems may be particularly high.

[0077] Another consideration related to the use of such configuration data is the ability to accommodate different types of decoding devices. For example, a given encoder may transmit information to multiple decoders, each with different characteristics, such as processing power and / or storage capacity. It is desirable that the use of configuration data in image reconstruction is independent of the type and / or capabilities of the decoder device used to perform the reconstruction. In contrast to some known techniques, the examples described herein enable different types of decoders with different operating characteristics to reliably process configuration data.

[0078] Another consideration related to the use of such configuration data is the amount and / or relative complexity of processing involved at the encoder and / or decoder in order to process the configuration data. Compared to some known techniques, the examples described herein reduce the amount and / or complexity of processing performed by the encoder and / or decoder in order to obtain and / or process the configuration data. Simplifying the processing performed by the encoder and / or decoder improves the efficiency of the encoder and / or decoder.

[0079] Other considerations include the ability of the decoder to accurately and reliably reconstruct the image. Accurately and reliably reconstructing the image may affect the final visual quality of the displayed image, and therefore may affect the viewer's engagement with the image and / or the video including the image. One factor that may affect the accuracy and / or reliability of the decoder's reconstruction is providing the decoder with up-to-date and / or accurate information about how the reconstruction is to be performed. For example, configuration data may change over time and / or between different images. If the decoder is not informed of such changes in a timely manner, the visual quality of the reconstructed image may be affected. Compared to some known techniques, the examples described herein allow configuration information to be signaled to the decoder relatively frequently, thereby enabling the decoder to use relatively up-to-date configuration information in image reconstruction.

[0080] Error handling is also a consideration. For example, the ability of a decoder to handle potential errors in the processing of configuration data can be a factor in the reliability of the decoder. Compared to some known techniques, the examples described herein allow the decoder to handle potential errors and the reception of unexpected or unrecognized data. This can effectively improve the reliability of the decoder.

[0081] refer to Figure 3 , schematically illustrates an example of a data processing technique 300. The data processing technique 300 may be performed by a device including an encoder, such as the first device 102 described above. The data processing technique 300 involves performing byte-by-byte processing of configuration data, as described in more detail below.

[0082] Configuration data 310 is obtained. Configuration data 310 is related to the processing of residual data (e.g., residual data 216 as described above). It should be understood that Figure 3 The depicted configuration data 310 is an illustrative representation of the content of such configuration data. In actual use, the configuration data may be read and / or written as a series of 1s and 0s, for example, [01000010].

[0083] Configuration data 310 includes a value set 320. The value set 320 corresponds to a configuration parameter set. The value of a given configuration parameter has a given bit length. The bit length is the number of bits used to represent the value of a given configuration parameter. The bit length of the value of a given configuration parameter can be fixed or can be variable. For a variable bit length, the number of bits used to represent the value of a given configuration parameter varies based on the value of the given configuration parameter.

[0084] In some examples, configuration data 310 includes a reference to a lookup table that stores possible values ​​for a given configuration parameter. For example, configuration data 310 may include a value that can be used to obtain a value for a configuration parameter from the lookup table.

[0085] Configuration data 310 is arranged into a byte sequence 330. Byte sequence 330 includes an integer number of bytes. In this example, the value of configuration parameter 320 is arranged into a byte sequence including 5 bytes, but it should be understood that other numbers of bytes can be used in other examples. In this example, a given byte in byte sequence 330 includes 8 bits. Byte sequence 330 can be part of a larger byte sequence or byte stream. Arranging configuration data 310 into byte sequence 330 enables byte-by-byte processing of configuration data 310. In byte-by-byte processing, data is processed on a byte-by-byte basis.

[0086] Processing the configuration data 310 in a byte-by-byte manner may be more efficient than processing the configuration data 310 in a bit-by-bit manner. Bit-by-bit processing may involve tracking which bit within a given byte is the next bit to be read or written. When a bit group representing a given parameter crosses a byte boundary, additional processing may be performed in order to read the bit group and obtain the given parameter. Such processing may be reduced in the case of performing byte-by-byte processing. In some instances, multiple execution cycles may be reduced when processing data in a byte-by-byte manner compared to a bit-by-bit manner.

[0087] In some examples, a sequence of bytes 330 is written to a memory. If the configuration data 310 is processed in a byte-by-byte manner, less processing may be involved when storing the configuration data 310 in a memory and / or retrieving the configuration data 310 from the memory than when the configuration data 310 is processed in a bit-by-bit manner. The smallest unit of addressable memory may include a byte. Therefore, information may be stored in a memory in a byte-by-byte manner. Therefore, if the configuration data 310 is packaged into a sequence of bytes, fewer steps may be involved when storing the configuration data 310 in a memory and / or retrieving the configuration data 310 from the memory. The amount of padding used to package the configuration data 310 into addressable memory cells may also be reduced by providing the configuration data 310 to the memory in a byte-by-byte form.

[0088] In addition, if the configuration data 310 is arranged as a sequence of bytes, the transmission of the configuration data 310 via a network (such as the Internet) can be more efficiently performed. The data can be transmitted over the Internet via the Transmission Control Protocol (TCP). TCP runs at the transport layer of the protocol stack. TCP obtains data from a stream, packages the data into TCP segments, and transmits each TCP segment to a recipient via the Internet using an Internet module (e.g., Internet Protocol (IP)). The data stream can be obtained from a memory. A TCP segment may include an integer number of bytes. Therefore, if TCP obtains data to be packaged into a TCP segment from a byte stream, less processing may be involved than obtaining data to be packaged into a TCP segment from a bit stream. The amount of padding used to package data into a TCP segment can also be reduced by providing data to TCP in a byte-by-byte form.

[0089] In this example, the configuration data 310 is arranged into a byte sequence 330 at the application layer of the protocol stack. Therefore, the application layer can process the configuration data 310 in a byte-by-byte manner rather than a bit-by-bit manner.

[0090] In some examples, configuration data 310 includes values ​​of multiple configuration parameters having different bit lengths. Configuration data 310 may be packed into byte sequence 330 by arranging the values ​​in a predetermined order. The predetermined order may be a different order than the order in which the values ​​are obtained. Therefore, the values ​​may be rearranged.

[0091] The values ​​of configuration parameters 320 may be arranged such that a given byte in byte sequence 330 is populated with values ​​represented by different bit lengths. By populating a given byte in byte sequence 330 with values ​​represented by different bit lengths arranged in a predetermined order, the amount of padding used in the byte sequence 330 may be reduced. Thus, data may be packed more efficiently into byte sequence 330 than if the values ​​represented by different bit lengths were not arranged in a predetermined order. Packing data more efficiently into byte sequence 330 may reduce the number of bytes used to store configuration data 310. Thus, the amount of information to be transmitted over a network may be reduced.

[0092] The byte sequence 330 is output for processing by a decoder, such as the decoder 110 , to enable the decoder to reconstruct the input data 206 using the configuration data 310 .

[0093] refer to Figure 4 , schematically illustrates an example of entry point data 400. Entry point data 400 may be output by encoder 108 and received by decoder 110. Entry point data 400 is a type of configuration data. In some examples, entry point data 400 forms part of configuration data 310. Entry point data 400 may be read by decoder 110 before reading other configuration data. Thus, entry point data 400 may be the first configuration data processed by decoder 110 for a given received video stream.

[0094] The entry point data 400 includes a version parameter 420. The version parameter 420 may indicate a version number. The version may be a version of a syntax according to which the configuration data is arranged. The syntax may be considered to be a framework that defines how the configuration data is to be processed and / or defines configuration parameters to be included in the configuration data. The encoder 108 and / or the decoder 110 may be configured to operate according to the version of the syntax indicated by the version parameter 420.

[0095] The value of version parameter 420 has a fixed bit length. In some examples, the value of version parameter 420 has a bit length of 8 bits. Therefore, the value of version parameter 420 can be stored in a fixed length element in byte sequence 330. The fixed length element includes a predetermined integer number of bytes.

[0096] In this example, the entry point data 400 includes an extended version parameter 425. The value of the extended version parameter 425 has a variable bit length. The value of the extended version parameter 425 can be stored in a variable length element in the byte sequence 330.

[0097] A variable length element includes an integer number of bytes in the byte sequence 330. A variable length element may be referred to as a "multi-byte" element, although in some instances the variable length element includes a single byte. One or more bytes of a variable length element may be contiguous in the byte sequence 330. In some instances, one or more bytes of a variable length element are not contiguous in the byte sequence 330.

[0098] At least one given byte of a variable length element has one or more predetermined positions, and these predetermined positions are arranged to indicate whether the variable length element includes one or more additional bytes except at least one given byte. In some instances, the one or more predetermined positions include a single predetermined position. In other instances, the one or more predetermined positions include more than one predetermined position. In some instances, each byte of the variable length element includes one or more predetermined positions. In other instances, at least one byte of the variable length element does not include one or more predetermined positions.

[0099] In the case where the one or more predetermined bits include a single predetermined bit, the single predetermined bit can be considered as a flag that indicates whether one or more additional bytes are present in the variable length element. For example, if the variable length element contains additional bytes in addition to at least one given byte, the value of the predetermined bit can be set to 1. On the other hand, if the variable length element does not contain additional bytes in addition to at least one given byte, the value of the predetermined bit can be set to 0. In other words, the predetermined bit indicates whether the variable length element exceeds at least one given byte.

[0100] One or more predetermined bits may be arranged at the end of at least one given byte. For example, one or more predetermined bits may include one or more last bits of at least one given byte. In some instances, one or more predetermined bits include one or more first bits of at least one given byte. In some instances, each of the at least one given byte includes seven bits arranged to store the value of the extended version parameter 425 and one bit arranged to indicate whether the variable length element includes one or more additional bytes other than the at least one given byte.

[0101] Therefore, the value of extended version parameter 425 can be represented using variable length elements, regardless of the bit length of the value of extended version parameter 425. Variable length elements include the minimum number of bytes that can be used to store the value of extended version parameter 425. Compared with the situation of using fixed length elements with the size corresponding to the maximum imaginable bit length of the value of extended version parameter 425, the use of variable length elements can therefore reduce the amount of information stored, processed and / or transmitted. Variable length elements enable the number of possible values ​​that can be used to represent extended version parameter 425 to be increased as needed and in a flexible manner. Therefore, variable length elements can be used to provide future development and / or modification of configuration data 310, for example, by allowing the increase of the bit length of the value of extended version parameter 425, without the need for relatively complex modification of the structure of configuration data 310.

[0102] In this example, an extended version parameter 425 is optionally used. Figure 4 This is depicted using dashed lines in . If it is determined that the fixed length version parameter 420 is insufficient to store the value of the version number, the extended version parameter 425 is used. In some instances, in response to determining that the value of the version number has a predetermined relationship with a predetermined threshold, the value of the version number is stored in the extended version parameter 425. In some instances, the predetermined relationship includes the value of the version number being greater than or equal to the predetermined threshold. In some instances, if it is determined that the value of the version number does not have a predetermined relationship with the predetermined threshold, the value of the version number is stored in the version parameter 420 rather than in the variable length extended version parameter 425. Using variable length elements selectively reduces the amount of information used, stored and / or transmitted compared to a situation in which variable length elements are used regardless of the value of the version number.

[0103] In some examples, the predetermined threshold is related to the bit length of the fixed length element (i.e., the bit length of version parameter 420). For example, if the fixed length element includes eight bits, then values ​​between 0 and 255 can be represented by the fixed length element. For example, the predetermined threshold in this case can be 255. Therefore, if the value of the version number is greater than 255, a variable length extended version parameter 425 can be used to store the value of the version number instead of or in addition to the fixed length version parameter 420.

[0104] In some examples, data is stored in version parameter 420 to indicate that a version number is stored in extended version parameter 425. In some examples, the data includes at least one predetermined bit in version parameter 420. The data stored in version parameter 420 enables decoder 110 to determine the value of the version number stored in extended version parameter 425.

[0105] The above-described use of variable length elements and / or selectivity of such variable length elements comprise other configuration parameters that may also be applied to the configuration data output by the encoder 108 .

[0106] In addition or to alternatives to selectively including configuration parameters corresponding to variable length elements, encoder 108 may selectively include configuration parameters corresponding to fixed length elements. In some instances, the configuration data generated by encoder 108 includes a parameter presence indicator. The parameter presence indicator specifies whether a given configuration parameter set is included in a given portion of the configuration data. A given configuration parameter set has a predetermined format. In an instance related to the above-mentioned version number, the parameter presence indicator includes a predetermined bit in version parameter 420 indicating that an extended version parameter 425 is being included. The parameter presence indicator corresponds to a nested logic statement (such as an IF statement). That is, if the parameter presence indicator has or is given a specific value, a given configuration parameter set is included in the configuration data. Including a given configuration parameter set selectively reduces the amount of information used, stored and / or transmitted compared to a situation in which a given configuration parameter set is not selectively included.

[0107] Reference Figure 5 , schematically illustrates an example of a configuration message 500. The configuration message 500 may form part of the configuration data 310. The configuration message 500 may be generated by the encoder 108. The configuration message 500 may be received and / or read by the decoder 110 after the entry point data 400 has been received and / or read.

[0108] Configuration message 500 includes header portion 510 and payload portion 520. Payload portion 520 includes configuration parameter set 550. Configuration parameter set 550 is related to processing of residual data (eg, residual data 216 as described above). In some examples, configuration message 500 is an integer number of bytes long.

[0109] Header portion 510 includes a message type parameter 512. Message type parameter 512 specifies a message type of configuration message 500. The message type specified by message type parameter 500 may indicate how configuration message 500 is to be processed. Exemplary types of configuration messages are described below.

[0110] The message type parameter 512 indicates a payload portion 520 having a given payload format. The payload format of the payload portion 520 can be determined based on the message type of the configuration message 500. The payload format can define a set of configuration parameters included in the payload portion 520. In some instances, the payload format definition of the payload portion 520 specifies a specific processing logic for how to process the content of the payload portion 520. The payload portion 520 has a format size. The format size of the payload portion 520 corresponds to the payload format of the payload portion 520. The format size of the payload portion having a given payload format is predetermined or can be determined from the content of the payload portion. For example, the format size can be determined based on the combined size of the various configuration parameters defined by the payload format. The format size can be considered as a possible size or a suggested size of the payload portion 520.

[0111] The header portion 510 also includes a payload size parameter 514. The payload size parameter 514 specifies the actual size of the payload portion 520. The actual size of the payload portion 520 may be an integer number of bytes.

[0112] In some examples, the value of payload size parameter 514 is stored in a fixed length element. The fixed length element storing the value of payload size parameter 514 may include a predetermined number of bits in a given byte of header portion 510. For example, the value of payload size parameter 514 may be stored in a fixed length element that includes three bits in a one-byte header portion.

[0113] In some examples, payload size parameter 514 includes a reference to a payload size lookup table that stores possible values ​​for the actual size of payload portion 520. For example, payload size parameter 514 may include a value that can be used to obtain the actual size of payload portion 520 from the payload size lookup table.

[0114] In some instances, the value of the payload size parameter 514 is stored in a variable length element. For example, this may be the case when the actual size of the payload portion 520 is not one of the values ​​specified in the payload size lookup table. The actual size of the payload portion 520 specified by the payload size parameter 514 may have a variable bit length. Therefore, the payload size parameter 514 may be represented using a variable length element regardless of the bit length of the payload size parameter 514. The variable length element includes the minimum number of bytes that can be used to store the value of the payload size parameter 514.

[0115] In some instances, data is stored in a fixed length element that indicates that the value of the payload size parameter 514 is stored in a variable length element. For example, if it is determined that the value of the payload size parameter 514 is one of a predetermined set of values, the value may be stored in a fixed length element and / or the fixed length element may include a reference to a payload size lookup table. On the other hand, if it is determined that the value of the payload size parameter 514 is not one of a predetermined set of values, a variable length element may be used to store the value, and the fixed length element may be arranged to indicate, for example, via a flag included in the fixed length element, that the variable length element is being used to store the value.

[0116] In this example, header portion 510 includes only message type parameter 512 and payload size parameter 514. In other examples, header portion 510 includes one or more additional parameters. In some examples, header portion 510 is one byte long. For example, message type parameter 512 can be represented by five bits, and payload size parameter 514 can be represented by the remaining three bits of the one-byte header portion 510. In other examples, for example, in the case where a variable length element is used to represent the actual size of payload portion 520, header portion 510 includes more than one byte.

[0117] The actual size of the payload portion 520 may be the same as or different from the format size of the payload portion 520. Regardless of whether the actual size and the format size are the same, a payload size parameter 514 specifying the actual size is included in the header portion 510 of the configuration message 500.

[0118] When the likely size of the payload portion 520 (i.e., the format size) can be inferred from the message type parameter 512, including a payload size parameter 514 that specifies the actual size of the payload portion 520 may appear inefficient and / or duplicative of information, particularly when the format size and the actual size are the same. Including the payload size parameter 514 in the configuration message 500 involves a greater amount of data being used for the configuration message 500 than where such a parameter is not included. However, including the payload size parameter 514 in the configuration message 500 facilitates more efficient and reliable processing and / or improved error handling by the decoder 110 than where such a parameter is not used, as will be described in more detail below.

[0119] refer to Figure 6 , shows an example of a method 600 for processing configuration data. The method 600 may be performed by a device including a decoder (such as the second device 104 described above).

[0120] In some examples, residual data is received. In this example, the received residual data is the above-mentioned residual data 216. The residual data 216 can be received from an encoder (such as encoder 108).

[0121] At item 610, a configuration message is received. In this example, the received configuration message is the configuration message 500 described above. The configuration message 500 may be received from the encoder 108.

[0122] At item 620, the received configuration message 500 is processed using the message type parameter 512 and the payload size parameter 514. The configuration message 500 is processed to obtain the configuration parameter set 550 stored in the payload portion 520 of the configuration message 500.

[0123] In some examples, processing the configuration message 500 includes parsing a header portion 510 of the configuration message 500 to obtain a message type parameter 512 and a payload size parameter 514. Based on the payload size parameter 514, determining an actual size of the payload portion 520. Based on the message type parameter 512, determining a message type of the configuration message 500. Processing the configuration message 500 according to the determined message type and the determined actual size of the payload portion 520.

[0124] Processing the configuration message 500 includes parsing the payload portion 520 to obtain the configuration data 550. In some examples, processing the configuration message 500 includes tracking the number of bytes of the configuration message 500 that have been read while parsing the payload portion 520. The decoder 110 can detect the occurrence of an underread if there is a determined discrepancy between the number of bytes read while parsing the payload portion 520 and the actual size of the payload portion 520 as specified by the payload size parameter 514. An underread can be a potential source of errors when processing configuration messages.

[0125] In some instances, payload size parameter 514 is used to determine the start of a subsequent configuration message. The following configuration message may be the next consecutive configuration message in a sequence of configuration messages received by decoder 110 relative to configuration message 500. The sequence of configuration messages may form part of the configuration data. Configuration message 500 and a subsequent configuration message may correspond to the same image or different images. For example, configuration message 500 may correspond to a first image in a sequence of images, and a subsequent configuration message may correspond to a second different image in the sequence of images.

[0126] In some examples, processing configuration message 500 includes discarding at least a portion of the contents of payload portion 520. Payload size parameter 514 may be used to discard at least a portion of the contents of payload portion 520. For example, decoder 110 may skip to the beginning of the next configuration message, i.e., the next configuration message that has been determined using payload size parameter 514, without processing at least a portion of the contents of payload portion 520.

[0127] Therefore, using the payload size parameter 514 enables the decoder 110 to process possible insufficient reading in an effective manner. Using the payload size parameter 514 to locate the beginning of subsequent messages also enables the decoder 110 to maintain the correct alignment or synchronization with the byte stream comprising multiple configuration messages. For example, even if the content and / or format of a given configuration message is not recognized, not expected and / or has been updated without the knowledge of the decoder 110, the decoder 110 can still locate the beginning of subsequent configuration messages and thereby keep synchronization with the byte stream. In addition, if the byte stream format is wrong, the decoder 110 can detect when excessive reading will occur and respond accordingly. For example, the decoder 110 can stop the processing of the current configuration message and reset the byte stream to the determined position of the beginning of the next configuration message. Compared with the situation where such parameters are not used, using the payload size parameter 514 to process the configuration message 500 has therefore improved the reliability and / or adaptability of the decoder 110, because the decoder 110 can process the configuration message that has been modified and / or cannot be recognized and / or is not expected. The decoder 110 is also able to handle potential under-reads and over-reads without being out of sync with the byte stream, thereby improving the ability of the decoder 110 to handle potential errors.

[0128] In some instances, processing the configuration message 500 includes parsing the first content of the payload portion 520. The first content corresponds to the format size of the payload portion 520. The first content may include a configuration parameter set 550. The first content may include first configuration data. Processing the configuration message 500 may further include discarding the remaining content of the payload portion 520 using the payload size parameter 514. The remaining content of the payload portion 520 includes content of the payload portion 520 that is not included in the first content. The remaining content includes additional configuration data. The presence of the remaining content may be determined by detecting a difference between the format size and the actual size of the payload portion 520. In other words, if it is determined that the actual size is greater than the format size, it is determined that there is additional content in the payload portion 520 other than the content expected based on the format size.

[0129] At item 630, the input data 206 is reconstructed using the obtained set of configuration parameters 550. The input data 206 may be reconstructed according to the obtained set of configuration parameters 550 by using the residual data 216.

[0130] In some instances, the payload portion 520 of the configuration message 500 includes a first payload portion and a second payload portion. The payload size parameter 514 may indicate the total size of the first payload portion and the second payload portion. The first payload portion and the second payload portion may include first configuration data and second configuration data, respectively. In some instances, the second configuration data includes supplementary configuration data. The supplementary configuration data is arranged to supplement the first configuration data. The second payload portion may be arranged in a subsequent manner relative to the first payload portion. Therefore, before receiving, processing and / or parsing the second payload portion, the first payload portion may be received, processed and / or parsed by the decoder 110.

[0131] In some instances, header portion 510 does not specify a size of the first payload portion and / or a size of the second payload portion. In some instances, the size of the first payload portion may be determined based on the format size indicated by message type parameter 512. Thus, the format size indicated by message type parameter 512 may reflect the contents of the first payload portion but not the contents of the second payload portion.

[0132] The first configuration data relates to a first set of one or more decoders. The first set of one or more decoders has a first operating characteristic. An example of an operating characteristic is processing power. The second configuration data relates to a second set of one or more decoders. The second set of one or more decoders has a second operating characteristic. The decoder 110 to which the configuration message 500 is transmitted may be included in the first set or the second set. The encoder 108 that generates the configuration message 500 may not know in which decoder set the decoder 110 is included. The encoder 108 may transmit the configuration message 500 to multiple decoders across two decoder sets.

[0133] In some examples, the first operating characteristic corresponds to a first version according to which a first set of one or more decoders are configured to operate, and the second operating characteristic corresponds to a second, different version according to which a second set of one or more decoders are configured to operate. The second version may be a newer version relative to the first version. The first version and the second version may be different versions of a syntax framework that defines how configuration messages are to be processed and / or defines the content of such configuration messages. Thus, when the syntax is updated from the first version to the second version, the second configuration data is added to the configuration message that contains the first configuration data from the first version. The second configuration data may, for example, be appended to the first configuration data.

[0134] Adding the second configuration data to a configuration message of an existing message type rather than generating a new message type to accommodate the second configuration data reduces the amount of data stored, transmitted and / or processed compared to a situation in which a new message type is generated. By modifying the configuration of an existing message type rather than defining a new message type, the total number of configuration messages used to transmit all relevant configuration data can be reduced. Each configuration message has a corresponding header, which can be at least one byte long. Therefore, by reducing the total number of configuration messages, the amount of data stored, transmitted and / or processed can be reduced.

[0135] In some instances, decoder 110 is not able to process both the first payload part and the second payload part. For example, encoder 108 may have been updated, according to which the second configuration data is included in the configuration message 500, but decoder 110 may not have been updated yet. In other words, encoder 108 and decoder 110 can operate according to different versions of syntax. In such instances, there may be differences in the format size of payload part 520 explained by encoder 108 and explained by decoder 110. For example, the format size according to encoder 108 (which knows the additional content of payload part 520 and the updated payload format after the version update) is different from the format size according to decoder 110, because the decoder 110 does not know the additional content and the updated payload format and operates according to the previous version. For encoder 108, the format size is equal to the actual size of the updated payload part 520, but for decoder 110, it is not. The independent format size may therefore be an unreliable indicator of the actual size of payload part 520. Including the payload size parameter 514 that specifies the actual size of the payload portion 520 thus facilitates more reliable processing by the decoder 110 .

[0136] In some instances, for example, if the decoder 110 is in a first decoder set, the first payload portion is parsed by the decoder 110, and the second payload portion is not parsed by the decoder 110. Parsing the first payload portion but not parsing the second payload portion enables the decoder 110 to obtain the first configuration data instead of the second configuration data. In some instances, for example, if the decoder 110 is in a second decoder set, the second payload portion is parsed by the decoder 110, and the first payload portion is not parsed by the decoder 110. Parsing the second payload portion but not parsing the first payload portion 522 enables the decoder 110 to obtain the second configuration data instead of the first configuration data. In some instances, for example, if the decoder 110 is in a second decoder set, both the first payload portion and the second payload portion are parsed to obtain both the first configuration data and the second configuration data.

[0137] As described above, the payload size parameter 514 can be used to determine the start of a subsequent configuration message. In some instances, for example, if the decoder 110 is in the first decoder set, the contents of the second payload portion are discarded. The contents of the second payload portion can be discarded by using the total size of the payload portion 520 specified in the payload size parameter 514. Using the payload size parameter 514 to locate the start of a subsequent message enables the decoder 110 to remain synchronized with the byte stream regardless of whether the decoder 110 is in the first decoder set or the second decoder set.

[0138] In the case where the decoder 110 obtains both the first configuration data and the second configuration data, both the first configuration data and the second configuration data are used to reconstruct the first representation 206 of the image. In the case where the decoder 110 processes only one of the first payload portion and the second payload portion, the obtained one of the first configuration data and the second configuration data may be used to reconstruct the first representation 206 of the image. Thus, the use of the configuration message 500 to reconstruct the first representation 206 may be independent of the type of decoder and / or the version number according to which the decoder 110 is configured to operate.

[0139] refer to Figure 7 , which is a schematic diagram showing an example of a configuration message 700. The configuration message 700 may form part of the configuration data 310. The configuration message 700 may be generated by the encoder 108. Figure 7 Some of the projects depicted are related to Figure 5 The items shown are similar. Therefore, corresponding reference numerals increased by 200 have been used for similar items accordingly.

[0140] Configuration message 700 is a more specific example of configuration message 500. Configuration message 700 includes a header portion 710 and a payload portion 720. Header portion 710 includes a message type parameter 712 and a payload size parameter 714. Payload portion 720 includes a configuration parameter set. The configuration parameter set is related to the processing of residual data 216. In some examples, configuration message 700 is an integer number of bytes long.

[0141] The message type of configuration message 700 is an image set configuration message. An image set configuration message corresponds to a plurality of images in an image set. In some instances, an image set includes 75 images. An image set may be a part of a video signal. For example, an image set may include continuous frames of a video. A video may be a streaming video. An example of a video is a broadcast video. For example, a video may be broadcast via a data communication network 106. In some instances, an image set is encoded as a group of pictures (GOP). A GOP is an encoding of an image sequence, and each image in the image sequence can be decoded using information completely contained within the GOP. A video may include a plurality of such GOPs.

[0142] In some examples, the picture set configuration message corresponds to each picture in the picture set. The payload portion 720 includes a picture set configuration parameter set. The picture set configuration parameter set has a predetermined format.

[0143] In this example, the image set configuration parameter set includes a transform type parameter 751. The transform type parameter 751 relates to a transform operation performed on the residual data 216. The transform operation may be a directional decomposition operation. An example of a directional decomposition operation is a Hadamard transform. The transform operation may exploit spatial and / or directional correlations between residual elements in the residual data 216. Performing a transform operation on the residual data 216 produces transformed residual data. The transformed residual data may be smaller than the residual data 216. The transformed residual data may have a lower coding entropy than the residual data 216. Therefore, it may be more efficient for the encoder 108 to transmit the transformed residual data than to transmit the residual data 216. The decoder 110 receiving the transformed residual data may then perform an inverse transform operation on the transformed residual data to obtain the residual data 216. The encoder 108 therefore signals to the decoder 110 via the transform type parameter 751 which transform operation has been applied to the residual data 216. In some examples, the transform type parameter 751 specifies a transform operation from a plurality of possible transform operations that may be applied to the residual data 216. In some examples, the value of the transform type parameter 751 has a fixed bit length. For example, the value of the transform type parameter 751 may have a bit length of two bits.

[0144] In this example, the image set configuration parameter set includes an upsampling type parameter 752. The upsampling type parameter 752 relates to an upsampling operation. For each image in the image set, an upsampling operation is performed on a representation of the image at a relatively low quality level. An upsampling operation is performed on the representation at a relatively low quality level to derive upsampled data 214. The encoder 108 performs such an upsampling operation to derive residual data 216. The decoder 110 can perform a corresponding upsampling operation to reconstruct the input data 206, i.e., by combining the upsampled data 214 with the residual data 216. The encoder 108 therefore signals to the decoder 110 via the upsampling type parameter 752 which upsampling operation to use. In some examples, the upsampling type parameter 752 specifies an interpolation operation from a plurality of possible interpolation operations. Examples of possible interpolation operations include nearest neighbor interpolation, bilinear interpolation, bicubic interpolation, and Lanczos interpolation. In some examples, the upsampling type parameter specifies whether upsampling is performed in one dimension or two dimensions. In other words, the upsampling type parameter 752 can specify whether to perform one or both of horizontal and vertical upsampling on the representation at the relatively low quality level. In some instances, the configuration parameters related to the upsampling operation can specify whether to perform horizontal upsampling and vertical upsampling on the representation at the relatively low quality level. In some instances, the value of the upsampling type parameter 752 has a fixed bit length. In one instance, the value of the upsampling type parameter 752 has a bit length of three bits. In another instance, the value of the upsampling type parameter 752 has a bit length of four bits.

[0145] In this example, the image set configuration parameter set includes a depth type parameter 753. The depth type parameter 753 relates to bit depth. The bit depth is associated with the residual data 216 for a given image in the image set. The bit depth may also be referred to as color depth. In some examples, the bit depth indicates the number of bits used to indicate the color of a given pixel in a given image. Therefore, the bit depth may indicate how finely the color may be represented in a given image. In some examples, the bit depth indicates the number of bits used for each color component of a given pixel in a given image. In some examples, a given residual element in the residual data 216 for a given image includes multiple color components (e.g., for YUV or RGB data). The bit depth may indicate the number of bits used for each color component of a given residual element. Different color components may have different bit lengths. In some examples, the value of the depth type parameter 753 has a fixed bit length. In one example, the value of the depth type parameter 753 has a bit length of two bits. In another example, the value of the depth type parameter 753 has a bit length of four bits.

[0146] In this example, the image set configuration parameter set includes a temporal processing parameter 754. The temporal processing parameter 754 specifies the degree of temporal processing associated with reconstructing the input data 206 for a given image in the image set. Performing the temporal processing includes using data based on a plurality of images in the image set. In some examples, each of the images in the image set corresponds to a different temporal sample of the video. For example, the temporal processing may use data for different frames of the video (such as data derived from residual data for different frames) to adjust data for a given frame of the video (such as data derived from residual data for the frame). The data based on the plurality of images may indicate the degree of temporal correlation between the residual data for the given image and the residual data for one or more additional images in the image set. The one or more additional images may correspond to one or more earlier temporal samples in the video relative to the temporal sample corresponding to the given image. For example, data derived from at least the residual data associated with a previous frame of the video may be used to reconstruct a current frame of the video via modification of data derived from at least the residual data associated with the current frame.

[0147] In some instances, the data based on the plurality of images includes temporal correlation data. The temporal correlation data may include data based on differences between residual data for a given image and corresponding residual data for one or more additional images. This may be based on differences between data for a current frame and data for a previous frame. For image sequences that exhibit a relatively high degree of temporal correlation, using and / or outputting temporal correlation data may be more efficient than residual data for a given image.

[0148] In some instances, the value of the time processing parameter 754 is based on an optimization metric. An optimization operation may be performed to determine whether the use and / or output of the residual data or the time correlation data for a given image is more efficient. Based on the optimization metric, the value of the time processing parameter 754 may indicate that the decoder 110 reconstructs a given frame of the video based on one of the residual data and the time correlation data. This may reflect the processing that has been performed at the corresponding encoder. The optimization operation may include a rate-distortion analysis, as described in more detail below.

[0149] The value of the temporal processing parameter 754 may have a bit length of one bit. For example, this may indicate whether temporal processing is to be performed (a binary determination), for example, in relation to a given frame of a video. Thus, the temporal processing parameter 754 may be stored in a predetermined bit in a given byte of the configuration data.

[0150] In this example, the image set configuration parameter set includes a resolution type parameter 755 that specifies the resolution of the input data 206 for a given image in the image set. In some examples, the value of the resolution type parameter 755 has a fixed bit length. In one example, the value of the resolution type parameter 755 has a bit length of seven bits.

[0151] In some examples, the image set configuration parameter set includes a custom resolution parameter 756. The custom resolution parameter relates to the resolution of the input data 206. The custom resolution parameter 756 may be optionally included in the image set configuration message 700. Figure 7 This is depicted using dashed lines in FIG. 7 . For example, if it is determined that the resolution of the input data 206 is not one of the predetermined set of values ​​that can be specified by the resolution type parameter 755, then a custom resolution parameter 756 may be included. Data may be stored in the resolution type parameter 755 to indicate that the custom resolution parameter 756 is included in the image set configuration message 700. Such data may include a predetermined bit in the resolution type parameter 755. The predetermined bit thus includes a parameter presence indicator that indicates that the custom resolution parameter 756 is present in the image set configuration message 700.

[0152] The image set configuration parameter set may include Figure 7 The parameters described may be more, fewer or different parameters.

[0153] The image set configuration message 700 corresponds to a plurality of images in an image set. In this example, the image set configuration message 700 corresponds to all images in the image set. Thus, a single image set configuration parameter set applies to each image in the image set. Image set configuration data may also be referred to as "global configuration data".

[0154] The picture set configuration message 700 is output to the decoder 110 to enable the decoder 110 to reconstruct the input data 206 using the upsampled data 214 and the residual data 216 for each picture in the picture set.

[0155] The number of times the image set configuration message 700 is output for an image set is less than the number of images in the image set. Outputting the image set configuration message 700 for a number of images less than the number of images in the image set reduces the amount of information transmitted over a given time period. In this example, the number of times the image set configuration message 700 is output for an image set is one. The image set configuration message 700 may be output only for the first image in the image set. The first image in the image set may be the first image output, received, processed and / or displayed in the image set. Outputting the image set configuration message 700 for the first image in the image set may enable the recipient to receive the image set configuration data as early as possible and / or may enable the recipient to apply the image set configuration data to as many images in the image set as possible. In some examples, the image set configuration message 700 is output together with or in conjunction with the first image in the image set. In some examples, the image set configuration message 700 is output for images other than the first image in the image set. In some examples, the number of times the image set configuration message 700 is output for an image set is greater than one but less than the number of images in the image set. In one example, the image set configuration message 700 is output once for an image set including 75 images.

[0156] In some instances, the number of times the image set configuration message 700 is output for an image set is based on an optimization metric. An example of an optimization metric is a rate-distortion metric. Rate-distortion optimization optimizes the amount of data transmitted for the amount of visual quality loss. The less frequently the image set configuration message 700 is output, the less data is transmitted. However, since the decoder 110 is not provided with the latest configuration data, outputting the image set configuration message 700 too infrequently may result in undesirable quality loss. Therefore, the number of times the image set configuration message 700 is output is optimized to reduce the amount of data sent while reducing any loss in visual quality.

[0157] In some examples, the ratio of the number of times the image set configuration message 700 is output for an image set to the number of images in the image set is less than a predetermined threshold. The predetermined threshold may be based on an optimization metric, such as the rate-distortion metric described above.

[0158] Reference Figure 8 , schematically illustrates an example of a configuration message 800. The configuration message 800 may form part of the configuration data 310. The configuration message 800 may be generated by the encoder 108. Figure 8 Some of the projects depicted are related to Figure 5 The items shown are similar. Therefore, corresponding reference numerals increased by 300 have been used for similar items accordingly.

[0159] Configuration message 800 is a more specific example of configuration message 500. Configuration message 800 includes a header portion 810 and a payload portion 820. Header portion 810 includes a message type parameter 812 and a payload size parameter 814. Payload portion 820 includes a configuration parameter set. The configuration parameter set is related to the processing of residual data 216. In some examples, configuration message 800 is an integer number of bytes long.

[0160] The message type of configuration message 800 is an image configuration message. An image configuration message may correspond only to a given image in an image set. Therefore, a different image configuration message may be generated and / or output for each image in an image set. Payload portion 820 includes an image configuration parameter set.

[0161] In this example, the image configuration parameter set includes a residual usage parameter 851. The value of the residual usage parameter 851 specifies whether the residual data 216 for a given image is to be used to reconstruct the input data 206 for the given image. If the input data 206 is to be reconstructed, then the additional configuration parameter set is included in the image configuration message 800. If the input data 206 is not to be reconstructed, then the additional configuration parameter set is not included in the image configuration message 800. Thus, the additional configuration parameter set is selectively included. Figure 8 This is depicted using dashed lines.

[0162] In this example, the value of the residual usage parameter 851 indicates whether an additional set of configuration parameters is included in the image configuration message 800. In this example, the value of the residual usage parameter 851 may therefore indicate whether an additional set of configuration parameters is included in the image configuration message 800. Therefore, the residual usage parameter 851 may include a parameter presence indicator. The residual usage parameter 851 may be used by the decoder 110 to detect whether the residual data 216 is to be used to reconstruct the input data 206 and whether an additional set of configuration parameters is included in the configuration message 800.

[0163] In some examples, when additional configuration parameter sets are included in the image configuration message 800, the payload portion 820 of the configuration message 800 is 5 bytes long. When additional configuration parameter sets are not included in the configuration message 800, the payload portion 820 of the configuration message 800 may be 1 byte long.

[0164] In this example, the picture configuration parameter set includes quantization data 852. The quantization data 852 is included in another configuration parameter set, which includes an indication by the residual usage parameter 851. The quantization data 852 indicates a quantization level of data that the decoder 110 can use to reconstruct the input data 206 for a given picture using the upsampled data 214.

[0165] Quantization involves approximating a relatively large set of values ​​by a relatively small set of values, for example, by rounding, thresholding, or truncating the values ​​in the relatively large set. The goal of the quantization operation is to reduce the amount of data in the quantized output data compared to the amount of data in the input data to the quantization operation. The quantization level of given data can indicate how aggressively the given data is quantized. The quantization level can indicate the granularity of the quantized data obtained via the quantization process. In other words, the quantization level can indicate how much information is lost due to such quantization. For example, the quantization level can indicate one or more thresholds or bins used for quantization, which indicate how the data is quantized.

[0166] In some examples, the quantization data 852 indicates a quantization level of the residual data 216. Thus, data that can be used by the decoder 216 to reconstruct the input data 206 can be data at a relatively high quality level. The quantization level can indicate to the decoder 110 how to inverse quantize the residual data 216 that has been quantized. For example, at the decoder 110, the received residual data set 216 can have values ​​within a first quantization range and then be expanded to a larger value range after inverse quantization. The quantization data 852 can be used to perform this operation.

[0167] In some examples, the quantization data 852 indicates a quantization level of the correction data. For a given image, the correction data may be used to correct errors when encoding and / or decoding a representation at a relatively low quality level, as described above with reference to FIG. 2. Thus, the data that may be used by the decoder 110 to reconstruct the input data 206 may be data at a relatively low quality level.

[0168] In some examples, the quantization data 852 indicates a first quantization level and a second quantization level. The first quantization level is the quantization level applied to the residual data 216. The second quantization level is the quantization level applied to the correction data. Thus, both the residual data 216 and the correction data may be quantized using separately defined quantization levels indicated by the quantization data 852. The quantization data 852 may therefore include levels at which one or more of the correction data and the residual data are quantized at the encoder (e.g., in a subsequent transform as described herein) and then sent to the decoder to allow inverse quantization and reconstruction of the original values ​​associated with the correction data and the residual data (due to information lost during the quantization process, the reconstructed values ​​may not match the original values ​​prior to quantization at the encoder).

[0169] The quantized data 852 may include an integer number of bytes. In some instances, the quantized data 852 includes two bytes. In some instances, the quantized data 852 includes four bytes.

[0170] In this example, the picture configuration parameter set includes a picture type parameter 853. The picture type parameter 853 is included in another configuration parameter set, which includes the indication indicated by the residual usage parameter 851. The picture type parameter 853 specifies whether a given picture corresponds to a frame or a field. In other words, the picture type parameter 853 indicates whether the given picture is progressive (in which case the picture type parameter 853 specifies a frame) or interlaced (in which case the picture type parameter 853 specifies a field). In the case where the given picture is interlaced, the picture type parameter 853 may also specify a field type, for example, a top field or a bottom field. The picture type parameter 853 may have a bit length of one or two bits.

[0171] In this example, the image configuration parameter set includes a time refresh parameter 854. The time refresh parameter 854 is included in another configuration parameter set, which includes an indication by the residual use parameter 851. The time refresh parameter 854 is used to indicate when to refresh the time data. For example, the value of the time refresh parameter 854 can specify whether to clear the buffer storing the time data. For example, as described herein, the residual data (or transformed residual data) for the current frame of the video can be modified based on the residual data (or transformed residual data) for the previous frame of the video. The latter data can be stored in the buffer and used to modify the previous data. This can be repeated in an iterative manner for each frame. In one case, at the encoder, the contents of the current data set and the buffer can be compared, and the difference can be calculated. At the decoder, the time processing can include: adding the contents of the buffer to the received data to reconstruct the residual data or transform the residual data. In the current case, the value of the time refresh parameter 854 can indicate to the decoder that the data related to the previous frame stored in the buffer is to be cleared. This can effectively indicate that the time processing is not applied (for example, a value of 0 added to the received data will not modify the received data). Temporal data includes data that can be used in temporal processing, for example, data based on multiple images.The bit length of the value of temporal refresh parameter 854 may be one bit.

[0172] Image configuration parameter sets may include Figure 8 The parameters described may be more, fewer or different parameters.

[0173] An image configuration message 800 is output for a plurality of images in the image set. In some instances, the number of images in the plurality of images is equal to the number of images in the image set. In other words, the image configuration message 800 may be output for each image in the image set. In other instances, the number of images in the plurality of images is less than the number of images in the image set. The number of images in the plurality of images may be based on an optimization metric. The optimization metric may be a rate-distortion metric.

[0174] Both the image set configuration message 700 and the image configuration message 800 may be output for a given image set. The number of times the image set configuration message 700 is output for an image set is less than the number of times the image configuration message 800 is output for the image set. For example, the image set configuration message 700 may be output only once for an image set, and the image configuration message 800 may be output for each image in the image set. Thus, the image configuration message 800 is output more frequently than the image set configuration message 700.

[0175] Outputting the image configuration message 800 relatively frequently uses more data than outputting the image configuration message 800 relatively infrequently. However, outputting the image configuration message 800 relatively frequently provides the decoder 110 with up-to-date information that enables the decoder 110 to reconstruct the image with high accuracy and / or high reliability. For example, the set of image configuration parameters stored in the image configuration message 800 may change between different images. There is therefore a trade-off between the amount of data transmitted to the decoder 110 and the accuracy and / or reliability with which the decoder 110 can reconstruct using the received data. For the image set configuration message 700 and the image configuration message 800, the best result of such a trade-off is that the image configuration message 800 is sent more frequently than the image set configuration message 700. Sending the image set configuration parameters at a relatively low frequency and sending the image configuration parameters at a relatively high frequency can therefore reduce the amount of information sent over a given period of time while enabling the decoder 110 to reconstruct at a predetermined level of accuracy and / or reliability.

[0176] Reference Fig. 9 , schematically illustrates an example of a configuration message 900. The configuration message 900 may form a part of the configuration data 310. The configuration message 900 may be generated by the encoder 108. Fig. 9 Some of the projects depicted are related to Figure 5 The items shown are similar. Therefore, corresponding reference numerals increased by 400 have been used for similar items accordingly.

[0177] Configuration message 900 is a more specific example of configuration message 500. Configuration message 900 includes a header portion 910 and a payload portion 920. Header portion 910 includes a message type parameter 912 and a payload size parameter 914. Payload portion 920 includes a configuration parameter set. The configuration parameter set is related to the processing of residual data 216. In some examples, configuration message 900 is an integer number of bytes long.

[0178] The message type of configuration message 900 is a coded data message. The coded data message may correspond to a given image in the image set. Therefore, a different coded data message may be generated and / or output for each image in the image set.

[0179] The payload portion 920 includes a configuration parameter set 950. The payload portion 920 may also include data 960 based on the residual data 216 for a given image. The data 960 based on the residual data 216 may include the residual data 216 itself. In some examples, the data 960 based on the residual data 216 includes transformed residual data. The transformed residual data includes data that is the output of a transform operation performed on the residual data 216. The transform operation may be a directional decomposition operation that exploits correlations between residual elements in the residual data 216.

[0180] In some examples, data 960 based on residual data 216 includes encoded data. The encoded data may be encoded residual data or encoded transformed residual data. The encoded data may be derived using one or both of a run length encoding operation and a Huffman encoding operation. As described above, the residual data or transformed residual data may be quantized prior to encoding.

[0181] In some examples, data 960 based on residual data 216 is stored in one or more variable length elements, each variable length element comprising an integer number of bytes, as described above.

[0182] In this example, the configuration parameter set 950 indicates whether the coded data is derived using a Huffman encoding operation, a run-length encoding operation, or both. In some examples, the configuration parameter set 950 indicates whether the coded data is output for a given image. In some examples, the configuration parameter set 950 relates to the size of the coded data. For example, the configuration parameter may relate to the size of the run-length encoded data and / or the size of the Huffman encoded data.

[0183] The encoded data message 900 is output for a plurality of images in the image set. In some instances, the number of images in the plurality of images is equal to the number of images in the image set. In other words, the encoded data message 900 may be output for each image in the image set. In other instances, the number of images in the plurality of images is less than the number of images in the image set. The number of images in the plurality of images may be based on an optimization metric. The optimization metric may be a rate-distortion metric.

[0184] In some examples, the coded data message 900 is selectively generated and / or output. Whether to generate and / or output a coded data message for a given image may depend on whether the residual data 216 for the given image is to be used to reconstruct the input data 206 for the given image. Whether to reconstruct the input data 206 for the given image may be indicated by the residual use parameter 851 of the corresponding image configuration message 800 for the given image. By selectively outputting the coded data message 900, the amount of information transmitted may be reduced compared to a situation in which the coded data message 900 is not selectively output.

[0185] The configuration message generated by the encoder 108 may have other message types in addition to the above message types. Another exemplary message type is an encoder metadata message. The encoder metadata message includes metadata related to one or more encoder characteristics. Another exemplary message type is a decoder control message. The decoder control message includes instructions for controlling the decoder. The decoder control message may include configuration parameters related to image jitter. Another exemplary message type is a research message. The research message contains a custom payload for research purposes.

[0186] refer to Fig.10 , shows an example of a method 1000 for processing configuration data. The method 1000 may be performed by a device including an encoder (such as the first device 102 described above).

[0187] At item 1010, for each image in the set of images, a first representation of the image at the relatively high quality level is used to derive a representation of the image at the relatively low quality level. The representation at the relatively low quality level is output for processing by decoder 110.

[0188] At item 1020, residual data is output for one or more selected images in the set of images. The residual data may be used to reconstruct a first representation using a second representation of the image at a relatively high quality level, the second representation being based on a representation of the image at a relatively low quality level. The residual data is derived based on the first representation and the second representation.

[0189] In some instances, one or more selected images include all images in the image set. In some instances, one or more selected images are selected based on image content. For example, if the first representation of the first image contains a relatively large amount of visual detail, it can be determined to use and / or output residual data for the first image so that the first representation of the first image can be reconstructed. On the contrary, if the first representation of the second image contains relatively few visual details, for example, if the second image corresponds to a "black screen", it can be determined not to use and / or output residual data for the second image. Using residual data to reconstruct the first representation of the second image may not significantly improve the display visual quality of the second image due to the lack of visual details contained therein. Selectively using and / or outputting residual data can reduce the amount of information transmitted and / or processed over a given time period compared to not selectively using and / or outputting residual data.

[0190] At item 1030, configuration data is output for processing by decoder 110. The configuration data enables decoder 110 to detect whether to use residual data for a given image to reconstruct a first representation for the given image in the image set. The given image may be included in one or more selected images. Configuration data may be output for each image in the image set. The configuration data may include an image configuration message 800. The configuration data may, for example, include residual usage parameters 851. In some instances, the configuration data includes an encoding data message 900.

[0191] In some instances, the number of times configuration data is output for an image set is less than the number of images in the image set. The configuration data may correspond to a plurality of images in the image set. For example, the configuration data may include the image set configuration message 700. In some instances, the number of times configuration data is output for an image set is based on an optimization metric.

[0192] refer to Fig.11 , shows an example of a method 1100 for use in processing a representation of an image. The method 1100 may be performed by the decoder 110 described above.

[0193] At item 1110, data that can be used to derive a representation at a relatively low quality level of an image is received. Before being received by decoder 110, encoder 108 uses a first representation of the image at a relatively high quality level to derive a representation at a relatively low quality level. In some instances, the image is an image in a set of images. The set of images can be part of a video signal. In some instances, the received data is processed to obtain a representation at a relatively low quality level. For example, the received data can be decoded to obtain a representation at a relatively low quality level.

[0194] At item 1120, a determination is made as to whether predetermined configuration data has been received. The predetermined configuration data corresponds to an image.

[0195] If it is determined at item 1120 that predetermined configuration data corresponding to an image has been received, then at item 1130 a first representation of the image is reconstructed. The first representation is reconstructed using the predetermined configuration data. The first representation is reconstructed using a second representation of the image at a relatively high quality level. In some instances, the representation at a relatively low quality level is processed to derive the second representation. For example, the second representation may be derived by upsampling the representation at the relatively low quality level. The first representation may be reconstructed by combining the second representation with the residual data.

[0196] At item 1135, the first representation is output. For example, the first representation can be output for display on a display device.

[0197] If it is determined at item 1120 that the predetermined configuration data corresponding to the image has not been received, the first representation is not reconstructed. Instead, at item 1140, the representation at the relatively low quality level is processed to derive a second representation. The second representation may be derived by upsampling the representation at the relatively low quality level.

[0198] At item 1145, the second representation is output. For example, the second representation can be output for display on a display device.

[0199] In some instances, if it is determined that the predetermined configuration data corresponding to the image has not been received, the second representation is not derived. For example, a representation at a relatively low quality level may be output instead of the second representation. In some instances, data based on the representation at the relatively low quality level is output. The data based on the representation at the relatively low quality level is different from the first representation and the second representation.

[0200] In the case where the image is one of an image in an image set, the predetermined configuration data may correspond to multiple images in the image set. The predetermined configuration data may not be received with each image in the image set. In one instance, the predetermined configuration data is received only with the first image in the image set. The predetermined configuration data may apply to each subsequent image in the image set. Therefore, if the decoder 110 does not receive the first image in the image set, it may also not receive the predetermined configuration data for the image set. The predetermined configuration data may be image set configuration data. The predetermined configuration data may include the image set configuration message 700.

[0201] In the case where the image set is part of a streaming video, the decoder can enter the video stream halfway through the image set. In this case, although the first representation is not reconstructed due to the failure to receive the predetermined configuration data, a representation at a relatively high-quality level, i.e., a second representation, can still be output. Compared to the case where the representation at the relatively high-quality level is not output, outputting the second representation for display reduces the potential loss of visual quality due to the failure to receive the predetermined configuration data. Compared to outputting the first representation (which can be considered as part of the "full operating mode"), outputting the second representation can be considered as part of the "minimum operating mode". In the case where the decoder 110 enters the video stream halfway through the image set, the "minimum operating mode" can be used for the remainder of the image set. Of course, when the first image in the subsequent image set is received, the predetermined configuration data for the subsequent image set is also received. Therefore, the subsequent image set can be processed according to the "full operating mode".

[0202] refer to Fig.12, schematically illustrating an example of a data processing technique 1200. Some or all of the data processing technique 1200 may be performed by a device including an encoder (such as the first device 102 described above) and / or by a device including a decoder (such as the second device 104).

[0203] The first Low Complexity Enhancement Video Coding (LCEVC) standard is now being published as ISO / IEC 23094-2, due for release in October 2021.

[0204] The examples that will now be described add extensions to the first LCEVC standard to provide a new LCEVC standard, which may be referred to herein as an "extended" LCEVC standard.

[0205] The extended LCEVC standard aims to maintain the low complexity of the first LCEVC standard.

[0206] The extended LCEVC standard is configured so that any extension of the LCEVC bitstream of the first LCEVC standard maintains backward compatibility with the first LCEVC standard. For example, an LCEVC decoder (which may be referred to as "traditional", "unextended" or "old" decoder hereinafter) designed to decode the LCEVC bitstream according to the first LCEVC standard should be able to decode the bitstream (which may be referred to as "extended" bitstream) generated according to the extended LCEVC standard at least to some extent (e.g., without crashing). In other words, the traditional LCEVC decoder configured to decode the first LCEVC standard should also be able to process the bitstream generated according to the extended LCEVC standard and output data suitable for display on the screen. The output data may not be exactly the same as the data that may be output by the "extended" LCEVC decoder (which may also be referred to as the "updated" (LCEVC) decoder) (i.e., the decoder configured to decode the extended LCEVC standard). However, the output data is at least data suitable for displaying an image. The less effective result is that, in such scenarios, a conventional LCEVC decoder configured to decode the first LCEVC standard crashes when receiving and / or processing an extended bitstream generated according to the extended standard. In such situations, users of such devices will not be able to watch any video at all.

[0207] The examples described herein address these issues.

[0208] Although the examples described herein relate to the first LCEVC standard and the extended LCEVC standard, it should be understood that they are equally applicable to any different (eg, future) versions of the LCEVC standard.

[0209] The examples described below use the general principle that LCEVC (data) blocks are independent of each other. For example, this includes signaling blocks (e.g., configuration blocks). The term "block" is generally used in this article to refer to a group of one or more bytes with semantics associated with a byte. A block may include an integer number of bytes.

[0210] According to an example, a (e.g., LCEVC) encoder is provided that is configured to generate a configuration block 1210 (i.e., a configuration data block). In some examples, the configuration block 1210 includes global configuration data. The configuration block 1210 instructs the decoder to use a given configuration. The configuration block 1210 signals the decoder to select a given configuration from a plurality of possible configurations. In an example, the configuration block 1210 signals the selection via a configuration value 1220. The encoder may generate one or more configuration blocks 1210. Each of the one or more configuration blocks 1210 may include one or more configuration values ​​1220.

[0211] By way of specific and non-limiting example, the configuration block 1210 may include a one-byte block. The configuration block 1210 may include a one-bit 'time R symbol' parameter, a one-bit 'time enable' parameter, a three-bit 'upsample' parameter, a one-bit 'filter' parameter, and a 2-bit 'sml' parameter. In this example, the three-bit upsample parameter may therefore take one of eight different values, some or all of which may indicate different upsample types. Again, by way of specific and non-limiting example, an upsample parameter value of '0' may indicate a 'nearest' upsample type, '1' may indicate a 'linear' upsample type, '2' may indicate a 'three-time' upsample type, '3' may indicate a 'modified three-time' upsample type, '4' may indicate an 'adaptive three-time' type, '5' may indicate a first reserved value, '6' may indicate a second reserved value, and '7' may indicate an unspecified upsample type.

[0212] In principle, the type of upsampler associated with each upsampling parameter value can be updated over time. For example, an upsampler parameter value of '5' can be changed from reserved to a specific type of upsampler in the extended LCEVC standard. Although an extended legacy decoder would then be able to use that specific type of upsampler, a legacy decoder would interpret the upsampler parameter value '5' as a reserved value and might crash.

[0213] Thus, in an example, the encoder is also configured to generate a new extended configuration block 1240 (i.e., an extended configuration data block). In some examples, the extended configuration block 1240 includes global configuration data. The extended configuration block 1240, when processed by the decoder, indicates one or more predetermined actions to be performed related to the selection of the configuration indicated by the configuration block 1210. In an example, the extended configuration block 1240 signals the action via the extended configuration value 1250. The encoder may generate one or more extended configuration blocks 1240. Each of the one or more extended configuration blocks 1240 may include one or more extended configuration values ​​1250.

[0214] By way of specific and non-limiting example, the extended configuration block 1240 may include a one-byte block. In addition to one or more additional parameters, the extended configuration block 1240 may also include three upsampling parameters. In this example, the three upsampling parameters may therefore take one of eight different values. In some examples, each of the three upsampling parameters indicates a different upsampling type. However, as will be explained in more detail below, in other examples, at least one of the upsampling parameters does not therefore indicate an upsampling type.

[0215] The configuration block 1210 may be used to signal (i.e., to a decoder) the selection of a particular upsampler. Specifically, the value 1220 of the configuration block 1210 may signal (i.e., to a decoder) the selection of a first (specific) upsampler. For example, a value 1220 of the configuration block 1210 equal to '1' may cause the decoder to select a linear upsampler. A value 1220 of the configuration block 1210 equal to '2' may cause the decoder to select a cubic upsampler.

[0216] The extended configuration block 1240 may be used to perform one or more predetermined actions related to the configuration signaled by the configuration block 1210. Examples of such predetermined actions include, but are not limited to, updating, modifying, adjusting, using, overwriting, and ignoring the configuration signaled by the configuration block 1210. As used herein, the term "update" does not mean modifying a configuration; updating a configuration may result in the same or a different configuration.

[0217] For example, the encoder may be configured to generate a configuration block 1210 having a value 1220 equal to '1' and an extended configuration block 1240 having a value 1250 equal to '2'. In this instance, a properly configured (i.e., updated) decoder may update the configuration value 1220 (i.e., the value 1220'1' of the configuration block 1210) to an extended configuration value 1250 (i.e., the value 1250'2' of the extended configuration block 1240) to generate an updated configuration value. For example, the decoder may increase the configuration value 1220'1' by the extended configuration value 1250'2' to obtain an updated configuration value '3' and a corresponding updated configuration. On the other hand, a decoder that has not yet been configured to process the extended configuration block 1240 ignores the extended configuration block 1240 and continues decoding according to the configuration value 1220 (i.e., '1'). In some instances, the extended configuration value 1250 is selected so that the updated configuration value falls outside the range of the values ​​in the configuration block 1210, thereby indicating that the value in the extended configuration block 1240 should be used. For example, where configuration value 1220 is '1' and extended configuration value 1250 is '7', the updated configuration value will fall outside the range of values ​​in configuration block 1210 ('0' to '7') and the appropriate value in extended configuration block 1240 (ie, '1') will be used.

[0218] In another example, in the case where the extended configuration block 1240 has a value 1250 equal to '0' (an instance of a first predetermined extended configuration value), the appropriately configured (i.e., updated) decoder uses the configuration value 1220 of the configuration block 1210 (rather than the different value 1250 in the extended configuration block 1240) to select the upsampling type. Therefore, the decoder effectively ignores the upsampling type in the extended configuration block 1240 and uses the upsampling type from the configuration block 1210. However, in the case where the extended configuration block 1240 has a non-zero value 1250 (i.e., the value 1250 is not equal to '0') (an instance of a second predetermined extended configuration value), the appropriately configured (i.e., updated) decoder uses the extended configuration value 1250 of the extended configuration block 1240. Therefore, the decoder effectively ignores the upsampling type in the configuration block 1210 and uses the upsampling type from the extended configuration block 1240.

[0219] In some instances, a suitably configured (i.e., updated) decoder determines whether the extended configuration value 1250 has one or more first predetermined extended configuration values ​​(in this instance, only '0'). If the decoder determines that the extended configuration value 1250 does not have one or more first predetermined extended configuration values ​​(in this instance, only '0') - in other words, the extended configuration value 1250 has one or more second predetermined extended configuration values ​​(in this instance, '1', '2', '3', '4', '5', '6', or '7'), the decoder does not determine the upsampler type parameter value of the configuration block 1210 because it will not be used. In other instances, the decoder can still determine the upsampler type parameter value of the configuration block 1210.

[0220] In some examples, configuration block 1210 and extended configuration block 1240 refer to (ie, signal to a decoder) a type of upsampler.

[0221] In such an example, a configuration value 1220 of '1' may signal to the decoder to use a linear upsampler. If the extended decoder then adjusts the value to '3', per the example above, the decoder may continue to decode the data using a different (eg, modified cubic upsampler).

[0222] In this example, it can therefore be seen that the conventional decoder can still decode the data. This is because the conventional decoder proceeds with a configuration value 1220 of '1', as signaled by configuration block 1210. However, the conventional decoder uses one upsampler type to decode the data, while the decoder according to the example (i.e., the extended encoder) may use another upsampler type to decode the data. The decoding may only 'possibly' use another upsampler type because in some examples the extended decoder may still use the same upsampler type as the conventional decoder.

[0223] According to an example, a bitstream is provided that includes a configuration block 1210. The bitstream may include an extended configuration block 1240. Generally speaking, the configuration block 1210 and the extended configuration block 1240 are associated with a common (or "shared") configuration. The extended configuration block 1240 may be configured to adjust and / or modify the configuration signaled by the configuration block 1210.

[0224] A conventional decoder (e.g., a decoder configured to decode according to a first LCEVC standard) is configured to process a configuration block 1210. In an example, a conventional decoder is configured to process a configuration block 1210 to determine a configuration, e.g., a value 1220 and / or setting and / or selection for a configuration. An unextended decoder ignores the extension block 1240. The unextended decoder therefore decodes the bitstream according to the configuration block 1210. The decoding performed by the conventional decoder is not affected by the presence of the extension configuration block 1250 in the received bitstream. The bitstream may be a bitstream that complies with LCEVC.

[0225] like Fig.12 As shown, in an example, the extended configuration block 1240 is separated from the configuration block 1210 in the bitstream. In this particular example, the extended configuration block 1240 follows the configuration block 1210 and all other configuration blocks 1210 in the bitstream, i.e., the extended configuration block 1240 is at the end of the bitstream. In this way, a conventional decoder can stop reading the configuration at the end of the configuration block 1210, thereby ignoring the extended configuration block 1240. However, in other examples, the extended configuration block 1240 can be at a different position in the bitstream.

[0226] According to an example, a decoder configured to process the extension of configuration block 1210 is provided. In an example, the extended decoder is configured to process configuration block 1210 to determine the configuration. Configuration block 1210 is associated with the configuration. The extended decoder is configured to process extended configuration block 1240. By processing extended configuration block 1240, the extended decoder updates the configuration (indicated by configuration value 1220) or performs another predetermined action to determine the "extended" (or "extended") configuration. The extended decoder is configured to decode according to the extended configuration. In some examples, the extended decoder processes the extended configuration block 1240 to cover and / or exceed the configuration of configuration block 1210.

[0227] In an example, the extended decoder can increase the configuration value 1220 associated with the configuration block 1210 in response to processing the extended configuration block 1240. In an example, the increment is indicated by the extended configuration value 1250 of the extended configuration block 1240.

[0228] In some instances where increments are used, the extended configuration value 1250 may be selected so that the increase does not result in a reserved value being derived, as this could cause the decoder to crash.

[0229] Thus, the examples may be used for configurations and / or tools with 'fields' and / or 'options' already in the bitstream. This may be, for example, selection of a type of upsampler to use, selection of a time setting and / or selection of a filter.

[0230] Thus, the examples can be used to provide new options for such configurations and / or settings without crashing any traditional decoder. In a preferred example, the encoder or decoder is part of a layer-based hierarchical coding scheme or format. Examples of layer-based hierarchical coding schemes include LCEVC: MPEG-5 Part 2 LCEVC ("Low Complexity Enhanced Video Coding") and VC-6: SMPTE VC-6ST-2117, the former described in PCT / GB2020 / 050695 (and related standard documents) published as WO 2020 / 188273, and the latter described in PCT / GB2018 / 053552 (and related standard documents) published as WO 2019 / 111010, all of which are incorporated herein by reference. However, the concepts shown herein are not necessarily limited to these specific hierarchical coding schemes. Additional examples are described in WO2018 / 046940, which is incorporated herein by reference.

[0231] LCEVC (Low Complexity Enhancement Video Coding) is a standardized coding method described in a standard specification document, which includes the text of ISO / IEC 23094-2 Version 1 Low Complexity Enhancement Video Coding published in October 2021, which is incorporated herein by reference. LCEVC can be used to efficiently encode and decode images, and additional specific implementation details of these types of encoders and decoders are described in earlier published patent applications GB1615265.4 and WO2020188273, each of which is incorporated herein by reference.

[0232] For additional specific implementation details, for example, the LCEVC bitstream in published patent applications WO2020 / 089617, WO2020 / 089615, WO2020 / 089616, and WO2020 / 089614, each of which is incorporated herein by reference.

[0233] refer to Fig.13 , a schematic block diagram of an example of device 1300 is shown.

[0234] In an example, the device 1300 includes an encoder. In another example, the device 1300 includes a decoder.

[0235] Examples of apparatus 1300 include, but are not limited to, a mobile computer, a personal computer system, a wireless device, a base station, a telephone device, a desktop computer, a laptop computer, a notebook computer, a netbook computer, a mainframe computer system, a handheld computer, a workstation, a network computer, an application server, a storage device, a consumer electronic device such as a camera, a camcorder, a mobile device, a video game console, a handheld video game device, or generally any type of computing or electronic device.

[0236] In this example, device 1300 includes one or more processors 1301 configured to process information and / or instructions. One or more processors 1301 may include a central processing unit (CPU). One or more processors 1301 are coupled to bus 1302. Operations performed by one or more processors 1301 may be performed by hardware and / or software. One or more processors 1301 may include multiple co-located processors or multiple fully remotely located processors.

[0237] In this example, device 1300 includes computer usable volatile memory 1303 configured to store information and / or instructions for one or more processors 1301. Computer usable volatile memory 1303 is coupled to bus 1302. Computer usable volatile memory 1303 may include random access memory (RAM).

[0238] In this example, device 1300 includes computer usable non-volatile memory 1304 configured to store information and / or instructions for one or more processors 1301. Computer usable non-volatile memory 1304 is coupled to bus 1302. Computer usable non-volatile memory 1304 may include read only memory (ROM).

[0239] In this example, device 1300 includes one or more data storage units 1305 configured to store information and / or instructions. One or more data storage units 1305 are coupled to bus 1302. One or more data storage units 1305 may include, for example, magnetic or optical disks and disk drives or solid state drives (SSDs).

[0240] In this example, the device 1300 includes one or more input / output (I / O) devices 1306 configured to communicate information to and / or from one or more processors 1301. One or more I / O devices 1306 are coupled to bus 1302. One or more I / O devices 1306 may include at least one network interface. At least one network interface may enable the device 1300 to communicate via one or more data communication networks. Examples of data communication networks include, but are not limited to, the Internet and a local area network (LAN). One or more I / O devices 1306 may enable a user to provide input to the device 1300 via one or more input devices (not shown). The one or more input devices may include, for example, a remote control, one or more physical buttons, etc. One or more I / O devices 1306 may enable information to be provided to a user via one or more output devices (not shown). The one or more output devices may, for example, include a display screen.

[0241] Various other entities are described for the device 1300. For example, when present, the operating system 1307, the data signal processing module 1308, one or more additional modules 1309, and the data 1310 are shown as residing in one or a combination of the computer usable volatile memory 1303, the computer usable non-volatile memory 1304, and the one or more data storage units 1305. The data signal processing module 1308 can be implemented by computer program code stored in a memory location within the computer usable non-volatile memory 1304, a computer readable storage medium within the one or more data storage units 1305, and / or other tangible computer readable storage media. Examples of tangible computer readable storage media include, but are not limited to: optical media (e.g., CD-ROM, DVD-ROM, or Blu-ray), flash memory cards, floppy disks, or hard disks, or any other medium capable of storing computer readable instructions such as firmware or microcode in at least one ROM or RAM or a programmable ROM (PROM) chip or as an application specific integrated circuit (ASIC).

[0242] The device 1300 may thus include a data signal processing module 1308 that may be performed by one or more processors 1301. The data signal processing module 1308 may be configured to include instructions for implementing at least some of the operations described herein. During operation, the one or more processors 1301 initiate, run, execute, interpret or otherwise perform the instructions in the signal processing module 1308.

[0243] Although at least some aspects of the examples described herein with reference to the drawings include computer processes performed in a processing system or processor, the examples described herein also extend to computer programs, such as computer programs on or in a carrier, which are suitable for putting the examples into practice. The carrier can be any entity or device capable of carrying the program.

[0244] It should be understood that the device 1300 may include more, fewer, and / or more Fig.13 The components shown are different components.

[0245] Device 1300 may be located at a single location or may be distributed across multiple locations. Such locations may be local or remote.

[0246] The techniques described herein may be implemented in software or hardware, or may be implemented using a combination of software and hardware. They may include configuring a device to implement and / or support any or all of the techniques described herein.

[0247] Examples described herein relate to syntax for configuration data for use involving processing of residual data, which can be used in image reconstruction. Syntax provides a processing framework to enable an encoder to obtain and output configuration data and to enable a decoder to receive and process the configuration data. The syntax framework is conducive to the generation and processing of configuration messages such as those described above. The decoder uses the received configuration data to reconstruct the representation of an image at a high-quality level, such as a part of a video stream. The syntax framework described herein allows a decoder to perform such reconstruction with a relatively high degree of efficiency, reliability, and accuracy.

[0248] Compared to some known systems, the syntactic framework described herein enables the overhead cost of transmitting data to be reduced. When residual data is not used to reconstruct a representation at a high quality level for a given image, the syntactic framework described herein may involve up to 3 bytes of overhead for the given image. When residual data is used to reconstruct a representation at a high quality level, up to 43 bytes of overhead cost may be involved for a given image. This is in contrast to some known systems, which may involve overhead costs between 108 and 188 bytes per image. Therefore, by using the syntactic framework described herein, the amount of information generated, stored, processed and / or transmitted in a given time period is significantly reduced.

[0249] It should be understood that any feature described with respect to any embodiment may be used alone or in combination with other features described, and may also be used in combination with one or more features of any other embodiment, or in combination with any combination of any other embodiment. In addition, equivalents and modifications not described above may also be employed without departing from the scope of the invention as defined by the appended claims.

[0250] Further features are listed in the following numbered clauses:

[0251] 1. A method comprising:

[0252] for each image in the set of images, deriving a representation of the image at a first quality level in a hierarchical structure having a plurality of quality levels using a first representation of the image at a second higher quality level in the hierarchical structure;

[0253] For each picture in the set of pictures, outputting the representation of the picture at the first quality level for processing by a decoder; and

[0254] Output configuration data for processing by the decoder to enable the decoder to detect whether to use residual data for a given image in the image set to reconstruct the first representation of the given image, the residual data being usable to reconstruct the first representation using a second representation of the image at the second quality level, the second representation being based on the representation of the image at the first quality level, the residual data being derived based on the first representation and the second representation.

[0255] 2. The method according to clause 1, comprising: outputting the residual data for one or more selected images in the image set.

[0256] 3. A method according to clause 1 or clause 2, wherein the configuration data is output for each image in the set of images.

[0257] 4. A method according to clause 1 or clause 2, wherein the number of times the configuration data is output for the image set is less than the number of images in the image set.

[0258] 5. A method according to any one of clauses 1 to 4, wherein the one or more selected images include all images in the image set.

[0259] 6. A method according to any one of clauses 1 to 5, wherein the configuration data corresponds to a plurality of images in the image set.

[0260] 7. The method according to any one of clauses 1 to 6, comprising: performing byte-wise processing of the configuration data.

[0261] 8. A method according to any one of clauses 1 to 7, wherein, in the case where the configuration data indicates that the residual data for the given image is not to be used to reconstruct the first representation of the given image, the configuration data is at most three bytes long.

[0262] 9. A method according to any one of clauses 1 to 8, wherein, in the case where the configuration data indicates that the residual data for the given image is to be used to reconstruct the first representation of the given image, the configuration data is at most forty-three bytes long.

[0263] 10. A method comprising:

[0264] for each picture in the set of pictures, obtaining a representation of the picture at a first quality level in a hierarchical structure having a plurality of quality levels, the representation of the picture at the first quality level having been derived by an encoder using a first representation at a second, higher quality level in the hierarchical structure;

[0265] detecting, based on the received configuration data, whether to reconstruct the first representation of a given image in the set of images using residual data for the given image, the residual data being usable to reconstruct the first representation using a second representation of the image at the second quality level, the second representation being based on the representation of the image at the first quality level, the residual data being derived based on the first representation and the second representation; and

[0266] For each image in the set of images, a representation of the image at the first quality level is processed based on the received configuration data.

[0267] 11. A method according to clause 10, comprising: receiving the residual data for one or more selected images.

[0268] 12. A method according to clause 10, comprising: in response to detecting that the residual data for the given image is not used to reconstruct the first representation of the given image:

[0269] processing the representation of the given image at the first quality level by upsampling the representation of the image at the first quality level to obtain the second representation of the given image at the second quality level; and

[0270] The second representation of the given image is output.

[0271] 13. A method according to any of clauses 10 to 12, comprising: in response to detecting that the residual data for the given image is to be used to reconstruct the first representation of the given image:

[0272] processing the representation of the given image at the first quality level by upsampling the representation of the image at the first quality level to obtain the second representation of the given image at the second quality level;

[0273] obtaining the first representation of the given image using the residual data for the given image and the second representation of the given image; and

[0274] The first representation of the given image is output.

[0275] 14. A method according to any of clauses 10 to 13, comprising receiving the configuration data for each image in the set of images.

[0276] 15. A method according to any of clauses 10 to 14, wherein the configuration data corresponds to a plurality of images in the set of images.

[0277] 16. A method according to any of clauses 10 to 15, wherein the one or more selected images include all images in the set of images.

[0278] 17. A method according to any of clauses 10 to 16, wherein the given image is included in the one or more selected images.

[0279] 18. A method for processing configuration data, the method comprising:

[0280] obtaining residual data based on a first representation of an image at a first quality level in a hierarchical structure having a plurality of quality levels and a second representation of the image at the first quality level, the second representation being based on a representation of the image at a second, lower quality level in the hierarchical structure, the residual data usable by a decoder to reconstruct the first representation using the second representation;

[0281] generating configuration data relating to processing of the residual data, the configuration data being usable by the decoder to reconstruct the first representation, wherein the configuration data comprises a parameter presence indicator specifying whether a given set of configuration parameters is included in a given portion of the configuration data, wherein the given set of configuration parameters has a predetermined format; and

[0282] The configuration data is output for processing by the decoder to enable the decoder to reconstruct the first representation using the second representation and the residual data.

[0283] 19. A method according to clause 18, wherein the given configuration parameter set includes configuration parameters having a variable bit length and stored in a variable length element, the variable length element including at least one given byte having one or more predetermined bits, the one or more predetermined bits being arranged to indicate whether the variable length element includes one or more additional bytes in addition to the at least one given byte.

[0284] 20. A method according to clause 18 or clause 19, wherein the given set of configuration parameters comprises configuration parameters relating to a resolution of the first representation of the image.

[0285] 21. A method according to clause 18 or clause 19, wherein the parameter presence indicator is included in a residual usage parameter, wherein the residual usage parameter is usable by the decoder to detect whether to use the residual data to reconstruct the first representation of the image.

[0286] 22. A method according to any of clauses 18 to 21, wherein the given set of configuration parameters comprises quantization data indicating a quantization level of data usable by the decoder to reconstruct the first representation of the image.

[0287] 23. A method according to any of clauses 18 to 22, wherein the given set of configuration parameters comprises a picture type parameter specifying whether the first representation of the picture relates to progressive video or interlaced video.

[0288] 24. A method according to any of clauses 18 to 23, wherein in case the first representation of the image relates to interlaced video, the picture type parameter also specifies a field type.

[0289] 25. A method according to any of clauses 18 to 24, wherein the given set of configuration parameters comprises a version number according to which the decoder is configured to operate.

[0290] 26. A method according to any of clauses 18 to 25, wherein the parameter presence indicator comprises at least one predetermined bit in a predetermined byte of the configuration data.

[0291] 27. A method according to any of clauses 18 to 26, comprising byte-wise processing of the configuration data.

[0292] 28. A method for processing configuration data, the method comprising:

[0293] receiving configuration data relating to processing of residual data, the residual data having been obtained, for a given image in the set of images, based on a first representation of the image at a first quality level in a hierarchical structure having a plurality of quality levels and a second representation of the image at the first quality level, the second representation being based on a representation of the image at a second, lower quality level in the hierarchical structure, the residual data being usable to reconstruct the first representation using the second representation; and

[0294] processing the configuration data to reconstruct the first representation for the given image using the second representation and the residual data,

[0295] The configuration data includes a parameter presence indicator, the parameter presence indicator specifies whether a given configuration parameter set is included in a given portion of the configuration data, the given configuration parameter set having a predetermined format.

[0296] 29. A method according to clause 28, wherein the given configuration parameter set includes configuration parameters having a variable bit length and stored in a variable length element, the variable length element including at least one given byte having one or more predetermined bits, the one or more predetermined bits being arranged to indicate whether the variable length element includes one or more additional bytes in addition to the at least one given byte.

[0297] 30. A method according to clause 28 or clause 29, wherein the given set of configuration parameters comprises configuration parameters relating to a resolution of the first representation of the image.

[0298] 31. A method according to clause 28 or clause 29, wherein the parameter presence indicator is included in a residual usage parameter, wherein the residual usage parameter can be used to detect whether to use the residual data to reconstruct the first representation of the image.

[0299] 32. A method according to any of clauses 28 to 31, wherein the given set of configuration parameters comprises quantization data indicating a quantization level of data usable by the decoder to reconstruct the first representation of the image.

[0300] 33. A method according to any of clauses 28 to 32, wherein the given set of configuration parameters comprises a picture type parameter specifying whether the first representation of the picture relates to progressive video or interlaced video.

[0301] 34. A method according to any of clauses 28 to 33, wherein in case the first representation of the image relates to interlaced video, the picture type parameter also specifies a field type.

[0302] 35. A method according to any of clauses 28 to 34, wherein the given set of configuration parameters comprises a version number.

[0303] 36. A method according to any of clauses 28 to 35, wherein the parameter presence indicator comprises at least one predetermined bit in a predetermined byte of the configuration data.

[0304] 37. A method according to any of clauses 28 to 36, comprising byte-wise processing of the configuration data.

[0305] 38. A method comprising:

[0306] generating an image configuration message including a header portion and a payload portion, the payload portion including an image configuration parameter set, the image set configuration parameters relating to processing of residual data;

[0307] Output the image configuration message,

[0308] The image configuration message only corresponds to a given image in the image set.

[0309] 39. A method according to clause 38, wherein the image set configuration parameter set includes a residual usage parameter, wherein the value of the residual usage parameter specifies whether residual data for the given image is to be used to reconstruct input data for the given image.

[0310] 40. A method according to clause 39, wherein the image set configuration parameters include quantisation data indicating a quantisation level of data that can be used to reconstruct the input data for the given image using upsampled data.

[0311] 41. A method according to clause 39 or clause 40, wherein the image set configuration parameters include a picture type parameter specifying whether the given image is progressive or interlaced.

[0312] 42. A method according to any of clauses 38 to 41, comprising generating and / or outputting a different image configuration message for each image in the set of images.

[0313] 43. A method comprising:

[0314] receiving an image configuration message including a header portion and a payload portion, the payload portion including an image configuration parameter set, the image configuration parameter set relating to processing of residual data;

[0315] Use the received image configuration message to perform image reconstruction,

[0316] The image configuration message only corresponds to a given image in the image set.

[0317] 44. A method according to clause 43, wherein the set of image set configuration parameters includes a residual usage parameter, wherein a value of the residual usage parameter specifies whether residual data for the given image is to be used to reconstruct input data for the given image.

[0318] 45. A method according to clause 43, wherein the image set configuration parameters include quantisation data indicating a quantisation level of data that can be used to reconstruct the input data for the given image using upsampled data.

[0319] 46. ​​A method according to clause 44 or clause 45, wherein the picture set configuration parameters include a picture type parameter specifying whether the given picture is progressive or interlaced.

[0320] 47. A method as described in any of clauses 43 to 46, comprising receiving a different image configuration message for each image in the set of images.

[0321] 48. A method comprising:

[0322] generating a picture set configuration message including a header portion and a payload portion, the payload portion including a picture set configuration parameter set, the picture set configuration parameter set relating to processing of residual data;

[0323] Output the image set configuration message.

[0324] The image set configuration message corresponds to a plurality of images in the image set.

[0325] 49. A method according to clause 48, wherein the set of picture set configuration parameters includes a transform type parameter relating to a transform operation performed on the residual data.

[0326] 50. A method according to clause 48 or clause 49, wherein the picture set configuration parameter set comprises an upsampling type parameter relating to the upsampling operation performed.

[0327] 51. A method according to any of clauses 48 to 50, wherein the picture set configuration parameter set comprises a depth type parameter relating to a bit depth associated with the residual data.

[0328] 52. A method according to any of clauses 48 to 51, wherein the image set configuration parameter set comprises a resolution type parameter specifying a resolution of input data for a given image in the image set.

[0329] 53. A method according to clause 52, wherein the value of the resolution type parameter has a fixed bit length.

[0330] 54. A method according to clause 53, wherein the value of the resolution type parameter has a bit length of seven bits.

[0331] 55. A method according to any of clauses 52 to 54, wherein the image set configuration parameter set includes a custom resolution parameter relating to the resolution of the input data for the given image in the image set.

[0332] 56. A method comprising:

[0333] receiving a picture set configuration message including a header portion and a payload portion, the payload portion including a picture set configuration parameter set, the picture set configuration parameter set relating to processing of residual data;

[0334] Use the received image set configuration message to perform image reconstruction,

[0335] The image set configuration message corresponds to a plurality of images in the image set.

[0336] 57. A method according to clause 56, wherein the set of picture set configuration parameters includes a transform type parameter relating to a transform operation performed on the residual data.

[0337] 58. A method according to clause 56 or clause 57, wherein the picture set configuration parameter set includes an upsampling type parameter relating to the upsampling operation performed.

[0338] 59. A method according to any of clauses 56 to 58, wherein the picture set configuration parameter set comprises a depth type parameter relating to a bit depth associated with the residual data.

[0339] 60. A method according to any of clauses 56 to 59, wherein the set of image set configuration parameters comprises a resolution type parameter specifying a resolution of input data for a given image in the image set.

[0340] 61. A method according to clause 60, wherein the value of the resolution type parameter has a fixed bit length.

[0341] 62. A method according to clause 61, wherein the value of the resolution type parameter has a bit length of seven bits.

[0342] 63. A method according to any of clauses 60 to 62, wherein the image set configuration parameter set includes a custom resolution parameter relating to the resolution of the input data for the given image in the image set.

[0343] 64. An apparatus configured to perform a method according to any one of clauses 1 to 63.

[0344] 65. A computer program comprising instructions which, when executed, cause an apparatus to carry out a method according to any one of clauses 1 to 63.

[0345] 66. A computer readable medium comprising a computer program according to clause 65.

Claims

1. A method comprising: obtaining a configuration block including configuration values, the configuration block being operable to reconstruct a first representation of an image at a first quality level in a hierarchical structure having a plurality of quality levels using a second representation of the image at the first quality level, the second representation being based on a representation of the image at a second, lower quality level in the hierarchical structure; as well as Get an extended configuration block containing extended configuration values, Wherein the configuration value and / or the extended configuration value may be used by a decoder to reconstruct the first representation of the image at the first quality level. 2 . The method of claim 1 , wherein the extended configuration value indicates one or more predetermined actions to be performed by the decoder in relation to the configuration value.

3. The method of claim 2, wherein the one or more predetermined actions include: An updated configuration value is derived by updating the configuration value based on the extended configuration value, the updated configuration value being usable by the decoder to reconstruct the first representation of the image at the first quality level.

4. The method of claim 3, wherein the updating of the configuration value comprises: The configuration value is adjusted based on the extended configuration value.

5. The method according to claim 3 or 4, wherein the updating of the configuration value comprises: The configuration value is increased based on the extended configuration value.

6. The method according to claim 2, wherein: When the extended configuration value has one or more first predetermined extended configuration values, the one or more predetermined actions include reconstructing the first representation of the image at the first quality level using the configuration values.

7. The method according to claim 2 or 6, wherein: When the extended configuration value has one or more second predetermined extended configuration values, the one or more predetermined actions include reconstructing the first representation of the image at the first quality level using the extended configuration values.

8. The method according to any one of claims 1 to 7, wherein the configuration value and / or the extended configuration value indicates a type of upsampler to be used for reconstructing the first representation of the image at the first quality level.

9. The method according to any one of claims 1 to 8, wherein the configuration value and / or the extended configuration value indicates a type of filter to be used for reconstructing the first representation of the image at the first quality level.

10. The method according to any one of claims 1 to 9, wherein the configuration value and / or the extended configuration value indicates a time setting to be used for reconstructing the first representation at the first quality level of the image.

11. The method according to any one of claims 1 to 10, wherein the obtaining of the configuration block comprises generating the configuration block, and wherein the obtaining of the extended configuration block comprises generating the extended configuration block.

12. The method according to any one of claims 1 to 11, wherein the obtaining of the configuration block comprises receiving the configuration block, and wherein the obtaining of the extended configuration block comprises receiving the extended configuration block.

13. A method according to any one of claims 1 to 12, wherein the configuration block can be used by a first type of decoder and a second type of decoder to reconstruct the first representation of the image at the first quality level, wherein an extended configuration block can be used by the second type of decoder to reconstruct the first representation of the image at the first quality level, and wherein the extended configuration block cannot be used by the first type of decoder to reconstruct the first representation of the image at the first quality level.

14. The method of claim 13, wherein the second type of decoder is a newer type of decoder than the first type of decoder.

15. A method comprising: Storing and / or processing a bitstream comprising a configuration block and an extended configuration block according to any one of claims 1 to 14.

16. A bit stream comprising the configuration block and the extended configuration block according to any one of claims 1 to 14.

17. The bitstream of claim 16, wherein the extended configuration block follows the configuration block in the bitstream.

18. The bitstream according to claim 16 or 17, wherein the extended configuration block is at the end of the bitstream.

19. An apparatus configured to perform the method according to any one of claims 1 to 15.

20. A computer program comprising instructions which, when executed, cause an apparatus to carry out a method according to any one of claims 1 to 15.

21. A computer readable medium comprising a computer program according to claim 20.

22. An encoder configured to: generating a configuration block comprising a configuration value, wherein the configuration value indicates a configuration to be selected by a decoder; and An extended configuration block is generated that includes extended configuration values ​​indicating one or more predetermined actions to be performed by the decoder related to the selected configuration.

23. A decoder configured to: Gets a configuration block containing configuration values selecting a configuration from a plurality of configurations based on the configuration value; Get an extended configuration block including extended configuration values; performing one or more actions associated with the selected configuration based on the extended configuration value; reconstructing, using the configuration block, a first representation of an image at a first quality level in a hierarchical structure having a plurality of quality levels using a second representation of the image at the first quality level, the second representation being based on a representation of the image at a second, lower quality level in the hierarchical structure; Wherein the decoder reconstructs the first representation of the image at the first quality level using the extended configuration value.

Images