Out-of-loop smoothing of a boundary between two image areas

Abstract

A method for processing at least one decoded image area is proposed, the method comprising: - at the output of a decoding loop that has decoded at least one current image area, processing of the decoded current image area using a boundary smoothing module that employs metadata relating to at least one boundary between the current image area and a neighboring image area. Abstract figure: Figure 5

Description

Title of the invention: Out-of-loop smoothing of coding of a boundary between two image areas Technical field

[0001] The present disclosure relates to the field of video compression.

[0002] More particularly, the present disclosure relates to a method for processing at least one decoded image area, a computer program, a recording medium, a digital signal, and a data processing circuit. Prior art

[0003] Standardized video compression schemes have been based on the same principles since the first generation of MPEG standards, MPEG-2. In chronological order, the following standards are H.264 / AVC (2003), HEVC (2013), and VVC (2020). The AOM, VP9, ​​and AVI coding schemes also follow the same concepts.

[0004] A video sequence to be encoded is divided into images. Each image is divided into blocks of fixed size which can themselves be divided subsequently. For a given image, an encoder processes the blocks sequentially, from the block located at the top left of the image to the block located at the bottom right of the image. The encoder generates, as output, a binary signal comprising, for each image, the result of the sequential processing of the blocks which compose it.

[0005] The binary signal containing the video sequence thus compressed can then be broadcast and processed by a decoder whose operation, modeled on that of the encoder, considers the blocks sequentially in order to reconstruct the initial video sequence.

[0006] Reference is now made to [Fig. 1] which represents an example of an HEVC encoder built around a coding loop configured to carry out different processing of a block of a source video sequence (100) provided as input to the encoder.

[0007] One of these treatments is a prediction of the provided block, using information already encoded and decoded. The first image, called "Intra", is coded using a spatial prediction (118) using only pixels reconstructed in the vicinity of the block being processed. The following images, called "Inter", can use a spatial prediction and in addition a temporal prediction (116) which exploits the images previously coded using a motion compensation (114) signaled by a motion vector and which generally allows a very effective prediction. The images thus coded then decoded and used for the encoding of future images are grouped within a memory called "Decoded Picture Buffer" (DPB) (112).

[0008] Another processing performed in the coding loop is the coding of the difference between the prediction result and the block provided as input, or "pixel residues". This coding is performed after a transformation and quantization step (104). The quantization step is performed for a given quantization parameter (QP) associated with each block and signaled in the bitstream. The QP represents a compromise between the desired image quality after decoding and the desired degree of video compression. The higher the value of QP, the lower the amount of information relating to pixel residues in the encoded video sequence, and the higher the degree of video compression of the encoded video sequence. Conversely, the lower the value of QP, the higher the amount of information relating to pixel residues, and the better the reconstruction quality at the decoder receiving the encoded video sequence.A quantization and inverse transformation step allows the pixel residues to be reconstructed.

[0009] Other processing operations carried out in the coding loop are successive filtering of the block being processed by different filters. The HEVC standard provides two filters called "Sample Adaptive Offset" (SAO) (108) and "Deblocking Filter" (110) and which can be translated into French as "adaptive sample offset" and "antiblock filter". These filters modify the reconstructed pixels of the block being processed without having an impact on the prediction of neighboring blocks within the same image but by impacting the prediction of future blocks within subsequent images since the images in the DPB are those post-filtering. In addition to these two filters, the VVC standard introduced an additional filter called "Adaptive Loop Filter" (ALF), which can be translated into French as "adaptive loop filter".

[0010] As already explained, the encoder thus generates, at output, a binary signal (124) comprising, for each image, the result of the sequential processing of the blocks which compose it.

[0011] Transversally to these treatments, various high-level image divisions have been introduced into the standards to address different applications: for example, “Slices” and “Tiles” according to the HEVC standard and sub-images or “SubPictures” according to the VVC standard. These examples of divisions are described in particular in nplcitl. [Fig.2] represents an example of division of an image (200) according to the VVC standard into Tiles (delimited by thick solid lines) and Slices (delimited by thick dotted lines). In this example, the image is also divisible into blocks or “CTUs”, marked with thin lines.

[0012] One of the main uses of high-level image cutting concerns applications involving a large quantity of pixels to be coded. Examples include the encoding of video sequences with high image resolution (4K, 8K or 16K for example), high frame rate video sequences (greater than 60 fps for example) or 360° video sequences, such as those used for example for virtual reality applications.

[0013] High-level image divisions, for example into Tiles, are used in HEVC in order to allow parallelism on several encoding cores, for example by planning to process one Tile per encoding core, and thus respond to the high quantity of calculations required with limited or even harmful data sharing between the encoding cores. Parallelism can implement several threads or several cores of one or more data processing circuits which can be, for example, CPU, ASIC or FPGA circuits.

[0014] Generally, the video stream thus compressed is decoded by a decoder acting on a single decoding core and not having knowledge of the parallelism implemented during encoding. Decoding on several decoding cores is nevertheless also possible.

[0015] It is possible, in the HEVC standard, to implement, in the coding loop, processing to improve the visual quality between neighboring Tiles, namely a sharing of information between the encoding cores for the pixels at the boundaries between Tiles and specific processing at the boundaries within each encoding core. With regard to the filters, for example the SAO and anti-block filters, normative parameters can be provided in the encoded video sequence, to indicate whether or not these filters have been implemented in the coding loop in order to encode a given Slice or Tile. In the HEVC standard, these parameters are named "loop_filter_across_slice" and "loop_filter_across_tile". The notion of Motion Constrained Tile Set (MCTS), which relates to these specific modifications, is described in nplcit2.The MCTS is a set of measures taken at the encoder to make the coding / decoding of each tile independent of the coding / decoding of other tiles. It thus allows parallelization, at least, of the prediction and reconstruction processing. In addition, a parameter named "loop_filter_across_tile" and which can take values ​​"0" or "1" allows to indicate whether the parallelization can be extended to the border filtering processing (case of the value "0"), or if, on the contrary, the border filtering processing requires data relating to the coding / decoding of adjacent tiles (case of the value "1").

[0016] The VVC standard also introduced sub-images called "sub-pictures" intended to replace the MCTS. These sub-images are designed to be respectively processed by a respective encoding core in a completely independent manner. Several contributions such as nplcit3 were made by standardization actors to arrive at this design in the VVC standard.

[0017] Whatever the high-level cutting considered, it is desirable to be able to smooth, at the decoder level, the boundaries between Tiles, Slices, or sub-images having been processed by different encoding cores. Indeed, a lack of smoothing results, post-decoding, in a visually disturbing visibility of the boundaries.

[0018] With the current architecture of video compression standards, it is only possible to apply boundary smoothing at the decoder level without risk of drift if this smoothing has previously been applied at the encoder level. Thus, the VVC standard provides, when boundary smoothing is applied at the encoder level, to signal in the binary stream, by means of a parameter named "sps_loop_filter_across_subpic_enabled_flag", that this boundary smoothing is to be applied again at the decoder level.

[0019] Furthermore, considering a boundary between two neighboring Tiles, Slices or sub-images, the encoding of which is carried out, for one, by a first encoding core and, for the other, by a second distinct encoding core, smoothing of the boundary can only be implemented at the encoder level provided that pixels are transferred either between the first and second encoding cores, or from the first and second encoding cores to a third encoding core dedicated to implementing the smoothing of the boundary.

[0020] It follows from the above that, with the current architecture of video compression standards and in the case where the encoding is implemented by parallelization on several encoding cores, applying boundary smoothing induces significant constraints at the encoder level.

[0021] To avoid these constraints, it may be considered to signal, in the binary stream, that boundary smoothing is to be carried out at the decoder level, without however applying this smoothing at the encoder level.

[0022] Such processing is asymmetrical, because, in order to decode sub-images, it involves applying an anti-block filter at the decoder level to the boundaries of the sub-images, without a corresponding anti-block filter also having been applied to the boundaries at the encoder level to code these same sub-images.

[0023] Such asymmetric processing between the encoder and the decoder induces a drift, which is initially small and limited to the pixels on either side of the boundary between two sub-images. The more inter-predictions succeed one another, the more the drift increases. This drift can be minimized by prohibiting the activation, at the level of the encoder as well as the decoder, of the SAO and ALF filters at the level of the CTUs at the boundaries. Nevertheless, even with such provisions, such a drift can generate significant visual artifacts at the end of decoding.

[0024] To illustrate this aspect, a sequence of 60 images was encoded with the following procedure. Each image was previously divided into two halves separated by a vertical border. The left image halves were encoded by a first encoding core and in parallel the right image halves were encoded by a second separate encoding core. No communication was set up between the two encoding cores and no anti-blocking filter was implemented at the encoder level. [Fig. 3] represents the twentieth image (300), the fortieth image (302) and the sixtieth image (304) of the image sequence, as obtained after decoding, aggregation of the decoded image halves and implementation of an anti-blocking filter at the decoder level only, in the decoding loop. The drift appears very significant from the fortieth image and then continues to increase until it affects approximately three-quarters of the sixtieth image.

[0025] With the aim of preserving processing at the encoder and decoder, a solution has also been proposed in nplcit4 to allow filtering without impacting decoding due to expectations of previous results. This approach is nevertheless expensive in terms of computation time.

[0026] Finally, related work nplcit5 and nplcitô took place within the framework of the JPEG 2000 image compression standard. The authors introduce the notion of "detiling", which involves, at the encoder level, decomposing into wavelets the digital signals corresponding to groupings (in Tiles) of CTUs to be encoded. The authors also plan to apply filtering at the encoder level in the wavelet-transformed domain. Summary

[0027] The present disclosure improves the situation.

[0028] A method of processing at least one decoded image area is proposed, the method comprising: - at the output of a decoding loop having decoded at least one current image zone, processing of the decoded current image zone using a border smoothing module using metadata relating to at least one border between the current image zone and a neighboring image zone.

[0029] By "image area" is meant any area, delimited by a closed line, within an image. When two closed lines each delimiting an image area have a common portion, this common portion forms a boundary between these image areas which are then said to be neighbors.

[0030] By “decoding loop” is meant a set of logical instructions allowing, at least, to: - receive, as input, an extract of digital signal relating to the current image area, in encoded form, - decode the digital signal extract from, at least, information already decoded by the decoding loop, this already decoded information being able, for example, to relate to one or more other zones of the same image and / or to one or several other images, - provide, as output, the digital signal extract in decoded form, also called “decoded current image area”, and - store the current decoded image area as already decoded information usable for predicting image areas relating to future images.

[0031] By "metadata" is meant digital data associated with the boundary between the current image area and a neighboring image area. The metadata is used, at least, for the processing implemented by the boundary smoothing module.

[0032] It is understood that, according to the proposed method, the processing using the boundary smoothing module is carried out outside the decoding loop, and that as such the result of this processing does not impact any future decoding of image areas by the decoding loop. Thus, the proposed method makes it possible to carry out post-decoding boundary smoothing while maintaining symmetry of processing at the level of the encoder and the decoder. The proposed method therefore provides less visibility of the boundaries without generating visual artifacts, hence increased viewing comfort for a viewer.

[0033] Furthermore, the implementation of the proposed method at the output of the decoding loop does not require any modification of the existing coding algorithms or devices. In particular, the proposed method is fully compatible with encoders implementing parallelization techniques by means of a plurality of encoding cores, an encoding core being responsible for encoding an image area from a set of image areas resulting from a high-level partitioning of an image.

[0034] In some examples, the decoded current image area includes a border region corresponding to the boundary and a region distant from the boundary and processing the decoded current image area includes processing the border region and does not include processing the region distant from the boundary.

[0035] Thus, the processing of an image obtained by aggregation of image zones at the output of the decoder can be limited to regions located at the boundaries between the image zones, and without impacting the image as a whole. In other words, it is possible to implement differentiated processing by regions of image zones.

[0036] In some examples, the method further comprises controlling the processing of the current image area using a controller using first data relating to a decoding of the current image area and second data relating to a decoding of a neighboring image area.

[0037] Such a controller makes it possible to refine the processing implemented by the boundary smoothing module and in particular makes it possible to take into account possible differences between the data relating to the decoding on either side of the boundary, for example example of possible differences between the quantization parameters (QP) associated with the current image area and the neighboring image area.

[0038] A computer program is also provided comprising instructions for implementing the above method when this program is executed by a processor.

[0039] Also provided is a non-transitory recording medium readable by a computer on which a program is recorded for implementing the above method when this program is executed by a processor.

[0040] Also provided is a digital signal comprising at least one encoded current image area and metadata relating at least to one boundary between the current image area and a neighboring image area.

[0041] The digital signal is a signal that can be transported through a communication channel, stored in a memory, or read by a processor. The encoded current image area can be decoded by a suitable decoder.

[0042] The metadata may be used to process the decoded current image area, at least at the aforementioned boundary. In some examples, the metadata relates to at least one aspect of a smoothing to be applied at the boundary.

[0043] Examples of aspects of smoothing to be applied at the boundary include: - activation of smoothing, or - a smoothing force adjustment, or - a choice of a smoothing function from a set of predetermined smoothing functions.

[0044] Generally, the aspects considered may relate to a delimitation of one or more image regions where to implement the smoothing and / or a manner of implementing the smoothing in one or more regions among the delimited regions.

[0045] Also provided is a data processing circuit comprising: - an input interface configured to receive the aforementioned digital signal, and - at least one output interface configured to provide the current encoded image area as input to a decoding loop, and provide the metadata to a processing module provided at the output of the decoding loop.

[0046] The data processing circuit can be integrated into a single device or be formed of physical modules distributed among a plurality of devices placed in a communication network.

[0047] The decoding loop may or may not be part of the data processing circuit, as may the processing module. The processing module is configured at least to process the metadata, i.e., for example, to read it, store it, modify, relay, erase, etc. In some examples, the processing module is configured to use the metadata as an indication guiding the implementation of a smoothing operation of a boundary between neighboring image areas contained in the digital signal.

[0048] The output interface may further be configured to not provide metadata to the decoding loop. Brief description of the drawings

[0049] Other characteristics, details and advantages will appear on reading the detailed description below, and on analyzing the attached drawings, in which: Fig.l

[0050] [Fig.l] schematically represents an example of an encoder according to the HEVC standard. Fig. 2

[0051] [Fig.2] represents an example of cutting an image according to the VVC standard. Fig. 3

[0052] [Fig.3] represents three images in a sequence of decoded images, with an implementation of an anti-block filter at the decoder level only, in the decoding loop. Fig. 4

[0053] [Fig.4] represents an encoding of an image zone according to an exemplary embodiment. Fig. 5

[0054] [Fig.5] represents a decoding of an encoded image zone according to an exemplary embodiment. Fig. 6

[0055] [Fig.6] represents an encoder comprising four independent encoding cores, according to an exemplary embodiment. Fig. 7

[0056] [Fig.7] represents a decoder configured to decode four image zones encoded according to an example of realization. Description of the embodiments

[0057] The proposed technique aims to solve the problems previously described by introducing a normative smoothing of image area boundaries resulting from a high-level partitioning of the image. This smoothing aims, like the classic filtering used in conventional video coding schemes, to attenuate potentially visible boundaries between image areas such as Tiles, Slices, or sub-images.

[0058] The smoothing carried out by the proposed technique has the particularity of being carried out outside the coding loop. The image areas kept in the DPB and which serve as reference to image areas that will be coded in the future do not benefit from the proposed smoothing and the resulting visual attenuation of the border. This allows the smoothing of the borders of the image areas to be applied only on the decoder side. Indeed, since the reconstruction operation on the encoder side is limited to the generation of the same images that will be kept in the DPB on the decoder side, the smoothing of the borders of the sub-images does not occur there.

[0059] A particular example of embodiment is now described with reference to [Fig.4] which schematically represents an encoder.

[0060] A video sequence to be encoded (400) is provided in the form of a digital signal at the input of the encoder. The video sequence to be encoded comprises a plurality of images. It is considered that these images have been the subject of a high-level partitioning (not shown) into image zones. The encoder is configured to process the image zones contained in the video sequence sequentially. It may be provided that for a given image, the encoder first processes, for example, a first image zone located at the top left of the image, then a second image zone, for example adjacent to the first, and so on until processing a last image zone located, for example, at the bottom right of the image.

[0061] A particularity of the proposed technique is: that the video sequence to be encoded also includes, in addition to the image areas to be encoded, metadata, and / or that this metadata is made accessible by a separate digital signal associated with the video sequence to be encoded.

[0062] The metadata may relate to the entirety of the boundaries between image areas of a given image or of several given consecutive images. Alternatively, the metadata may relate to one or more particular boundaries. Particular examples of metadata are presented below in connection with their intended use.

[0063] In the remainder of the description, we focus on the processing of a current image area provided as input to the encoder. We also consider that the metadata relates to at least one boundary between the current image area and at least one neighboring image area.

[0064] The processing of a current image area provided as input to the encoder comprises a prediction of the current image area on the basis of image areas previously encoded then decoded and stored in a memory (408). More precisely, the prediction of the current image area may be an intra prediction (414) implemented from one or more previously processed areas of the same image. The prediction of the current image area may also be an inter prediction (412) implemented on the basis of an estimation (410) of a motion vector, itself implemented from areas of one or more previously processed images. Alternatively, it is also possible to implement in parallel an intra prediction (414) and an inter prediction (412) as described previously and to provide the results of these predictions to a decision module (416). The prediction of the current image area is then a prediction according to a prediction mode chosen by the decision module (416).

[0065] The result of the prediction of the current image area is then compared to the current image area provided as input to the encoder, and the difference, called pixel residues, is transformed and quantified (402). The current image area is then ready to be encoded. For this, an entropic coder (418), i.e. lossless, for example of the CABAC or CAVLC type, is supplied with all the information necessary to carry out the encoding, namely: - an indication of the prediction mode chosen to perform the prediction of the current image area (i.e. intra, inter, or a prediction mode chosen by the decision module), - prediction information according to the chosen mode, for example a motion vector in the case of an inter prediction mode or an intra prediction function in the case of the intra mode, and - the result of the transformation and quantification of pixel residues.

[0066] The same information is used to decode the current image area. An inverse quantization followed by an inverse transformation (404) of the transformed pixel residues is implemented, which makes it possible to reconstruct the pixel residues before transformation. The reconstructed pixel residues and the result of the prediction of the current image area are then added to reconstruct the current image area. Different filtering operations (406) can optionally be implemented. Finally, the reconstructed and optionally filtered current image area is stored in the memory (408) and can now be used for the implementation of future inter predictions, as part of the processing of future image areas.

[0067] As thus described, the processing of the current image area comprises a setting in implementation of a coding loop defined as a sequence of processing operations, based on processing operations of at least one previous image area and enabling the processing of at least one subsequent image area to be fed. Specifically, the current image area is first predicted on the basis of previously processed image areas, so these are processing operations of the current image area based on processing operations of at least one previous image area. From this prediction comes the determination of the information necessary for coding the current image area. This information is used to determine the result of the decoding of the encoded current image area. Finally, this result is stored in memory for the processing of one or more subsequent image zones, these are therefore processings of the current image zone allowing the processing of at least one subsequent image zone to be fed.

[0068] At the end of the processing of the current image zone, the entropic coder (418) provides, at output, a binary signal (420) comprising the current image zone in encoded form. More generally, at the end of the processing of the video sequence (400) provided at the input of the encoder, the binary signal (420) provided at the output of the entropic coder comprises, in encoded form, all the zones of all the images of the video sequence (400).

[0069] A particularity of the proposed technique is that the binary signal (420) is enriched with the aforementioned metadata which, as specified, relate to at least one boundary between the current image area and at least one neighboring image area. In any event, this metadata is not processed by the coding loop, i.e. it is not used within the coding loop in any of the processing operations serving as a basis for determining the encoded current image area or any other encoded image area. Optionally, this metadata is not made accessible to the coding loop and is only provided to a post-processing module (not shown) which incorporates it or joins it to the binary signal from the entropy coder (418).

[0070] According to the proposed technique, this metadata is a form of signaling from the encoder to a smoothing module located outside the decoding loop for the purpose of smoothing a boundary between two neighboring image areas. The smoothing module is described later.

[0071] Referring to existing video coding standards, any signaling from an entity on the encoder side to an entity on the decoder side must be done via a particular format of messages conveyed by the binary signal containing the encoded video sequence. These messages are called "supplemental enhancement information" or SEL in English. Taking into account the SEIs on the decoder side is optional and intended to offer new functionalities conveyed by these SEIs. It is possible to edit the standards on the SEI parts a posteriori. For example, the standard corresponding to VVC includes a separate document nplcitô relating to the SEI part of the standard.

[0072] By not being limited to existing video coding standards, the signaling possibilities are more extensive and are not limited to SEIs. Signaling can thus be implemented at the level of an image sequence via one or more headers associated with the sequence such as SPS, PPS or VUI. Signaling can also be implemented at the level of an image via one or more headers associated with the image of the “slice header” or “picture header” type. In certain examples where the information of coding are made accessible to the smoothing module, signaling can also be implemented at the level of image regions located on either side of a boundary between neighboring image areas.

[0073] Reference is now made, in an exemplary embodiment, to [Fig.5] which schematically represents a decoder adapted to decode the aforementioned binary signal (420).

[0074] The decoder comprises an input interface configured to receive the binary signal. The encoded images, and more precisely the encoded image zones contained in the binary signal, are provided as input to an entropy decoder (502) and successively processed by the entropy decoder. For example, the implementation, by the entropy decoder, of the processing of the current encoded image zone generates, at the output of the entropy decoder, two types of information, namely: the pixel residues as resulting from the transformation and quantization (402) at the encoder level, and the prediction mode as well as the prediction information used at the encoder level for the prediction of the current image area.

[0075] This information is processed by a decoding loop operating in the same way as the aforementioned coding loop in that it relies on one or more previously decoded image areas to decode the current image area. After its decoding, the current image area is stored in a memory (408) and is in turn used to decode one or more subsequent image areas. The image areas stored in the memory (408) form a sequence of decoded image areas which can be aggregated, image by image, to form an image sequence, i.e. a decoded video sequence.

[0076] A particularity of the proposed technique is that the current decoded image area is transmitted to a boundary smoothing module (518), more simply referred to hereinafter as a “smoothing module”, located outside the decoding loop. The smoothing module also receives the metadata contained in the binary signal (420) and / or associated with the binary signal. The metadata may relate to any aspect of the processing implemented by the smoothing module. For example, the metadata may be indicative of smoothing activation for all of the boundaries between image areas or, on the contrary, only for certain particular boundaries. The smoothing module then implements boundary smoothing limited to these particular boundaries. For example, the metadata may be indicative of a smoothing strength which may be common to all of the boundaries between image areas or differentiated by boundary.The smoothing module then implements the smoothing, for each respective boundary, according to the respectively indicated smoothing strength. For example, the metadata may be indicative of a function of . smoothing to be implemented among a set of predefined smoothing functions. A Gaussian filter is an example of a filter that can implement a smoothing function. The metadata can thus be indicative, for one or more given boundaries, of a particular smoothing function to be implemented and / or of a particular parameter value entering into the definition of the smoothing function. Considering as a smoothing function a function implemented by a Gaussian filter, the number of taps is an example of a suitable parameter.

[0077] The smoothing module (518) uses the metadata relating to the smoothing of the boundary between the current image area and a neighboring image area to implement a smoothing of this boundary. The smoothing module thus provides as output, at least, a decoded and smoothed current image area.

[0078] In the example shown, the smoothing module receives the decoded video sequence, formed from a sequence of decoded images, one of these decoded images containing the current decoded image area. The smoothing module uses the metadata to process the video sequence and thus provides as output a modified decoded video sequence (520) at least in that the current decoded image area is also smoothed.

[0079] Another feature of the proposed technique is that the result of the smoothing of one or more areas of one or more images, obtained at the output of the smoothing module (518) is not provided to the decoding loop and is not used to decode subsequent image areas. The proposed technique therefore makes it possible to achieve a visual rendering after decoding without visible boundaries that are annoying for end users, while avoiding any passage of pixels or other coding information between sub-images on the encoder side, which is welcome in coding schemes using separate encoding cores that each process a sub-image in parallel. Indeed, limitations in terms of connectors make data transfers between encoding cores expensive with these architectures.

[0080] By way of illustration, [Fig.6] shows an example of encoding an image contained in a video sequence (400) to be encoded, the image being divided into four sub-images (600, 602, 604, 606). The video sequence (400) to be encoded is associated with, or comprises, metadata relating to at least one boundary between a sub-image considered (600) and a neighboring sub-image (602). The encoding of the image is implemented in parallel by four separate encoding cores (608, 610, 612, 614). The processing of a given image area (600) by a given encoding core (608) comprises the implementation of a coding loop specific to the given encoding core (608). This means in particular that each encoding core (608) has its own storage memory, configured to store the image areas encoded at that encoding core (608). Each encoding core operates independently, without any data sharing between encoding cores. The outputs of the four encoding cores are concatenated (616) so as to generate the binary signal (420) comprising the four sub-images in encoded form. This binary signal is also enriched with the metadata associated with, or included in, the video sequence (400) to be encoded. Unlike a method which would implement smoothing in the decoding loop on the decoder side but without implementing smoothing on the encoder side, the proposed technique has the advantage of being completely normative and of avoiding drifts such as those illustrated in [Fig.3].

[0081] Reference is now made to [Fig. 7], which illustrates an example of a decoder suitable for decoding encoded image areas (700, 702, 704, 706) of the same image contained in a binary signal (420). This decoding can be implemented indifferently in a parallel manner by a plurality of decoding cores (708, 710, 712, 714) or in a sequential manner by a single decoding core.

[0082] In this example, the decoded image areas are aggregated by an aggregator (716) so as to reconstruct in decoded form the image contained in encoded form in the binary signal (420). It is after this aggregation that the boundaries of the sub-images are smoothed.

[0083] The smoothing of a boundary between two neighboring image areas located on either side of the boundary is implemented by the smoothing module (518) placed outside the decoding loop. As already described, the smoothing module relies for this, at least, on the metadata.

[0084] Optionally, it may be provided that the smoothing module (518) has access to the coding information of these neighboring image zones. This coding information may be provided to it for example by the decoding core(s) having decoded these image zones, directly or indirectly. This is however not always possible, in particular due to system constraints. It is for example conceivable that, as at the encoder, the sub-images are each decoded on a separate decoding core, and that, once a sub-image has been decoded by a decoding core from the corresponding part of the binary signal, the coding information relating to this sub-image is no longer available.

[0085] It may be provided that when the smoothing module (518) has access to the coding information, it uses it to control the implementation of the smoothing. This allows for example the smoothing module to implement a standardized anti-block filter, insofar as, according to current standards, such an anti-block filter uses information such as the size of the transforms, the coding mode (inter / intra in particular), the QP or the motion vectors (in the case of inter coding) to determine the length and the power of the smoothing.

[0086] The smoothing module (518) can of course be configured to implement any other smoothing function capable of using all or part of these in coding formations to control the implementation of smoothing. A simple example of an applicable smoothing function would be a Gaussian function whose standard deviation would depend on the QP of the image areas on either side of the boundary to be smoothed.

[0087] Two particular examples of embodiments are now described.

[0088] In a first particular exemplary embodiment, a source image is partitioned into several sub-images which are each encoded. The term “sub-image” is to be understood here as equivalent to the term “sub-picture” defined according to the VVC video coding standard described for example in nplcitô. The sub-images are signaled through headers as completely independent, in particular at the level of smoothing. In other words, in the coding loop, no normative smoothing is carried out across the boundaries between sub-images. Several encoding cores are provided to parallelize the encoding of the sub-images. The encoding of a sub-image is implemented completely independently on an encoding core, without any sharing of pixels between the encoding cores. At the end of the encoding of the sub-images, a binary signal, comprising the encoded sub-images, is generated. The binary signal further includes an SEI which contains a digital value.

[0089] The binary signal is received at the decoder. The encoded sub-images are each decoded on a separate decoding core independently. At the output of the decoding cores, the decoded sub-images are aggregated into an image. The coding information of the blocks constituting the sub-images is not transmitted to the output of the decoding cores. After aggregation of the sub-images and before restitution of the image, an out-of-loop smoothing of the boundaries between sub-images is implemented. This smoothing does not use any coding information of the blocks constituting the sub-images because this information is not made accessible by the decoding cores. The implementation of the smoothing in question uses a simple five-tap Gaussian filter whose standard deviation, which determines the smoothing power, is fixed to the numerical value contained in the SEL. The smoothing power is thus identical regardless of the smoothed boundary and regardless of the image.

[0090] In a second particular example of embodiment, an image is partitioned into several “tiles”, the latter term being defined in the HEVC coding standard and described in nplcit7 in particular. The tiles are associated, via headers, with indications not to implement any smoothing of the boundaries between tiles in the coding loop.

[0091] The encoding (HEVC) of each tile is implemented on a separate encoding core independently. In this second particular exemplary embodiment, constraints are provided so that no sharing of pixels or information is implemented between encoding cores. Such constraints ensure that the encoding / decoding of each tile is independent of the encoding / decoding of other tiles and fall under the concept of MCTS already described.

[0092] As already explained, the encoding of a tile comprises a prediction of the tile, which may comprise an intra prediction and / or an inter prediction. In particular, the prediction of the tile may comprise a prediction of a motion vector, for example in an inter or “merge” coding mode, these coding modes both being defined in the HEVC standard.

[0093] A first example of constraint concerns the prediction of the motion vector: it is a matter of imposing that, during the prediction of a current tile, the prediction of the associated motion vector points in this current tile.

[0094] A second example of constraint concerns the “merge” coding mode for which a list of candidates is provided for the purpose of performing the prediction of a motion vector for a current tile: it is then a matter of providing only candidates such that the prediction of the motion vector for this candidate points to the current tile. Furthermore, a candidate noted “TMVP” for “temporal motion vector predictor” (in English) is typically provided, this TMVP candidate relating to a previously coded image. However, it is possible that the TMVP candidate relates to a tile having a different location, in the previously coded image, than that of the current tile of the current image. In other words, in such a case, the current tile and the tile to which the TMVP candidate relates are processed by separate encoding cores.One possibility is then to prohibit the selection of this TMVP candidate to avoid any exchange of vectors, or more generally of information, between encoding cores.

[0095] Tile decoding is implemented serially on a single decoding core. The decoded tiles are then aggregated into an image. After aggregation and before rendering the image, out-of-loop smoothing of the boundaries between the tiles is performed. This smoothing uses all the block coding information made available by the single decoding core. Out-of-loop smoothing is implemented by an anti-block filter whose operation is defined in the HEVC standard and relies on one or more SEI messages relating to the boundaries between blocks, and consequently relating to the boundaries between tiles.

[0096] For all useful purposes, the following non-patent documents are cited: nplcitl : Y. -K. Wang et al., "The High-Level Syntax of the Versatile Video Coding (VVC) Standard," in IEEE Transactions on Circuits and Systems for Video Technology, vol. 31, no. 10, pp. 3779-3800, Oct. 2021, doi: 10.1109 / TCSVT.2021.3070860 nplcit2 : WU, Yongjun, SULLIVAN, Gary J., ZHANG, Yifu, Motion-constrained tile set for région of interest coding, WO / 2014 / 168650 nplcit3 : Hendry, S. Hong, J. Chen, Y.-K. Wang (Huawei), JVET-N0109, CE12 / AHG12: Treating boundaries of independent tile groups as picture boundaries, Mars 2019 nplcit4 : S. Cho, H. Kim, H. Y. Kim and M. Kim, "Efficient In-Loop Filtering Across Tile Boundaries for Multi-Core HEVC Hardware Decoders With 4 K / 8 K-UHD Video Applications," in IEEE Transactions on Multimedia, vol. 17, no. 6, pp. 778-791, June 2015, doi: 10.1109 / TMM.2015.2418995 nplcit5 : Singh, S., Sharma, R. K., & Sharma, M. K. (2012). Post Processing Technique to Reduce Tile Boundary Artifacts in JPEG2000 Compressed Images. The International Journal of Multimedia & Its Applications, 4(1), 127 nplcitô : Schwartz, E. L., Berkner, K., & Gormish, M. J. (1999). Optimal tile boundary artifact removal with CREW. In Proc, of Picture Coding Symposium, (pp. 285-288) nplcit7 : SO / IEC CD 23002-7 Supplémentai enhancement information messages for coded video bitstreams nplcit8 : H.266 : Versatile video coding https: / / www.itu.int / rec / T-REC-H.266 nplcit9 : ITU-T H.265, High efficiency video coding https: / / www.itu.int / rnj-T / recommendations / rec.aspx?rec= 14107&lang=en nplcitlO : Y. He, M. Coban, M. Karczewicz, “AHG9 / AHG13: Film grain blending process for film grain characteristics SEI message”, JVET-Y0053, Jan. 2022.

Claims

Claims

1. Method for processing at least one decoded image zone, the method comprising: at the output of a decoding loop having decoded at least one current image zone, processing the decoded current image zone, using a boundary smoothing module (518) using metadata relating at least to a boundary between the current image zone and a neighboring image zone, in which the metadata relates at least to a choice of a smoothing function to be applied at the boundary from a set of predetermined smoothing functions.

2. The method of claim 1, wherein: the decoded current image area comprises a border region corresponding to the boundary and a region distant from the boundary, the processing of the decoded current image area comprises processing of the border region and does not comprise processing of the region distant from the boundary.

3. A method according to claim 1 or 2, the method further comprising: controlling the processing of the decoded current image area using a controller using first data relating to a decoding of the current image area and second data relating to a decoding of a neighboring image area.

4. Computer program comprising instructions for implementing the method according to one of claims 1 to 3 when this program is executed by a processor.

5. Non-transitory recording medium readable by a computer on which is recorded a program for implementing the method according to one of claims 1 to 3 when this program is executed by a processor.

6. Digital signal comprising at least one encoded current image area and metadata relating at least to a boundary between the current image area and a neighboring image area, wherein the metadata relates at least to a choice of a smoothing function to be applied at the boundary from a set of predetermined smoothing functions.

7. A digital signal according to claim 6, wherein the metadata further relates to an activation of smoothing, and / or to an adjustment

8.

9. of a smoothing force to be applied at the boundary. Data processing circuit comprising: an input interface configured to receive a digital signal comprising at least one encoded current image area and metadata relating at least to a boundary between the current image area and a neighboring image area, wherein the metadata relates at least to a choice of a smoothing function to be applied at the boundary from a set of predetermined smoothing functions, and at least one output interface configured to provide the current encoded image area as input to a decoding loop, and provide the metadata to a processing module (518) provided at the output of the decoding loop. The data processing circuit of claim 8, wherein the output interface is configured not to provide the metadata to the decoding loop.

Images