Decoding method, encoding method, decoder, and encoder

Abstract

The embodiment of the invention provides a decoding method, an encoding method, a decoder and an encoder, and the decoding method comprises the steps: decoding a code stream, and determining a current sub-image block in a current image; determining whether the current sub-image block is a sub-image block needing illumination transformation or not; and when the current sub-image block is the sub-image block needing illumination transformation, performing illumination transformation on the current sub-image block to obtain the sub-image block after illumination transformation. The decoding method provided by the invention can improve the decoding performance.

Description

Decoding method, encoding method, decoder and encoder Technical Field

[0001] The embodiments of the present application relate to the field of coding and decoding, and more specifically, to a decoding method, an encoding method, a decoder, and an encoder. Background Art

[0002] For the Moving Picture Experts Group Immersive Video (MIV) encoding process, a limited number of viewpoints can be selected from the reference viewpoints as base viewpoints, and the selected base viewpoints can express the visible range of the scene as much as possible. The image under the base viewpoint (also called the base view) can be transmitted as a complete image; then, by removing redundant pixels between the base view and the image under the non-base viewpoint (also called the additional view), only the valid information that is not repeated can be retained, and then the valid information can be extracted into sub-blocks and reorganized with the image under the base viewpoint to form a larger rectangular image (also called a mosaic or mosaic image). The mosaic image can be used to generate video data so that the encoder can encode the video data to obtain a bitstream. For example, a pixel pruning module can be used to detect repeated pixels between the base view and the additional view, and prune the repeated pixels in the additional view. However, since the detection of repeated pixels between the base view and the additional view is based on depth difference values ​​and brightness difference values, the retained pixels may include pixels that are greatly affected by lighting (such as highlight pixels). The information of highlight pixels will be degraded after video encoding and decoding, such as the loss of texture information of sub-blocks, which in turn reduces decoding performance.

[0003] Summary of the Invention

[0004] The embodiments of the present application provide a decoding method, an encoding method, a decoder, and an encoder, which can improve decoding performance.

[0005] In a first aspect, an embodiment of the present application provides a decoding method, comprising:

[0006] Decode the code stream and determine the current sub-block in the current image;

[0007] Determining whether to perform illumination transformation on the current sub-tile;

[0008] When it is determined to perform illumination transformation on the current sub-image block, illumination transformation is performed on the current sub-image block to obtain a sub-image block after illumination transformation.

[0009] In a second aspect, an embodiment of the present application provides an encoding method, including:

[0010] Determine the current sub-block in the current image;

[0011] Determining whether to perform illumination transformation on the current sub-tile;

[0012] When determining to perform illumination transformation on the current sub-image block, performing illumination transformation on the current sub-image block to obtain an illumination-transformed sub-image block;

[0013] The sub-image block after the illumination transformation is encoded to obtain a code stream.

[0014] In a third aspect, an embodiment of the present application provides a decoder, including:

[0015] A decoding unit, configured to decode the code stream and determine a current sub-block in the current image;

[0016] a determining unit, configured to determine whether to perform illumination transformation on the current sub-block;

[0017] The transformation unit is configured to, when determining to perform illumination transformation on the current sub-image block, perform illumination transformation on the current sub-image block to obtain a sub-image block after illumination transformation.

[0018] In a fourth aspect, an embodiment of the present application provides an encoder, including:

[0019] A first determining unit, configured to determine a current sub-block in a current image;

[0020] A second determining unit, configured to determine whether to perform illumination transformation on the current sub-block;

[0021] a transformation unit, configured to, when determining to perform illumination transformation on the current sub-block, perform illumination transformation on the current sub-block to obtain a sub-block after illumination transformation;

[0022] The encoding unit is used to encode the sub-image block after the illumination transformation to obtain a code stream.

[0023] In a fifth aspect, an embodiment of the present application provides a decoder, including:

[0024] a processor adapted to implement computer instructions; and,

[0025] A computer-readable storage medium stores computer instructions, wherein the computer instructions are suitable for being loaded by a processor and executing the decoding method in the first aspect or its various implementations mentioned above.

[0026] In one implementation, there are one or more processors and one or more memories.

[0027] In one implementation, the computer-readable storage medium may be integrated with the processor, or the computer-readable storage medium may be provided separately from the processor.

[0028] In a sixth aspect, an embodiment of the present application provides an encoder, including:

[0029] a processor adapted to implement computer instructions; and,

[0030] A computer-readable storage medium stores computer instructions, wherein the computer instructions are suitable for being loaded by a processor and executing the encoding method in the second aspect or its various implementations mentioned above.

[0031] In one implementation, there are one or more processors and one or more memories.

[0032] In one implementation, the computer-readable storage medium may be integrated with the processor, or the computer-readable storage medium may be provided separately from the processor.

[0033] In the seventh aspect, an embodiment of the present application provides a computer-readable storage medium, which stores computer instructions. When the computer instructions are read and executed by a processor of a computer device, the computer device executes the decoding method involved in the first aspect mentioned above or the encoding method involved in the second aspect mentioned above.

[0034] In an eighth aspect, embodiments of the present application provide a computer program product or computer program, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the decoding method of the first aspect or the encoding method of the second aspect.

[0035] In a ninth aspect, an embodiment of the present application provides a code stream, which is the code stream involved in the method described in the first aspect above or the code stream generated by the method described in the second aspect above.

[0036] Based on the above technical solution, by introducing illumination transformation for the current sub-block that needs to undergo illumination transformation on the decoder side, it is beneficial for the encoder to enhance the quality of the current sub-block by illumination transformation of the current sub-block. For the decoder, the illumination transformation of the current sub-block can not only restore the illumination effect of the current sub-block, but also ensure that the sub-block decoded by the decoder is the sub-block that has been quality-enhanced by the encoder. Furthermore, it can alleviate the problem of quality degradation of its pixel information after video encoding and decoding, such as alleviating the problem of texture information loss of the sub-block, thereby improving decoding performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] FIG1 is an example of a MIV encoding and decoding system provided in an embodiment of the present application.

[0038] FIG2 is an example of a preprocessing process provided in an embodiment of the present application.

[0039] FIG3 is an example of a depth map provided by an embodiment of the present application.

[0040] FIG4 is an example of a texture map provided in an embodiment of the present application.

[0041] FIG5 is an example of an occupancy map provided in an embodiment of the present application.

[0042] FIG6 is an example of a process for generating a mosaic image according to an embodiment of the present application.

[0043] FIG7 is a schematic flowchart of a decoding method provided in an embodiment of the present application.

[0044] FIG8 is a schematic flowchart of the encoding method provided in an embodiment of the present application.

[0045] FIG9 is an example of a mapping relationship between pixels in a base view and pixel blocks where pixels in a current sub-block are located, provided by an embodiment of the present application.

[0046] FIG10 is an example of a similar pixel pair determination process provided in an embodiment of the present application.

[0047] FIG11 is an example of incorporating illumination transformation into the judgment process of similar pixel pairs provided by an embodiment of the present application.

[0048] FIG12 is a schematic block diagram of a decoder provided in an embodiment of the present application.

[0049] FIG13 is a schematic block diagram of an encoder provided in an embodiment of the present application.

[0050] FIG14 is an example of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0051] The technical solutions in the embodiments of the present application will be described below with reference to the accompanying drawings.

[0052] FIG1 is an example of a MIV encoding and decoding system 10 provided in an embodiment of the present application.

[0053] As shown in FIG1 , the MIV encoding and decoding system 10 can be roughly divided into the following modules according to the task line: a data acquisition module 11 , a pre-processing module 12 , a video encoding framework 13 , a video decoding framework 14 and a post-processing module 15 .

[0054] The data acquisition module 11 is used to collect data. The data collected by the data acquisition module 11 may include not only texture information but also depth information corresponding to the texture information. The texture information may be a three-channel color image, and the depth information may be a depth map. Optionally, the pixel values ​​of the depth map can be used to reflect the distance measurement from the corresponding point to the acquisition device, that is, the pixel values ​​of the depth map can be used to reflect the geometric information of the corresponding point.

[0055] The preprocessing module 12 is used to organize and express the data collected by the data acquisition module 11 to obtain the video to be encoded. The input source format of the preprocessing module 12 is a multi-view texture plus depth video, and the format of the video can be a perspective-projected planar video or a panoramic video. Specifically, after the data acquisition module 11 collects the corresponding video sequence through cameras at different positions, the preprocessing module 12 can first perform lighting balance and color correction on the images in the video sequence, and can also perform camera calibration and image calibration. If the video format collected by the data acquisition module 11 is a panoramic video, the preprocessing module 12 can also use image stitching technology to stitch the images in the panoramic video, and map the stitched image into a two-dimensional plane video.

[0056] FIG2 is an example of a preprocessing process provided in an embodiment of the present application.

[0057] As shown in Figure 2, the reference viewpoint is the viewpoint used by the data acquisition module 11 to collect data. The reference viewpoint can be associated through viewpoint parameters, texture data and geometric data. The preprocessing module 12 can select a limited number of viewpoints as basic viewpoints from the reference viewpoint, and can enable the selected basic viewpoints to express the visual range of the scene as much as possible.

[0058] Among them, the image under the basic viewpoint (also called the basic view) can be transmitted as a complete image. Furthermore, the redundant pixels between the remaining non-basic viewpoints (also called additional views) and the basic viewpoint can be removed, that is, only the valid information of non-repeated expression is retained, and then the valid information is extracted into sub-blocks and reorganized with the image under the basic viewpoint to form a larger rectangular image (also called a mosaic or mosaic image).

[0059] Furthermore, the stitched image can be used to generate video data, and the video data may include texture video data and geometry video data. The video data can be used as an input image of the video encoding framework 13, and the output of the video encoding framework 13 is a code stream obtained by encoding the input image. The code stream output by the video encoding framework 13 may include a geometry code stream and an attribute code stream. The geometry code stream is a code stream generated by encoding a geometric depth map (such as the depth map shown in Figure 3), which is used to represent geometric information; the attribute code stream is a code stream generated by encoding a texture map (such as the texture map shown in Figure 4), which is used to represent attribute information. Optionally, the code stream output by the video encoding framework 13 may also include an occupancy code stream, which is a code stream generated by encoding an occupancy map (such as the occupancy map shown in Figure 5), which is used to indicate valid areas in the depth map and texture map; these three types of images are all encoded and decoded using a video encoder. The code stream output by the video encoding framework 13 can serve as the input code stream of the video decoding framework 14.

[0060] Of course, for the stitched image, there may also be auxiliary data related to the stitching of sub-blocks.

[0061] As shown in FIG2 , auxiliary data related to the stitching information of the sub-blocks can also be written into the bitstream, thereby reducing the transmission pixel rate while retaining scene information as much as possible, thereby ensuring that there is sufficient information for the post-processing module 15 to render the reconstructed image to obtain the final view.

[0062] FIG6 is an example of a process for generating a mosaic image according to an embodiment of the present application.

[0063] As shown in Figure 6, the reference viewpoints may include viewpoint 0, viewpoint 1 and viewpoint 2. The reference viewpoint image collected at viewpoint 0 can be used to extract sub-block 2 and sub-block 4, the reference viewpoint image collected at viewpoint 1 can be used to extract sub-block 8, and the reference viewpoint image collected at viewpoint 2 can be used to extract sub-block 3 and sub-block 7. Furthermore, mosaic image 0 can be obtained based on sub-block 2, sub-block 4 and sub-block 8, and mosaic image 1 can be obtained based on sub-block 3 and sub-block 7; then mosaic image 0 and mosaic image 1 are spliced ​​to the image under the basic viewpoint to obtain the final mosaic image, and the final mosaic image can be used as the input image of the video encoding framework 13 to encode the input image to obtain a code stream.

[0064] The video encoding framework 13 is used to encode and compress the video, and the video decoding framework 14 is used to decode and reconstruct the video.

[0065] It should be noted that this application does not limit the specific implementation of the video encoding framework 13 and the video decoding framework 14. For example, a codec framework that applies three-dimensional video coding technologies such as MV-HEVC and 3D-HEVC to multi-view video can be used. This codec framework has higher codec efficiency than HEVC. Of course, a codec framework that uses traditional planar video hybrid coding technologies such as HEVC and VVC can also be used.

[0066] The post-processing module 15 is used to synthesize and render the image decoded by the video decoding framework 14 to obtain the final view. For example, the target view at the user's viewpoint can be synthesized and rendered based on the decoded and reconstructed image and the user's current position information.

[0067] FIG7 is a schematic flow chart of a decoding method 200 provided in an embodiment of the present application. It should be understood that the decoding method 200 can be executed by a decoder. For example, it can be applied to the decoding framework 14 shown in FIG1 . For ease of description, the following description uses a decoder as an example.

[0068] As shown in FIG7 , the decoding method 200 may include some or all of the following:

[0069] S210, decoding the code stream to determine the current sub-block in the current image;

[0070] S220, determining whether to perform illumination transformation on the current sub-block;

[0071] S230 : When it is determined to perform illumination transformation on the current sub-image block, perform illumination transformation on the current sub-image block to obtain a sub-image block after illumination transformation.

[0072] Exemplarily, the current image may have an additional view.

[0073] Exemplarily, the current sub-image block may be an image block obtained by the encoder processing the additional view. For example, the encoder may remove redundant pixels between the base view and the additional view, that is, retain only valid pixels of non-repeated expression (also referred to as valid pixel clusters), and then divide and pack the valid pixel clusters to form image blocks of regular shapes (i.e., sub-image blocks), and then reorganize the base view and one or more sub-image blocks obtained based on the additional view (for example, splicing the base view and the sub-image blocks obtained based on the additional view) to form a larger rectangular image (also referred to as a spliced ​​image or spliced ​​image), which can be used to generate video data so that the encoder can encode the video data to obtain a bitstream.

[0074] Accordingly, the decoder may restore or decode the additional view based on the current sub-image block.

[0075] Specifically, the decoder may determine, based on the current sub-block, the valid pixel clusters retained by the encoder after cropping the redundant pixels in the additional view, and then restore the additional view based on the resulting valid pixel clusters. For example, the decoder may determine the position of the current sub-block in the additional view based on the cropping mask of the additional view, and based on the decoded position of the valid pixel clusters, use pixels in the base view to restore the redundant pixels cropped in the additional view, and then restore the additional view based on the pixels of the current sub-block and the cropped points in the additional view.

[0076] Exemplarily, the current sub-image block may be a rectangular image block.

[0077] In this embodiment, by introducing illumination transformation on the current sub-image block that needs to undergo illumination transformation on the decoder side, the encoder is able to enhance the quality of the current sub-image block by means of illumination transformation of the current sub-image block. For the decoder, illumination transformation of the current sub-image block can not only restore the illumination effect of the current sub-image block, but also ensure that the sub-image block decoded by the decoder is the sub-image block that has undergone quality enhancement by the encoder. Furthermore, it can alleviate the problem of quality degradation of its pixel information after video encoding and decoding, for example, it can alleviate the problem of texture information loss of the sub-image block, thereby improving decoding performance.

[0078] In some embodiments, the S220 may include:

[0079] Decoding the code stream to determine a sub-block identifier of the current sub-block; the sub-block identifier of the current sub-block indicates whether to perform illumination transformation on the current sub-block;

[0080] Based on the sub-image block identifier of the current sub-image block, it is determined whether to perform illumination transformation on the current sub-image block.

[0081] Exemplarily, if the sub-block identifier of the current sub-block indicates that illumination transformation is to be performed on the current sub-block, the decoder determines to perform illumination transformation on the current sub-block; if the sub-block identifier of the current sub-block indicates that illumination transformation is not to be performed on the current sub-block, the decoder determines not to perform illumination transformation on the current sub-block.

[0082] Of course, in other alternative embodiments, whether to perform illumination transformation on the current sub-tile may be determined in other ways.

[0083] In one possible implementation, the decoder may determine whether to perform an illumination transformation on the current sub-block based on whether the bitstream carries an identifier of the current sub-block. For example, if the bitstream carries the identifier of the current sub-block, the decoder determines to perform an illumination transformation on the current sub-block; if the bitstream does not carry the identifier of the current sub-block, the decoder determines not to perform an illumination transformation on the current sub-block.

[0084] In another possible implementation, the decoder may further perform a pixel-level illumination transformation or a view-level illumination transformation. For example, the decoder may further determine whether to perform an illumination transformation on the current pixel in the current sub-image block based on an identifier corresponding to the current pixel in the current sub-image block, where the identifier corresponding to the current pixel indicates whether to perform an illumination transformation on the current pixel. For another example, the decoder may further determine whether to perform an illumination transformation on the sub-image block in the additional view based on an identifier corresponding to the additional view to which the current sub-image block belongs, where the identifier corresponding to the additional drawing indicates whether to perform illumination changes with the sub-image block in the additional drawing.

[0085] In some embodiments, the decoder may also decode the code stream and determine an enable flag of the sub-image block identifier of the current sub-image block; the enable flag indicates whether the code stream carries the sub-image block identifier of the current sub-image block.

[0086] Exemplarily, the enabling flag may be carried in an Atlas sequence parameter set MIV extension syntax in the code stream.

[0087] For example, the Atlas sequence parameter set MIV extended syntax may be as shown in Table 1:

[0088] Table 1

[0089]

[0090]

[0091] As shown in Table 1, asme_ic_enabled_flag is an enable flag.

[0092] Among them, asme_ic_enabled_flag equal to 1 specifies that the pdu_ic_flag[tileID][p]syntax elements are present in the pdu_miv_extension()syntax structure.asme_ic_enabled_flag equal to 0 specifies that the pdu_ic_flag[tileID][p]syntax elements are not present in the pdu_miv_extension()syntax structure.When not present,the value of asme_ic_enabled_flag is inferred to be equal to 0).

[0093] Exemplarily, the sub-picture block identifier of the current sub-picture block may be carried in a patch data unit MIV extension syntax (Patch data unit MIV extension syntax) in the code stream.

[0094] Exemplarily, the sub-block data unit MIV extended syntax may be as shown in Table 2:

[0095] Table 2

[0096]

[0097] As shown in Table 2, pdu_ic_flag[tileID][p] is the sub-tile identifier.

[0098] Among them, pdu_ic_flag[tileID][p] equal to 1 specifies that the sub-tile performs lighting transformation. pdu_ic_flag[tileID][p] equal to 0 means that the sub-tile does not perform lighting transformation. When pdu_ic_flag[tileID][p] does not exist, its value defaults to 0. (pdu_ic_flag[tileID][p]equal to 1 specifies that the patch is an IC patch. When pdu_ic_flag[tileID][p] is equal to 0, then the patch is not an IC patch. When not present, the value of pdu_ic_flag[tileID][p] is inferred to be equal to 0.)

[0099] In some embodiments, the S230 may include:

[0100] Determining a target transformation mode used by the current sub-block;

[0101] Determining target transformation parameters used by the current sub-block;

[0102] Based on the target transformation mode and the target transformation parameters, the current sub-image block is subjected to illumination transformation to obtain the sub-image block after illumination transformation.

[0103] Exemplarily, when the decoder determines to perform illumination transformation on the current sub-image block, it first determines the target transformation method and the target transformation parameters, and then performs illumination transformation on the current sub-image block using the target transformation method based on the target transformation parameters to obtain the sub-image block after illumination transformation.

[0104] In some embodiments, the target transformation mode is a default transformation mode.

[0105] Exemplarily, the default transformation method may be a logarithmic transformation method, a gamma transformation method, or other transformation methods. The default transformation method may even be a transformation method based on machine learning, and this application does not make any specific limitations on this.

[0106] Of course, in other alternative embodiments, the decoder may also decode the code stream to determine the target transformation mode, which is not specifically limited in this application.

[0107] In some embodiments, the target transformation parameters are transformation parameters used by default in the target transformation mode.

[0108] Of course, in other alternative embodiments, the decoder may also decode the code stream to determine the target transformation parameters, which is not specifically limited in this application.

[0109] In some embodiments, based on the target transformation mode and the target transformation parameters, the contrast of the attribute information of the pixels in the current sub-block is adjusted to obtain the sub-block after the illumination transformation; wherein the contrast between the attribute information of the pixels in the sub-block after the illumination transformation is greater than the contrast between the attribute information of the pixels in the current sub-block.

[0110] In other words, the target transformation parameters used by the current sub-block can increase the contrast between the attribute information of the pixels in the current sub-block.

[0111] Exemplarily, the contrast of the attribute information of the pixels in the current sub-block may be the contrast between texture information of the pixels in the current sub-block. In other words, the target transformation parameters used by the current sub-block can increase the contrast between texture information of the pixels in the current sub-block.

[0112] Exemplarily, the contrast of the attribute information of the pixels in the current sub-block may be the contrast between the color information of the pixels in the current sub-block. In other words, the target transformation parameters used by the current sub-block can increase the contrast between the color information of the pixels in the current sub-block.

[0113] It is worth noting that when the target transformation mode and the target transformation mode are fixed, the difference between the contrast between the attribute information of the pixels in the sub-image block after the illumination transformation and the contrast between the attribute information of the pixels in the current sub-image block is also fixed. Therefore, the decoder can also determine the difference between the contrast between the attribute information of the pixels in the sub-image block after the illumination transformation and the contrast between the attribute information of the pixels in the current sub-image block by decoding the code stream, and then determine the target transformation mode and the target transformation parameters based on the difference between the contrast between the attribute information of the pixels in the sub-image block after the illumination transformation and the contrast between the attribute information of the pixels in the current sub-image block, as well as the contrast of the sub-image block after the illumination transformation. This application does not make specific restrictions on this.

[0114] In some embodiments, the method 200 may further include:

[0115] Determine a decoded image of the current image based on the sub-image block after the illumination transformation.

[0116] Exemplarily, the decoder determines a decoded image of the additional image to which the current sub-image block belongs based on the sub-image block after the illumination transformation.

[0117] The decoding method according to the embodiment of the present application is described in detail above from the perspective of a decoder. The encoding method according to the embodiment of the present application will be described below from the perspective of an encoder.

[0118] FIG8 is a schematic flow chart of an encoding method 310 provided in an embodiment of the present application. It should be understood that the encoding method 310 can be performed by an encoder. For example, it can be applied to the encoding framework 13 shown in FIG1 . For ease of description, the following description uses an encoder as an example.

[0119] As shown in FIG8 , the decoding method 310 may include some or all of the following:

[0120] S311, determining a current sub-block in the current image;

[0121] S312, determining whether to perform illumination transformation on the current sub-block;

[0122] S313, when it is determined to perform illumination transformation on the current sub-image block, performing illumination transformation on the current sub-image block to obtain a sub-image block after illumination transformation;

[0123] S314: Encode the sub-image block after the illumination transformation to obtain a code stream.

[0124] Exemplarily, the current image may have an additional view.

[0125] Exemplarily, the current sub-image block may be an image block obtained by the encoder processing the additional view. For example, the encoder may remove redundant pixels between the base view and the additional view, that is, retain only valid pixels of non-repeated expression (also referred to as valid pixel clusters), and then divide and pack the valid pixel clusters to form image blocks of regular shapes (i.e., sub-image blocks), and then reorganize the base view and one or more sub-image blocks obtained based on the additional view (for example, splicing the base view and the sub-image blocks obtained based on the additional view) to form a larger rectangular image (also referred to as a spliced ​​image or spliced ​​image), which can be used to generate video data so that the encoder can encode the video data to obtain a bitstream.

[0126] Exemplarily, the current sub-image block may be a rectangular image block.

[0127] In this embodiment, by introducing illumination transformation on the current sub-image block that needs to undergo illumination transformation on the encoder side, the encoder is able to enhance the quality of the current sub-image block through illumination transformation of the current sub-image block. For the decoder, illumination transformation of the current sub-image block can not only restore the illumination effect of the current sub-image block, but also ensure that the sub-image block decoded by the decoder is the sub-image block that has undergone quality enhancement by the encoder. Furthermore, it can alleviate the problem of quality degradation of its pixel information after video encoding and decoding, for example, it can alleviate the problem of texture information loss of the sub-image block, thereby improving decoding performance.

[0128] In some embodiments, the S314 may include:

[0129] Obtaining a sub-block identifier of the current sub-block; the sub-block identifier of the current sub-block indicates whether to perform illumination transformation on the current sub-block;

[0130] The sub-image block identifier of the current sub-image block and the sub-image block after the illumination transformation are encoded to obtain the code stream.

[0131] In this embodiment, by writing the sub-block identifier of the current sub-block into the bitstream, it is helpful for the decoder to decode the bitstream and determine whether to perform illumination transformation with the current sub-block. This can ensure that the decoder and encoder have consistent understanding of whether illumination transformation is required for the current sub-block, thereby improving encoding and decoding efficiency.

[0132] Of course, in other alternative embodiments, other methods may be used to indicate to the decoder whether to perform illumination transformation on the current sub-block.

[0133] In one possible implementation, the encoder encodes the identifier of a sub-block requiring illumination transformation, so that the decoder determines whether to perform illumination transformation based on the decoded sub-block identifier. For example, the decoder may determine whether to perform illumination transformation on the current sub-block based on whether the bitstream carries the identifier of the current sub-block. For example, if the bitstream carries the identifier of the current sub-block, the decoder determines to perform illumination transformation on the current sub-block; if the bitstream does not carry the identifier of the current sub-block, the decoder determines not to perform illumination transformation on the current sub-block.

[0134] In another possible implementation, the encoder may also encode a pixel-level illumination change identifier or a view-level illumination change identifier. For example, the decoder may determine whether to perform an illumination change on the current pixel based on the identifier corresponding to the current pixel in the current sub-image block, where the identifier corresponding to the current pixel indicates whether to perform an illumination change on the current pixel. For another example, the decoder may also determine whether to perform an illumination change on the sub-image block in the additional view based on the identifier corresponding to the additional view to which the current sub-image block belongs; wherein the identifier corresponding to the additional illustration indicates whether to perform illumination changes with the sub-image block in the additional illustration.

[0135] In some embodiments, the S312 may include:

[0136] Get the base view;

[0137] Performing illumination detection on a pixel pair formed by pixels in the base view and pixels in the current sub-block to determine whether the pixel pair is a pixel pair with inconsistent illumination;

[0138] Based on the number of pixel pairs with inconsistent lighting, it is determined whether to perform lighting transformation on the current sub-block.

[0139] Exemplarily, the encoder may first map any pixel of the base view to the additional view to which the current sub-image block belongs based on the parameters of the camera used to capture the base view, the parameters of the camera used to capture the additional view to which the current sub-image block belongs, the depth information of the base view, and the depth information of the additional view to which the current sub-image block belongs, to obtain one or more pixel pairs formed by pixels in the base view and pixels in the current sub-image block.

[0140] Exemplarily, the pixels in the pixel pair from the current sub-image block, when the pixels in the pixel pair from the base view are projected or mapped to the current sub-image block, are pixels in the current sub-image block at the projection position or mapping position.

[0141] FIG9 is an example of a mapping relationship between pixels in a base view and pixel blocks where pixels in a current sub-block are located, provided by an embodiment of the present application.

[0142] As shown in Figure 9, assuming that view v0 is the base view and view v1 is the additional view to which the current sub-image block belongs, any pixel of view v0 can be mapped to view v1 based on the parameters of the camera used to capture view v0, the parameters of the camera used to capture view v1, the depth information of view v0, and the depth information of view v1, to obtain view v0, so as to obtain one or more pixel pairs formed by pixels in view v0 and pixels in the current sub-image block; for example, the figure exemplarily shows a pixel pair formed by pixel 1 in view v0 and pixel 2 in the current sub-image block.

[0143] In some embodiments, a distribution difference value between color information of pixels in the base view and color information of pixels in the current sub-block is determined; and based on the distribution difference value, it is determined whether the pixel pair is a pixel pair with inconsistent illumination.

[0144] Exemplarily, the distribution difference value may be used to characterize the difference between the lighting effects of the color information of the pixels in the base view and the lighting effects of the color information of the pixels in the current sub-block. For example, if the distribution difference value is smaller, it indicates that the lighting effects of the color information of the pixels in the base view are closer to the lighting effects of the color information of the pixels in the current sub-block, thereby indicating that the pixel pair is not a pixel pair with inconsistent lighting. Conversely, if the distribution difference value is larger, it indicates that the lighting effects of the color information of the pixels in the base view and the color information of the pixels in the current sub-block are more different, thereby indicating that the pixel pair is a pixel pair with inconsistent lighting.

[0145] In some embodiments, the maximum difference values ​​between (1-R1) / (1-R2), (1-G1) / (1-G2) and (1-B1) / (1-B2) are determined as the distribution difference values; wherein R1, G1, and B1 respectively represent the values ​​of the B component, G component, and B component of the pixels in the current sub-block; and R2, G2, and B2 respectively represent the values ​​of the B component, G component, and B component of the pixels in the base view.

[0146] Exemplarily, the encoder may first determine the difference between (1-R1) / (1-R2) and (1-G1) / (1-G2), the difference between (1-R1) / (1-R2) and (1-B1) / (1-B2), and the difference between (1-B1) / (1-B2) and (1-G1) / (1-G2), and determine the maximum difference between the difference between (1-R1) / (1-R2) and (1-G1) / (1-G2), the difference between (1-R1) / (1-R2) and (1-B1) / (1-B2), and the difference between (1-B1) / (1-B2) and (1-G1) / (1-G2) as the distribution difference value.

[0147] Since the lighting effect of the pixels in the current sub-block is closer to the lighting effect of the highlight, the values ​​of the B component, G component, and B component of the pixels in the current sub-block are closer to 1. Therefore, in the embodiment of the present application, the closer (1-R1) / (1-R2), (1-G1) / (1-G2), and (1-B1) / (1-B2) are, the closer the lighting effect between the color information of the pixels in the current sub-block is to the highlight effect, and the detection of the highlighted sub-block can be achieved, which is equivalent to the encoder performing lighting transformation on the highlighted sub-block, thereby alleviating the problem of quality degradation of its pixel information after video encoding and decoding, such as alleviating the problem of texture information loss of the sub-block, thereby improving decoding performance.

[0148] Of course, in other alternative embodiments, the encoder may also determine the distribution difference value by taking the difference between two of (1-R1) / (1-R2), (1-G1) / (1-G2), and (1-B1) / (1-B2). This application does not impose any specific limitation on this.

[0149] For example, the encoder may determine the distribution difference value by the difference between (1-R1) / (1-R2) and (1-G1) / (1-G2). For another example, the encoder may determine the distribution difference value by the difference between (1-R1) / (1-R2) and (1-B1) / (1-B2). For another example, the encoder may determine the distribution difference value by the difference between (1-B1) / (1-B2) and (1-G1) / (1-G2).

[0150] In some embodiments, if the distribution difference value is less than or equal to a first threshold, the pixel pair is determined to be a pixel pair with inconsistent illumination; if the distribution difference value is greater than the first threshold, the pixel pair is determined not to be a pixel pair with inconsistent illumination.

[0151] Exemplarily, the first threshold may be a preset threshold, such as a standard predefined threshold.

[0152] Of course, in other alternative embodiments, the encoder may also determine whether the pixel pair is a pair of pixels with inconsistent illumination based on the range of the distribution difference value. For example, if the distribution difference value falls within a first range, the pixel pair is determined to be a pair of pixels with inconsistent illumination; if the distribution difference value falls within a second range, the pixel pair is determined not to be a pair of pixels with consistent illumination. Optionally, the maximum value of the first range is less than or equal to the minimum value of the second range.

[0153] In some embodiments, based on the number of the pixel pairs with inconsistent lighting, a proportion of the pixel pairs with inconsistent lighting is determined; and based on the proportion of the pixel pairs with inconsistent lighting, it is determined whether to perform lighting transformation on the current sub-block.

[0154] Exemplarily, the encoder may determine the ratio of the illumination-inconsistent pixel pairs to the number of pixels in the current sub-block as the proportion of the illumination-inconsistent pixel pairs.

[0155] In some embodiments, if the proportion of the pixel pairs with inconsistent lighting is greater than or equal to a second threshold, the current sub-block is determined to be a sub-block that requires lighting transformation; if the proportion of the pixel pairs with inconsistent lighting is less than the second threshold, the current sub-block is determined to be a sub-block that does not require lighting transformation.

[0156] Exemplarily, the second threshold may be a predefined threshold, such as a standard predefined threshold.

[0157] In some embodiments, it is determined whether the pixel pair is a similar pixel pair; if the pixel pair is not a similar pixel pair, illumination detection is performed on the pixel pair to determine whether the pixel pair is a pixel pair with inconsistent illumination.

[0158] Exemplarily, the encoder triggers illumination detection on the pixel pair only when it determines that the pixel pair is not a similar pixel pair, and further determines whether the pixel pair is a pixel pair with inconsistent illumination.

[0159] In some embodiments, the S312 may include:

[0160] determining a depth difference value between the pixel pairs;

[0161] Determine a brightness difference value between a pixel in the base view and a pixel block where a pixel in the current sub-block is located;

[0162] When the depth difference value is less than a third threshold and the brightness difference value is less than a fourth threshold, determining that the pixel pair is a similar pixel pair;

[0163] When the depth difference value is greater than or equal to the third threshold or the brightness difference value is greater than or equal to the fourth threshold, it is determined that the pixel pair is not a similar pixel pair.

[0164] Exemplarily, the encoder may determine the difference value between the brightness of the pixel in the base view and the brightness of each pixel in the pixel block, and determine the maximum value of the determined difference values ​​as the maximum difference value. For example, assuming that the pixel block is a 3×3 pixel block centered on the pixel in the current sub-image block, the encoder may determine the difference value (i.e., 9 difference values) between the brightness of the pixel in the base view and the brightness of each pixel in the 3×3 pixel block and determine the maximum value of the determined 9 difference values ​​as the maximum difference value.

[0165] It is worth noting that the encoder may determine the depth difference value in a pixel-by-pixel manner and determine the brightness difference value in a pixel-by-block manner. However, this application is not specifically limited to this. For example, in other alternative embodiments, the encoder may also determine the brightness difference value in a pixel-by-pixel manner.

[0166] Of course, in other alternative embodiments, the encoder may also perform illumination detection on the pixel pair when the depth difference value is greater than or equal to the third threshold to determine whether the pixel pair is a pixel pair with inconsistent illumination. That is, if the encoder determines that the pixel pair is not a pixel pair with similar brightness, it may trigger illumination detection on the pixel pair to further determine whether the pixel pair is a pixel pair with inconsistent illumination.

[0167] FIG. 10 is an example of a similar pixel pair determination process 320 provided in an embodiment of the present application.

[0168] As shown in FIG10 , the determination process 320 may include:

[0169] S321 , the encoder determines whether the depth difference value is less than t1 .

[0170] For example, the encoder may calculate a depth difference value between a pixel in the base view and a pixel in the additional image, and determine whether the depth difference value is less than t1. For example, if the depth difference value is less than t1, the encoder determines that the pixel pair is a depth-similar pixel pair; otherwise, the encoder determines that the pixel pair is not a depth-similar pixel pair. The pixel in the base view may be pixel 1 as shown in FIG. 9 , and the pixel in the additional image may be pixel 2 as shown in FIG. 9 .

[0171] S322, the encoder determines whether the brightness difference value is less than t2.

[0172] Exemplarily, the encoder may determine a difference value between the brightness of a pixel in the base view and the brightness of each pixel in a pixel block in which the pixel in the additional image is located, and determine a maximum value among the determined difference values ​​as the maximum difference value. For example, assuming that the pixel block is a 3×3 pixel block centered on the pixel in the additional image, the encoder may determine a difference value (i.e., 9 difference values) between the brightness of the pixel in the base view and the brightness of each pixel in the 3×3 pixel block, and determine a maximum value among the determined 9 difference values ​​as the brightness difference value, and determine whether the brightness difference value is less than t2. For example, if the brightness difference value is less than t2, the encoder determines that the pixel pair is a pixel pair with similar brightness; otherwise, the encoder determines that the pixel pair is not a pixel pair with similar brightness.

[0173] S323, execute the second stage.

[0174] Exemplarily, the above S321 and S322 can be understood as the first stage of similarity detection on the pixel pairs. After executing the first stage, the second stage can be executed on the suspected pixel pairs output by the first stage. After the second stage is executed, the final similar pixel pairs can be determined. Optionally, when the encoder performs the second stage detection on the suspected similar pixel pairs, it can determine whether the suspected similar pixel pairs are similar pixels based on the color difference value between the suspected similar pixel pairs. It is worth noting that the execution step of the second stage can be an optional step.

[0175] In this embodiment, after the encoder uses a pixel pruning module to detect pixels in the additional view and the base view, the pixels can be pruned in subsequent processing, that is, only valid pixels with non-repeated expressions (also called valid pixel clusters) are retained, and then the valid pixel clusters are divided and packed to form regularly shaped image blocks (i.e., sub-blocks). Then, the base view and one or more sub-blocks obtained based on the additional view are reorganized (for example, the base view and the sub-blocks obtained based on the additional view are spliced) to form a larger rectangular image (also called a spliced ​​image or spliced ​​image). The spliced ​​image can be used to generate video data, so that the encoder can encode the video data to obtain a bitstream.

[0176] FIG. 11 is an example of combining illumination transformation with the determination process 330 of similar pixel pairs provided in an embodiment of the present application.

[0177] As shown in FIG. 11 , the determination process 330 may include:

[0178] S331 , the encoder determines whether the depth difference value is less than t1 .

[0179] For example, the encoder may calculate a depth difference value between a pixel in the base view and a pixel in the additional image, and determine whether the depth difference value is less than t1. For example, if the depth difference value is less than t1, the encoder determines that the pixel pair is a depth-similar pixel pair; otherwise, the encoder determines that the pixel pair is not a depth-similar pixel pair. The pixel in the base view may be pixel 1 as shown in FIG. 9 , and the pixel in the additional image may be pixel 2 as shown in FIG. 9 .

[0180] S332, the encoder determines whether the brightness difference value is less than t2.

[0181] Exemplarily, the encoder may determine a difference value between the brightness of a pixel in the base view and the brightness of each pixel in a pixel block in which the pixel in the additional image is located, and determine a maximum value among the determined difference values ​​as the maximum difference value. For example, assuming that the pixel block is a 3×3 pixel block centered on the pixel in the additional image, the encoder may determine a difference value (i.e., 9 difference values) between the brightness of the pixel in the base view and the brightness of each pixel in the 3×3 pixel block, and determine a maximum value among the determined 9 difference values ​​as the brightness difference value, and determine whether the brightness difference value is less than t2. For example, if the brightness difference value is less than t2, the encoder determines that the pixel pair is a pixel pair with similar brightness; otherwise, the encoder determines that the pixel pair is not a pixel pair with similar brightness.

[0182] S333, execute the second stage.

[0183] Exemplarily, the above S321 and S322 can be understood as the first stage of similarity detection on the pixel pairs. After executing the first stage, the second stage can be executed on the suspected pixel pairs output by the first stage. After the second stage is executed, the final similar pixel pairs can be determined. Optionally, when the encoder performs the second stage detection on the suspected similar pixel pairs, it can determine whether the suspected similar pixel pairs are similar pixels based on the color difference value between the suspected similar pixel pairs. It is worth noting that the execution step of the second stage can be an optional step.

[0184] S334 , the encoder determines whether the distribution difference value is less than t3 .

[0185] For example, the encoder may also perform illumination detection on the pixel pair when the depth difference value is greater than or equal to t1 to determine whether the pixel pair is a pixel pair with inconsistent illumination. That is, if the encoder determines that the pixel pair is not a pixel pair with similar brightness, it may trigger illumination detection on the pixel pair to further determine whether the pixel pair is a pixel pair with inconsistent illumination. For example, if the distribution difference value of the pixel pair is less than t3, the pixel pair is determined to be a pixel pair with inconsistent illumination; otherwise, the pixel pair is determined not to be a pixel pair with inconsistent illumination.

[0186] S335 : The encoder determines whether to perform illumination transformation on the current sub-block based on the detection result of the pixel pairs with inconsistent illumination.

[0187] S336: The encoder performs a mapping transformation on the current sub-block.

[0188] It should be understood that the above S335 and S336 can refer to the description in method 310, and to avoid repetition, they are not repeated here.

[0189] In this embodiment, after the encoder uses a pixel pruning module to detect pixels in the additional view and the base view, it can prune the pixels in subsequent processing, that is, only retain valid pixels with non-repeated expressions (also called valid pixel clusters). The valid pixel clusters are then divided and packed to form regularly shaped image blocks (i.e., sub-blocks). Then, illumination transformation is performed on the sub-blocks that require illumination transformation among the one or more obtained sub-blocks to obtain illumination-transformed sub-blocks. The base view and the sub-blocks obtained based on the additional view that do not require illumination transformation, as well as the illumination-transformed sub-blocks obtained after illumination transformation, are then reorganized (for example, the base view and the sub-blocks obtained based on the additional view are spliced) to form a larger rectangular image (also called a spliced ​​image or spliced ​​image). The spliced ​​image can be used to generate video data, so that the encoder can encode the video data to obtain a bitstream.

[0190] In some embodiments, a target transformation method used by the current sub-block is determined; target transformation parameters used by the current sub-block are determined; and based on the target transformation method and the target transformation parameters, an illumination transformation is performed on the current sub-block to obtain the sub-block after the illumination transformation.

[0191] Exemplarily, when the encoder determines to perform illumination transformation on the current sub-image block, it first determines the target transformation method and the target transformation parameters, and then performs illumination transformation on the current sub-image block using the target transformation method based on the target transformation parameters to obtain the sub-image block after illumination transformation.

[0192] In some embodiments, the target transformation mode is a default transformation mode.

[0193] Exemplarily, the default transformation method may be a logarithmic transformation method, a gamma transformation method, or other transformation methods. The default transformation method may even be a transformation method based on machine learning, and this application does not make any specific limitations on this.

[0194] Of course, in other alternative embodiments, the encoder may also write the target transformation mode into the bitstream, and this application does not make any specific limitation on this.

[0195] In some embodiments, the target transformation parameters are transformation parameters used by default in the target transformation mode.

[0196] Of course, in other alternative embodiments, the encoder may also write the target transformation parameters into the bitstream, which is not specifically limited in this application.

[0197] In some embodiments, based on the target transformation mode and the target transformation parameters, the contrast of the attribute information of the pixels in the current sub-block is adjusted to obtain the sub-block after the illumination transformation; wherein the contrast between the attribute information of the pixels in the sub-block after the illumination transformation is less than the contrast between the attribute information of the pixels in the current sub-block.

[0198] In other words, the target transformation parameters used by the current sub-block can reduce the contrast between the attribute information of the pixels in the current sub-block.

[0199] Exemplarily, the contrast of the attribute information of the pixels in the current sub-block may be the contrast between texture information of the pixels in the current sub-block. In other words, the target transformation parameters used by the current sub-block can reduce the contrast between texture information of the pixels in the current sub-block.

[0200] Exemplarily, the contrast of the attribute information of the pixels in the current sub-block may be the contrast between the color information of the pixels in the current sub-block. In other words, the target transformation parameters used by the current sub-block can reduce the contrast between the color information of the pixels in the current sub-block.

[0201] It is worth noting that since the encoder enhances the contrast between the attribute information of the pixels in the current sub-block, the decoder needs to weaken the contrast between the attribute information of the pixels in the current sub-block when restoring the attribute information of the pixels in the current sub-block. Of course, in other alternative embodiments, the encoder performs an illumination transformation on the current sub-block, and for the decoder, its illumination transformation on the current sub-block can also be understood as the encoder performing an inverse illumination transformation on the current sub-block. This application does not specifically limit this.

[0202] It is worth noting that when the target transformation mode and the target transformation mode are fixed, the difference between the contrast between the attribute information of the pixels in the sub-image block after the illumination transformation and the contrast between the attribute information of the pixels in the current sub-image block is also fixed. Therefore, the encoder can also determine the difference between the contrast between the attribute information of the pixels in the sub-image block after the illumination transformation and the contrast between the attribute information of the pixels in the current sub-image block by decoding the code stream, and then determine the target transformation mode and the target transformation parameters based on the difference between the contrast between the attribute information of the pixels in the sub-image block after the illumination transformation and the contrast between the attribute information of the pixels in the current sub-image block, as well as the contrast of the sub-image block after the illumination transformation. This application does not make specific restrictions on this.

[0203] In some embodiments, the sub-image block after illumination transformation is spliced ​​with a base view to obtain a spliced ​​image; and pixels in the spliced ​​image and pixels in the sub-image block after illumination transformation are encoded to obtain the code stream.

[0204] Exemplarily, the encoder may remove redundant pixels between the base view and the additional view, that is, retain only valid pixels that are not repeatedly expressed (also referred to as valid pixel clusters), and then divide and pack the valid pixel clusters to form image blocks of regular shapes (i.e., sub-blocks), and then perform illumination transformation on the sub-blocks that need to be illuminated in the one or more obtained sub-blocks, and obtain the sub-blocks after illumination transformation; and then reorganize the base view and the sub-blocks that do not need to be illuminated obtained based on the additional view, as well as the sub-blocks that need to be illuminated and obtained after illumination transformation (for example, the base view and the sub-blocks obtained based on the additional view are spliced) to form a larger rectangular image (also referred to as a spliced ​​image or spliced ​​image), which can be used to generate video data so that the encoder can encode the video data to obtain a bitstream.

[0205] The preferred embodiments of the present application are described in detail above in conjunction with the accompanying drawings. However, the present application is not limited to the specific details in the embodiments mentioned above. Within the technical concept of the present application, the technical solution of the present application can be subjected to a variety of simple modifications, and these simple modifications all fall within the scope of protection of the present application. For example, the various specific technical features described in the specific embodiments mentioned above can be combined in any suitable manner without contradiction. In order to avoid unnecessary repetition, the present application will no longer describe the various possible combinations separately. For another example, the various different embodiments of the present application can also be arbitrarily combined, as long as they do not violate the ideas of the present application, they should also be regarded as the contents disclosed in the present application. It should also be understood that in the various method embodiments of the present application, the size of the sequence numbers of the processes mentioned above does not mean the order of execution. The order of execution of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0206] The method embodiment of the present application is described in detail above. The following describes in detail an apparatus embodiment of the present application in conjunction with Figures 12 to 14.

[0207] FIG12 is a schematic block diagram of a decoder 400 according to an embodiment of the present application.

[0208] As shown in FIG12 , the decoder 400 may include:

[0209] A decoding unit 410 is configured to decode the code stream and determine a current sub-block in the current image;

[0210] A determining unit 420 is configured to determine whether to perform illumination transformation on the current sub-block;

[0211] The transformation unit 430 is configured to perform the illumination transformation on the current sub-block when determining to perform the illumination transformation on the current sub-block to obtain the illumination-transformed sub-block.

[0212] In some embodiments, the determining unit 420 is specifically configured to:

[0213] Decoding the code stream to determine a sub-block identifier of the current sub-block; the sub-block identifier of the current sub-block indicates whether to perform illumination transformation on the current sub-block;

[0214] Based on the sub-image block identifier of the current sub-image block, it is determined whether to perform illumination transformation on the current sub-image block.

[0215] In some embodiments, the transform unit 430 is specifically configured to:

[0216] Determining a target transformation mode used by the current sub-block;

[0217] Determining target transformation parameters used by the current sub-block;

[0218] Based on the target transformation mode and the target transformation parameters, the current sub-image block is subjected to illumination transformation to obtain the sub-image block after illumination transformation.

[0219] In some embodiments, the target transformation mode is a default transformation mode.

[0220] In some embodiments, the target transformation parameters are transformation parameters used by default in the target transformation mode.

[0221] In some embodiments, the transform unit 430 is specifically configured to:

[0222] Based on the target transformation mode and the target transformation parameters, the contrast of the attribute information of the pixels in the current sub-image block is adjusted to obtain the sub-image block after the illumination transformation; wherein the contrast between the attribute information of the pixels in the sub-image block after the illumination transformation is greater than the contrast between the attribute information of the pixels in the current sub-image block.

[0223] In some embodiments, the decoding unit 410 is further configured to:

[0224] Determine a decoded image of the current image based on the sub-image block after the illumination transformation.

[0225] FIG13 is a schematic block diagram of an encoder 500 according to an embodiment of the present application.

[0226] As shown in FIG13 , the encoder 500 may include:

[0227] A first determining unit 510 is configured to determine a current sub-block in a current image;

[0228] A second determining unit 520 is configured to determine whether to perform illumination transformation on the current sub-block;

[0229] a transforming unit 530 configured to, when determining to perform illumination transformation on the current sub-block, perform illumination transformation on the current sub-block to obtain a sub-block after illumination transformation;

[0230] The encoding unit 540 is configured to encode the sub-image block after the illumination transformation to obtain a bit stream.

[0231] In some embodiments, the encoding unit 540 is specifically configured to:

[0232] Obtaining a sub-block identifier of the current sub-block; the sub-block identifier of the current sub-block indicates whether to perform illumination transformation on the current sub-block;

[0233] The sub-image block identifier of the current sub-image block and the sub-image block after the illumination transformation are encoded to obtain the code stream.

[0234] In some embodiments, the second determining unit 520 is specifically configured to:

[0235] Get the base view;

[0236] Performing illumination detection on a pixel pair formed by pixels in the base view and pixels in the current sub-block to determine whether the pixel pair is a pixel pair with inconsistent illumination;

[0237] Based on the number of pixel pairs with inconsistent lighting, it is determined whether to perform lighting transformation on the current sub-block.

[0238] In some embodiments, the second determining unit 520 is specifically configured to:

[0239] Determining a distribution difference value between color information of pixels in the base view and color information of pixels in the current sub-block;

[0240] Based on the distribution difference value, it is determined whether the pixel pair is a pixel pair with inconsistent illumination.

[0241] In some embodiments, the second determining unit 520 is specifically configured to:

[0242] The maximum difference value between (1-R1) / (1-R2), (1-G1) / (1-G2) and (1-B1) / (1-B2) is determined as the distribution difference value; wherein R1, G1, and B1 respectively represent the values ​​of the B component, G component, and B component of the pixels in the current sub-block; R2, G2, and B2 respectively represent the values ​​of the B component, G component, and B component of the pixels in the base view.

[0243] In some embodiments, the second determining unit 520 is specifically configured to:

[0244] If the distribution difference value is less than or equal to a first threshold, determining that the pixel pair is a pixel pair with inconsistent illumination;

[0245] If the distribution difference value is greater than the first threshold, it is determined that the pixel pair is not a pixel pair with inconsistent illumination.

[0246] In some embodiments, the second determining unit 520 is specifically configured to:

[0247] Determining a proportion of the pixel pairs with inconsistent lighting based on the number of the pixel pairs with inconsistent lighting;

[0248] Based on the proportion of the pixel pairs with inconsistent lighting, it is determined whether to perform lighting transformation on the current sub-block.

[0249] In some embodiments, the second determining unit 520 is specifically configured to:

[0250] If the proportion of the pixel pairs with inconsistent illumination is greater than or equal to a second threshold, determining that the current sub-block is a sub-block that requires illumination transformation;

[0251] If the proportion of the pixel pairs with inconsistent illumination is less than the second threshold, the current sub-image block is determined to be a sub-image block that does not require illumination transformation.

[0252] In some embodiments, the second determining unit 520 is specifically configured to:

[0253] determining whether the pixel pair is a similar pixel pair;

[0254] When the pixel pair is not a similar pixel pair, light detection is performed on the pixel pair to determine whether the pixel pair is a pixel pair with inconsistent light.

[0255] In some embodiments, the second determining unit 520 is specifically configured to:

[0256] determining a depth difference value between the pixel pairs;

[0257] Determine a brightness difference value between a pixel in the base view and a pixel block where a pixel in the current sub-block is located;

[0258] When the depth difference value is less than a third threshold and the brightness difference value is less than a fourth threshold, determining that the pixel pair is a similar pixel pair;

[0259] When the depth difference value is greater than or equal to the third threshold or the brightness difference value is greater than or equal to the fourth threshold, it is determined that the pixel pair is not a similar pixel pair.

[0260] In some embodiments, the transform unit 530 is specifically configured to:

[0261] Determining a target transformation mode used by the current sub-block;

[0262] Determining target transformation parameters used by the current sub-block;

[0263] Based on the target transformation mode and the target transformation parameters, the current sub-image block is subjected to illumination transformation to obtain the sub-image block after illumination transformation.

[0264] In some embodiments, the target transformation mode is a default transformation mode.

[0265] In some embodiments, the target transformation parameters are transformation parameters used by default in the target transformation mode.

[0266] In some embodiments, the transform unit 530 is specifically configured to:

[0267] Based on the target transformation mode and the target transformation parameters, the contrast of the attribute information of the pixels in the current sub-image block is adjusted to obtain the sub-image block after the illumination transformation; wherein the contrast between the attribute information of the pixels in the sub-image block after the illumination transformation is smaller than the contrast between the attribute information of the pixels in the current sub-image block.

[0268] In some embodiments, the encoding unit 540 is specifically configured to:

[0269] Splicing the sub-image block after the illumination transformation with the base view to obtain a spliced ​​image;

[0270] Pixels in the spliced ​​image and pixels in the sub-image block after the illumination transformation are encoded to obtain the code stream.

[0271] It should be understood that the device embodiment and the method embodiment may correspond to each other, and similar descriptions may refer to the method embodiment. To avoid repetition, they will not be described here. Specifically, the decoder 400 shown in Figure 12 may correspond to the corresponding subject in the method 200 for executing the embodiment of the present application, and the aforementioned and other operations and / or functions of the various units in the decoder 400 are respectively for implementing the corresponding processes in the method 210. The encoder 500 shown in Figure 13 may correspond to the corresponding subject in the methods 310 to 330 for executing the embodiment of the present application, that is, the aforementioned and other operations and / or functions of the various units in the encoder 500 are respectively for implementing the corresponding processes in the methods 310 to 330.

[0272] It should also be understood that the various units in the decoder 400 or encoder 500 involved in the embodiment of the present application can be separately or all merged into one or several other units to constitute, or some (some) units therein can also be split into multiple smaller units in function to constitute, which can achieve the same operation without affecting the realization of the technical effect of the embodiment of the present application. The units involved in the above are divided based on logical functions. In practical applications, the function of a unit can also be implemented by multiple units, or the function of multiple units is implemented by one unit. In other embodiments of the present application, the decoder 400 or encoder 500 can also include other units. In practical applications, these functions can also be implemented with the assistance of other units, and can be implemented by the collaboration of multiple units. According to another embodiment of the present application, the decoder 400 or encoder 500 involved in the embodiment of the present application can be constructed by running a computer program (including program code) that can execute each step involved in the corresponding method on a general-purpose computing device including processing elements and storage elements such as a central processing unit (CPU), a random access storage medium (RAM), a read-only storage medium (ROM), etc., to construct the decoder 400 or encoder 500 involved in the embodiment of the present application, and to implement the encoding method or decoding method of the embodiment of the present application. The computer program can be recorded on, for example, a computer-readable storage medium, loaded into an electronic device via the computer-readable storage medium, and run therein to implement the corresponding method of the embodiment of the present application.

[0273] In other words, the units mentioned above can be implemented in the form of hardware, can be implemented by instructions in the form of software, or can be implemented in the form of a combination of hardware and software. Specifically, the steps of the method embodiments in the embodiments of the present application can be completed by the hardware integrated logic circuit and / or software instructions in the processor, and the steps of the method disclosed in the embodiments of the present application can be directly embodied as being executed by a hardware decoding processor, or being executed by a combination of hardware and software in a decoding processor. Optionally, the software can be located in a mature storage medium in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, an electrically erasable programmable memory, a register, etc. The storage medium is located in a memory, and the processor reads the information in the memory and completes the steps in the method embodiments mentioned above in combination with its hardware.

[0274] FIG14 is a schematic structural diagram of an electronic device 600 provided in an embodiment of the present application.

[0275] As shown in Figure 14, the electronic device 600 includes at least a processor 610 and a computer-readable storage medium 620. The processor 610 and the computer-readable storage medium 620 may be connected via a bus or other means. The computer-readable storage medium 620 is used to store a computer program 621, which includes computer instructions. The processor 610 is used to execute the computer instructions stored in the computer-readable storage medium 620. The processor 610 is the computing core and control core of the electronic device 600 and is suitable for implementing one or more computer instructions, specifically loading and executing one or more computer instructions to implement the corresponding method flow or corresponding function.

[0276] For example, the processor 610 may also be referred to as a central processing unit (CPU). The processor 610 may include, but is not limited to, a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, transistor logic devices, discrete hardware components, and the like.

[0277] Exemplarily, the computer-readable storage medium 620 may be a high-speed RAM memory, or a non-volatile memory (Non-Volatile Memory), such as at least one disk memory; optionally, it may be at least one computer-readable storage medium located away from the aforementioned processor 610. Specifically, the computer-readable storage medium 620 includes, but is not limited to: volatile memory and / or non-volatile memory. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct RAM bus random access memory (DR RAM).

[0278] Exemplarily, the electronic device 600 may be an encoder or encoding framework involved in an embodiment of the present application; a first computer instruction is stored in the computer-readable storage medium 620; the processor 610 loads and executes the first computer instruction stored in the computer-readable storage medium 620 to implement the corresponding steps in the encoding method provided in an embodiment of the present application; in other words, the first computer instruction in the computer-readable storage medium 620 is loaded by the processor 610 and the corresponding steps are executed. To avoid repetition, it will not be repeated here.

[0279] Exemplarily, the electronic device 600 may be a decoder or decoding framework involved in an embodiment of the present application; a second computer instruction is stored in the computer-readable storage medium 620; the processor 610 loads and executes the second computer instruction stored in the computer-readable storage medium 620 to implement the corresponding steps in the decoding method provided in an embodiment of the present application; in other words, the second computer instruction in the computer-readable storage medium 620 is loaded by the processor 610 and the corresponding steps are executed, which will not be repeated here to avoid repetition.

[0280] According to another aspect of the present application, the present application also provides a coding and decoding system, including the encoder and decoder mentioned above.

[0281] According to another aspect of the present application, the present application also provides a computer-readable storage medium (Memory), which is a memory device in the electronic device 600 for storing programs and data. For example, a computer-readable storage medium 620. It can be understood that the computer-readable storage medium 620 here can include both the built-in storage medium in the electronic device 600 and, of course, the extended storage medium supported by the electronic device 600. The computer-readable storage medium provides a storage space, which stores the operating system of the electronic device 600. In addition, one or more computer instructions suitable for being loaded and executed by the processor 610 are also stored in the storage space. These computer instructions can be one or more computer programs 621 (including program codes).

[0282] According to another aspect of the present application, a computer program product or computer program is provided. The computer program product or computer program includes computer instructions stored in a computer-readable storage medium. For example, computer program 621 is provided. In this case, data processing device 600 may be a computer, and processor 610 reads the computer instructions from computer-readable storage medium 620. Processor 610 executes the computer instructions, causing the computer to perform the encoding method or decoding method provided in the various optional embodiments described above.

[0283] In other words, when implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process of the embodiment of the present application is run in whole or in part or the functions of the embodiment of the present application are implemented. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website, computer, server, or data center to another website, computer, server, or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode.

[0284] Those skilled in the art will appreciate that the units and process steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0285] Finally, it should be noted that the above content is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A decoding method, characterized in that: include: Decode the code stream and determine the current sub-block in the current image; Determining whether to perform illumination transformation on the current sub-tile; When it is determined to perform illumination transformation on the current sub-image block, illumination transformation is performed on the current sub-image block to obtain a sub-image block after illumination transformation.

2. The method according to claim 1, characterized in that The determining whether to perform illumination transformation on the current sub-tile includes: Decoding the code stream to determine a sub-block identifier of the current sub-block; the sub-block identifier of the current sub-block indicates whether to perform illumination transformation on the current sub-block; Based on the sub-image block identifier of the current sub-image block, it is determined whether to perform illumination transformation on the current sub-image block.

3. The method according to claim 1 or 2, characterized in that The performing illumination transformation on the current sub-image block to obtain an illumination-transformed sub-image block includes: Determining a target transformation mode used by the current sub-block; Determining target transformation parameters used by the current sub-block; Based on the target transformation mode and the target transformation parameters, the current sub-image block is subjected to illumination transformation to obtain the sub-image block after illumination transformation.

4. The method according to claim 3, characterized in that The target transformation mode is a default transformation mode.

5. The method according to claim 3, characterized in that The target transformation parameters are transformation parameters used by default in the target transformation mode.

6. The method according to claim 3, characterized in that The performing illumination transformation on the current sub-image block based on the target transformation mode and the target transformation parameters to obtain the sub-image block after illumination transformation includes: Based on the target transformation mode and the target transformation parameters, the contrast of the attribute information of the pixels in the current sub-image block is adjusted to obtain the sub-image block after the illumination transformation; wherein the contrast between the attribute information of the pixels in the sub-image block after the illumination transformation is greater than the contrast between the attribute information of the pixels in the current sub-image block.

7. The method according to any one of claims 1 to 6, characterized in that The method further comprises: Determine a decoded image of the current image based on the sub-image block after the illumination transformation.

8. A coding method, characterized in that include: Determine the current sub-block in the current image; Determining whether to perform illumination transformation on the current sub-tile; When determining to perform illumination transformation on the current sub-image block, performing illumination transformation on the current sub-image block to obtain an illumination-transformed sub-image block; The sub-image block after the illumination transformation is encoded to obtain a code stream.

9. The method according to claim 8, characterized in that The encoding of the sub-image block after the illumination transformation to obtain a code stream includes: Obtaining a sub-block identifier of the current sub-block; the sub-block identifier of the current sub-block indicates whether to perform illumination transformation on the current sub-block; The sub-image block identifier of the current sub-image block and the sub-image block after the illumination transformation are encoded to obtain the code stream.

10. The method according to claim 8, characterized in that The determining whether to perform illumination transformation on the current sub-tile includes: Get the base view; Performing illumination detection on a pixel pair formed by pixels in the base view and pixels in the current sub-block to determine whether the pixel pair is a pixel pair with inconsistent illumination; Based on the number of pixel pairs with inconsistent lighting, it is determined whether to perform lighting transformation on the current sub-block.

11. The method according to claim 10, characterized in that The performing illumination detection on a pixel pair formed by pixels in the base view and pixels in the current sub-block to determine whether the pixel pair is a pixel pair with inconsistent illumination includes: Determining a distribution difference value between color information of pixels in the base view and color information of pixels in the current sub-block; Based on the distribution difference value, it is determined whether the pixel pair is a pixel pair with inconsistent illumination.

12. The method according to claim 11, characterized in that The determining of a distribution difference value between color information of pixels in the base view and color information of pixels in the current sub-block includes: The maximum difference value between (1-R1) / (1-R2), (1-G1) / (1-G2) and (1-B1) / (1-B2) is determined as the distribution difference value; wherein R1, G1, and B1 respectively represent the values ​​of the B component, G component, and B component of the pixels in the current sub-block; R2, G2, and B2 respectively represent the values ​​of the B component, G component, and B component of the pixels in the base view.

13. The method according to claim 11, characterized in that The determining, based on the distribution difference value, whether the pixel pair is a pixel pair with inconsistent illumination includes: If the distribution difference value is less than or equal to a first threshold, determining that the pixel pair is a pixel pair with inconsistent illumination; If the distribution difference value is greater than the first threshold, it is determined that the pixel pair is not a pixel pair with inconsistent illumination.

14. The method according to claim 10, characterized in that The determining whether to perform illumination transformation on the current sub-block based on the number of pixel pairs with inconsistent illumination includes: Determining a proportion of the pixel pairs with inconsistent lighting based on the number of the pixel pairs with inconsistent lighting; Based on the proportion of the pixel pairs with inconsistent lighting, it is determined whether to perform lighting transformation on the current sub-block.

15. The method according to claim 14, characterized in that The determining whether to perform illumination transformation on the current sub-block based on the proportion of the pixel pairs with inconsistent illumination includes: If the proportion of the pixel pairs with inconsistent illumination is greater than or equal to a second threshold, determining that the current sub-block is a sub-block that requires illumination transformation; If the proportion of the pixel pairs with inconsistent illumination is less than the second threshold, the current sub-image block is determined to be a sub-image block that does not require illumination transformation.

16. The method according to claim 10, characterized in that The performing illumination detection on a pixel pair formed by pixels in the base view and pixels in the current sub-block to determine whether the pixel pair is a pixel pair with inconsistent illumination includes: determining whether the pixel pair is a similar pixel pair; When the pixel pair is not a similar pixel pair, light detection is performed on the pixel pair to determine whether the pixel pair is a pixel pair with inconsistent light.

17. The method according to claim 16, characterized in that Determining whether the pixel pair is a similar pixel pair includes: determining a depth difference value between the pixel pairs; Determine a brightness difference value between a pixel in the base view and a pixel block where a pixel in the current sub-block is located; When the depth difference value is less than a third threshold and the brightness difference value is less than a fourth threshold, determining that the pixel pair is a similar pixel pair; When the depth difference value is greater than or equal to the third threshold or the brightness difference value is greater than or equal to the fourth threshold, it is determined that the pixel pair is not a similar pixel pair.

18. The method according to any one of claims 8 to 17, characterized in that The performing illumination transformation on the current sub-image block to obtain an illumination-transformed sub-image block includes: Determining a target transformation mode used by the current sub-block; Determining target transformation parameters used by the current sub-block; Based on the target transformation mode and the target transformation parameters, the current sub-image block is subjected to illumination transformation to obtain the sub-image block after illumination transformation.

19. The method according to claim 18, characterized in that The target transformation mode is a default transformation mode.

20. The method according to claim 18, wherein The target transformation parameters are transformation parameters used by default in the target transformation mode.

21. The method according to claim 18, wherein The performing illumination transformation on the current sub-image block based on the target transformation mode and the target transformation parameters to obtain the sub-image block after illumination transformation includes: Based on the target transformation mode and the target transformation parameters, the contrast of the attribute information of the pixels in the current sub-image block is adjusted to obtain the sub-image block after the illumination transformation; wherein the contrast between the attribute information of the pixels in the sub-image block after the illumination transformation is smaller than the contrast between the attribute information of the pixels in the current sub-image block.

22. The method according to any one of claims 8 to 21, characterized in that The encoding of the sub-image block after the illumination transformation to obtain a code stream includes: Splicing the sub-image block after the illumination transformation with the base view to obtain a spliced ​​image; Pixels in the spliced ​​image and pixels in the sub-image block after the illumination transformation are encoded to obtain the code stream.

23. A decoder, characterized in that include: A decoding unit, configured to decode the code stream and determine a current sub-block in the current image; a determining unit, configured to determine whether to perform illumination transformation on the current sub-block; The transformation unit is configured to, when determining to perform illumination transformation on the current sub-image block, perform illumination transformation on the current sub-image block to obtain a sub-image block after illumination transformation.

24. An encoder, characterized in that include: A first determining unit, configured to determine a current sub-block in a current image; A second determining unit, configured to determine whether to perform illumination transformation on the current sub-block; a transformation unit, configured to, when determining to perform illumination transformation on the current sub-block, perform illumination transformation on the current sub-block to obtain a sub-block after illumination transformation; The encoding unit is used to encode the sub-image block after the illumination transformation to obtain a code stream.

25. An electronic device, characterized in that: include: a processor adapted to execute a computer program; A computer-readable storage medium having a computer program stored therein, wherein when the computer program is executed by the processor, the method according to any one of claims 1 to 7 or the method according to any one of claims 8 to 21 is implemented.

26. A computer-readable storage medium, characterized in that Used to store a computer program, the computer program causing a computer to execute the method according to any one of claims 1 to 7 or the method according to any one of claims 8 to 21.

27. A computer program product comprising a computer program / instructions, characterized in that The computer program / instructions, when executed by a processor, implement the method according to any one of claims 1 to 7 or the method according to any one of claims 8 to 21.

28. A code stream, characterized in that The code stream is a code stream in the method according to any one of claims 1 to 7 or a code stream generated by the method according to any one of claims 8 to 21.

Images