A method and apparatus for processing image blocks in immersive media encoding

Abstract

The application provides a method for processing image blocks in immersive media coding, comprising: for at least one image block in a multi-view image, for at least one split line in the image block, the split line is through and perpendicular to the boundary of the image block, calculating the effective information density of image sub-blocks on both sides of the split line respectively, the effective information is the additional information of the image sub-block in the multi-view relative to other views; when the image sub-blocks on both sides of the split line satisfy a cutting condition, the cutting condition is that the effective information density of one side of the image sub-block is greater than a threshold value and the effective information density of the other side of the image sub-block is less than the threshold value: cutting along the split line, cutting the image block into two image blocks, for the image block containing the image sub-block with the effective information density less than the threshold value, if there is no split line satisfying the cutting condition, the image block is discarded and is not assembled into a spliced image; otherwise: no cutting is performed.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of immersive media coding, and in particular to a processing method and device for multi-view image blocks. BACKGROUND

[0002] As a subjective evaluation, "immersion" refers to the viewer's perceptual sense of immersion in the virtual scene established and displayed by the multimedia system. With the increasing capabilities of acquisition devices and display devices year by year, immersive media, as a kind of visual multimedia that can bring strong immersion to the viewer, has become a research hotspot in the industry and the scientific community. Multi-view image plus depth information is an effective immersive media expression method, which is composed of texture images of multiple viewpoints and depth images corresponding to each texture image. Using depth image-based view synthesis technology, this expression method can render the image of the target viewpoint according to the camera parameters of the target image and the positional relationship between the target viewpoint and the existing viewpoints.

[0003] Due to the large information redundancy between multiple viewpoints, encoding and transmitting multi-view sub-block spliced images can significantly save the pixel rate. Before encoding and decoding, by analyzing the geometric texture relationship between multiple viewpoints, the main viewpoint (some images in the multi-view image that contain complete viewpoint information) image is used to eliminate as much as possible the redundant pixels of other viewpoints, so that only the unique effective information is retained in the images of other viewpoints except the main viewpoint. Considering the coding efficiency, the retained effective pixels are aggregated into rectangular sub-blocks, and finally multiple image blocks are assembled into a spliced image for encoding and transmission. At the decoding end, all block images are extracted from the decoded multi-view block spliced image using the decoded image block information, and finally the target image synthesis is performed in units of image blocks.

[0004] However, as the field of view of the video content becomes larger and the number of source viewpoints increases, the coding and transmission scheme based on spliced images exposes the deficiency in processing capacity when the input data volume is large. In many industrial scenarios, the pixel rate of the spliced image is limited, and the necessary sub-block images left after inter-view redundancy removal cannot be put into the limited size of the map set. The loss of sub-block images leads to information loss during decoding, and thus high-quality video cannot be synthesized. Therefore, in order to measure the importance of the image blocks for decision-making, the concept of effective information amount of pixels is introduced, which measures the non-overlapping degree of the current pixel with the pixels in other viewpoints. The higher the effective information amount, the lower the degree of representation by the pixels in other viewpoints, the more unique information, and the more it should be retained in the spliced image.

[0005] More specifically, the effective information usually includes two channels: geometry information and texture information, the value of each channel is independent of each other, and is calculated by the logarithm of the difference of the disparity and the difference of the luminance of the current pixel and the reference pixel at the current pixel position. If the current pixel has no reference pixel, the information amount is the maximum value, which is determined by the bit width of the disparity and the luminance. The specific calculation method is as follows: let the disparity difference of the current pixel and the reference pixel be d g , and the luminance difference be d t , then the effective information p is:

[0006] p = log d g + log d t

[0007] The geometry bit width b g and the luminance bit width b t of the video, then the maximum value p max of the effective information is:

[0008]

[0009] Since the reserved pixels are aggregated into image subblocks, the effective information of an image subblock is calculated by accumulating the effective information values of all the effective pixels in the image subblock.

[0010] p block = ∑ pixel∈block p pixel

[0011] Further, the concept of effective information density p is proposed, which is the effective information of a subblock divided by the number of effective pixels

[0012]

[0013] In the process of subblock image stitching, the effective information density p of the subblock is used as the basis to preferentially reserve the image subblocks with large effective information density under the limitation of the priority pixel rate, so as to improve the information utilization rate and the rendering quality at the decoding end.

[0014] However, the distribution of effective information in an image sub-block can be uneven, and selecting a priority reserved image sub-block according to the effective information density mu can discard local areas with high effective information density in some sub-blocks, or in the case where local areas with low effective information density are reserved in the reserved image sub-block, the pixel rate is wasted. Therefore, an image sub-block cutting method based on the distribution of effective information is needed, so that the distribution of effective information in the same sub-block after cutting is as uniform as possible, and the information rate can be improved when the mosaic image is organized, and the coding efficiency is further improved. However, how to make the cutting more sufficient, how to reduce invalid cutting, and how to prevent the continuity of the image region from being destroyed to affect the coding efficiency when the effective information density of two sub-blocks is greater than the reserved threshold are all difficult problems and challenges. SUMMARY

[0015] To solve the above technical problems, the present application provides a method and device for processing image blocks in immersive media coding.

[0016] For at least one image block in a multi-view image, for at least one split line in the image block that penetrates and is perpendicular to the boundary of the image block, the effective information density of the image sub-blocks on both sides of the split line is calculated respectively, and the effective information is the additional information of the image sub-block relative to other views in multi-view.

[0017] When the image sub-blocks on both sides of the split line satisfy the cutting condition, the cutting condition is that the effective information density of one side of the image sub-block is greater than the threshold and the effective information density of the other side of the image sub-block is less than the threshold, and the cutting is performed along the split line to cut the image block into two image blocks. For the image block containing an image sub-block with an effective information density less than the threshold, if there is no split line that satisfies the cutting condition, the image block is discarded and not assembled into a mosaic image; otherwise, no cutting is performed.

[0018] Further, for the image block containing an image sub-block with an effective information density less than the threshold after cutting, if there is a split line that satisfies the cutting condition, the image block is cut again.

[0019] Further, for the image block with multiple split lines with image sub-blocks on both sides satisfying the cutting condition, the split line with the maximum difference in effective information density of the image sub-blocks on both sides is selected as the actual split line.

[0020] Further, when the image sub-blocks on both sides of the split line satisfy the cutting condition, the threshold is updated according to the pixel rate limit of the mosaic image and the effective information density of the non-discarded image block.

[0021] Further, the present application provides a device for processing image blocks, comprising the following modules:

[0022] A calculating effective information density module is configured to calculate the effective information density of image sub-blocks on both sides of a split line that runs through and is perpendicular to a boundary of a multi-view image block, wherein the effective information is extra information of the image sub-blocks in the multi-view relative to other views

[0023] A threshold judging module is configured to judge whether the effective information density of the image sub-blocks on both sides of the split line meets a threshold condition, wherein if the condition is met, the split line is cut, and if the condition is not met, the split line is not cut

[0024] A split line decision module is configured to select a split line with the largest difference in effective information density of image sub-blocks on both sides of the split line as an actual split line for an image block with multiple split lines that meet the cutting condition

[0025] A cutting module is configured to cut along the split line if the cutting condition is met

[0026] An assembling and splicing decision module is configured to discard an image block that contains image sub-blocks with effective information density less than a threshold and does not have a split line that meets the cutting condition, and not assemble the image block into a spliced image

[0027] A threshold updating module is configured to update the threshold according to a pixel rate limit of the spliced image and the effective information density of the image blocks that are not discarded when the image sub-blocks on both sides of the split line meet the cutting condition

[0028] Further, the image block cutting device is configured to re-cut an image block that contains image sub-blocks with effective information density less than a threshold if there is a split line that meets the cutting condition.

[0029] The present application has the advantages that, through the judgment of image sub-block granularity, sufficient cutting of image blocks can be realized, and image blocks with higher effective information density can be organized under the limitation of limited pixel rate in the multi-view image splicing process, the information utilization rate of the spliced image is improved, and the rendering quality is improved at the decoding end; meanwhile, the threshold controls the occurrence of cutting, reduces invalid cutting in the scenario without discarding, and saves the encoding code rate; the threshold is updated according to the pixel rate limit and the effective information density of the image sub-blocks that are not discarded after each cutting, so that the threshold is more accurate, and the overall encoding efficiency is increased. BRIEF DESCRIPTION OF DRAWINGS

[0030] The principles of the present application can be explained in terms of the embodiments set forth below in connection with the drawings.

[0031] The drawings described herein are used to provide further understanding of the present application, form a part of the present application, and are used to explain the present application, which is only some specific examples, and the application scope of the present application is not limited to these examples.

[0032] Figure 1 This is a schematic diagram of three image sub-blocks A, B, and C of image block ABC in one embodiment of the present invention.

[0033] Figure 2 This is a schematic diagram of three image sub-blocks A, B, and C of image block ABC in another embodiment of the present invention.

[0034] Figure 3 This is a schematic diagram of three image sub-blocks A, B, and C of image block ABC in another embodiment of the present invention.

[0035] Figure 4 This is a schematic diagram of the cutting result in one embodiment of the present invention, wherein the image block inside the dashed box is discarded.

[0036] Figure 5 This is a schematic diagram of the cutting result in another embodiment of the present invention, wherein the image block inside the dashed box is discarded.

[0037] Figure 6 This is a schematic diagram of the cutting result in another embodiment of the present invention, wherein the image block inside the dashed box is discarded.

[0038] Figure 7 This is a schematic diagram of the cutting result in another embodiment of the present invention, wherein the image block inside the dashed box is discarded.

[0039] Figure 8 This is a schematic diagram of the cutting result in another embodiment of the present invention, wherein the image block inside the dashed box is discarded.

[0040] Figure 9 This is a schematic diagram of the cutting result in another embodiment of the present invention, wherein the image block inside the dashed box is discarded. Detailed Implementation

[0041] Example 1

[0042] like Figure 1 As shown, A, B, and C are three image sub-blocks of image block ABC. The colors of the image sub-blocks, from light to dark, represent the lowest to highest effective density of the corresponding image sub-blocks.

[0043] In this example, the initial threshold is between the effective information densities of image sub-block A and image sub-block B.

[0044] The common boundary between image sub-block A and image sub-block B is a dividing line that runs through and is perpendicular to the boundary of the image block. The effective information density of image sub-block A is less than the threshold, and the effective information density of image sub-block B is greater than the threshold, thus satisfying the cutting condition. The common boundary between image sub-block B and image sub-block C is a dividing line that runs through and is perpendicular to the boundary of the image block, but the effective information density of both image sub-block B and image sub-block C is greater than the threshold, thus not satisfying the cutting condition.

[0045] Therefore, using the common boundary between image sub-block A and image sub-block B as the dividing line, image block ABC is cut into image block A and image block BC; since the effective information density of image block A is less than the threshold and there is no dividing line that satisfies the cutting conditions, it is discarded and not assembled into the stitched image.

[0046] Remove image block ABC, add image block BC, recalculate the threshold, and update the threshold to the minimum effective information density of the image blocks and image sub-blocks that were not discarded within the pixel rate limit of the mosaic.

[0047] In this example, the updated threshold is less than the effective information density of image sub-block B.

[0048] For the image block BC generated after cutting, repeat the above process. The common boundary between image sub-block B and image sub-block C is a dividing line that runs through and is perpendicular to the boundary of the image block. However, the effective information density of image sub-block B and image sub-block C is greater than the threshold, and the cutting condition is not met.

[0049] Cutting terminated, such as Figure 4 As shown, the area within the dashed box represents the discarded image blocks. Ultimately, image block ABC is cut into image block A and image block BC, with image block A being discarded and not included in the stitched image.

[0050] Example 2

[0051] like Figure 1 As shown, A, B, and C are three image sub-blocks of image block ABC. The colors of the image sub-blocks, from light to dark, represent the lowest to highest effective density of the corresponding image sub-blocks.

[0052] In this example, the initial threshold is between the effective information densities of image sub-block B and image sub-block C.

[0053] The common boundary between image sub-block A and image sub-block B is a dividing line that runs through and is perpendicular to the boundary of the image block, but the effective information density of both image sub-block A and image sub-block B is less than the threshold, so the cutting condition is not met; the common boundary between image sub-block B and image sub-block C is a dividing line that runs through and is perpendicular to the boundary of the image block, and the effective information density of image sub-block B is less than the threshold, while the effective information density of image sub-block C is greater than the threshold, so the cutting condition is met.

[0054] Therefore, the image block ABC is cut into image block AB and image block C with the common boundary of image sub-block B and image sub-block C as the cutting line; since image block AB contains image sub-block A with effective information density less than the threshold, but the effective information density of image sub-block A and B are both less than the threshold, there is no cutting line satisfying the cutting condition, thus image block AB is discarded and not assembled into the mosaic image.

[0055] The cutting is terminated, and the threshold is not updated. Figure 5 As shown in the figure, the image block inside the dashed box is discarded, and the final image block ABC is cut into image block AB and image block C, wherein image block AB is discarded and not entered into the mosaic image.

[0056] The image block ABC is removed, and image block C is added, and the threshold is recalculated, and the threshold is updated to the minimum value of the effective information density of the image blocks and image sub-blocks that are not discarded within the pixel rate limit of the mosaic image.

[0057] Embodiment 3

[0058] As shown in the figure, A, B, and C are three image sub-blocks of image block ABC, and the colors of the image sub-blocks from light to dark represent the lowest to the highest effective density of the corresponding image sub-blocks. Figure 1 In this example, the initial threshold is between the effective information density of image sub-block A and image sub-block B.

[0059] The common boundary of image sub-block A and image sub-block B is a cutting line that penetrates and is perpendicular to the image block boundary, but the effective information density of image sub-block A and image sub-block B is greater than the threshold, and the cutting condition is not met; the common boundary of image sub-block B and image sub-block C is a cutting line that penetrates and is perpendicular to the image block boundary, but the effective information density of image sub-block B and image sub-block C is greater than the threshold, and the cutting condition is not met.

[0060] Therefore, no cutting is performed. The cutting is terminated, and the threshold is not updated.

[0061] By this method, the invalid cutting of image block ABC can be reduced, and the coding rate can be saved.

[0062] Embodiment 4

[0063] As shown in the figure, A, B, and C are three image sub-blocks of image block ABC, and the colors of the image sub-blocks from light to dark represent the lowest to the highest effective density of the corresponding image sub-blocks.

[0064] Figure 1 In this example, the initial threshold is between the effective information density of image sub-block A and image sub-block B.

[0065] In this example, the initial threshold is between the effective information density of image sub-block A and image sub-block B.

[0066] ​The common boundary of the image sub-block A and the image sub-block B is a split line running through and perpendicular to the image block boundary, and the effective information density of the image sub-block A is less than the threshold value, and the effective information density of the image sub-block B is greater than the threshold value, satisfying the cutting condition; the common boundary of the image sub-block B and the image sub-block C is a split line running through and perpendicular to the image block boundary, but the effective information density of the image sub-block B and the image sub-block C are both greater than the threshold value, not satisfying the cutting condition.

[0067] Therefore, the image block ABC is cut into the image block A and the image block BC according to the common boundary of the image sub-block A and the image sub-block B as the split line; since the effective information density of the image block A is less than the threshold value, and there is no split line satisfying the cutting condition, the image block A is discarded and not assembled into the mosaic image.

[0068] The image block ABC is removed, the image block C is added, the threshold value is recalculated, and the threshold value is updated as the minimum value of the effective information density of the image blocks and the image sub-blocks that are not discarded within the pixel rate limit of the mosaic image.

[0069] In this example, the updated threshold value is between the effective information density of the image sub-block B and the image sub-block C.

[0070] For the image block BC generated after cutting, the above process is repeated, the common boundary of the image sub-block B and the image sub-block C is a split line running through and perpendicular to the image block boundary, and the effective information density of the image sub-block B is less than the threshold value, and the effective information density of the image sub-block C is greater than the threshold value, satisfying the cutting condition.

[0071] Therefore, the image block BC is cut into the image block B and the image block C according to the common boundary of the image sub-block B and the image sub-block C as the split line; since the effective information density of the image block B is less than the threshold value, and there is no split line satisfying the cutting condition, the image block B is discarded and not assembled into the mosaic image.

[0072] The cutting is terminated, as shown in the figure, the image blocks within the dashed box are discarded, and finally the image block ABC is cut into the image block A, the image block B and the image block C, wherein the image sub-block A and the image block B are discarded and not assembled into the mosaic image. Figure 6

[0073] The image block BC is removed, the image block C is added, the threshold value is recalculated, and the threshold value is updated as the minimum value of the effective information density of the image blocks and the image sub-blocks that are not discarded within the pixel rate limit of the mosaic image.

[0074] Through this method, the threshold value is updated according to the pixel rate limit and the effective information density of the image sub-blocks that are not discarded after each cutting, so that the threshold value is more accurate, the cutting is more sufficient, and the overall coding efficiency is increased.

[0075] Example 5

[0076] ​like Figure 2 As shown, A, C, and B are three image sub-blocks of image block ACB. The colors of the image sub-blocks, from light to dark, represent the lowest to highest effective density of the corresponding image sub-blocks.

[0077] In this example, the initial threshold is between the effective information densities of image sub-block B and image sub-block C.

[0078] The common boundary between image sub-block A and image sub-block C is a dividing line that runs through and is perpendicular to the boundary of the image block. The effective information density of image sub-block A is less than a threshold, and the effective information density of image sub-block C is greater than a threshold, thus satisfying the cutting condition. The common boundary between image sub-block B and image sub-block C is a dividing line that runs through and is perpendicular to the boundary of the image block. The effective information density of image sub-block B is less than a threshold, and the effective information density of image sub-block C is greater than a threshold, thus satisfying the cutting condition.

[0079] Since the effective information density difference between image sub-block A and image sub-block C is greater, the common boundary between image sub-block A and image sub-block C is used as the actual dividing line, and image block ACB is cut into image block A and image block CB; since the effective information density of image block A is less than the threshold and there is no dividing line that satisfies the cutting conditions, it is discarded and not assembled into the stitched image.

[0080] Remove image block ACB, add image block CB, recalculate the threshold, and update the threshold to the minimum effective information density of the image blocks and image sub-blocks that were not discarded within the pixel rate limit of the mosaic.

[0081] In this example, the updated threshold lies between the effective information densities of image sub-block C and image sub-block B.

[0082] For the image block CB generated after cutting, repeat the above process. The common boundary between image sub-block C and image sub-block B is a dividing line that runs through and is perpendicular to the boundary of the image block. The effective information density of image sub-block B is less than the threshold, and the effective information density of image sub-block C is greater than the threshold, thus satisfying the cutting condition.

[0083] Therefore, using the common boundary between image sub-block C and image sub-block B as the dividing line, image block CB is cut into image block C and image block B; since the effective information density of image block B is less than the threshold and there is no dividing line that satisfies the cutting conditions, it is discarded and not assembled into the stitched image.

[0084] Cutting terminated, such as Figure 7 As shown, the area within the dashed box represents the discarded image blocks. Ultimately, image block ACB is cut into image blocks A, B, and C, with image blocks A and B being discarded and not included in the stitched image.

[0085] Remove image block CB, add image sub-block C, recalculate the threshold, and update the threshold to the minimum effective information density of the image blocks and image sub-blocks that were not discarded within the pixel rate limit of the mosaic.

[0086] Example 6

[0087] like Figure 2 As shown, A, C, and B are three image sub-blocks of image block ACB. The colors of the image sub-blocks, from light to dark, represent the lowest to highest effective density of the corresponding image sub-blocks.

[0088] In this example, the initial threshold is between the effective information densities of image sub-block A and image sub-block B.

[0089] The common boundary between image sub-block A and image sub-block C is a dividing line that runs through and is perpendicular to the boundary of the image block. The effective information density of image sub-block A is less than the threshold, and the effective information density of image sub-block C is greater than the threshold, thus satisfying the cutting condition. The common boundary between image sub-block C and image sub-block B is a dividing line that runs through and is perpendicular to the boundary of the image block, but the effective information density of both image sub-block C and image sub-block B is greater than the threshold, thus not satisfying the cutting condition.

[0090] Therefore, using the common boundary between image sub-block A and image sub-block C as the dividing line, image block ACB is cut into image block A and image block CB; since the effective information density of image block A is less than the threshold and there is no dividing line that satisfies the cutting conditions, it is discarded and not assembled into the stitched image.

[0091] Remove image block ACB, add image block CB, recalculate the threshold, and update the threshold to the minimum effective information density of the image blocks and image sub-blocks that were not discarded within the pixel rate limit of the mosaic.

[0092] In this example, the updated threshold is less than the threshold of image sub-block B.

[0093] For the image block CB generated after cutting, repeat the above process. The common boundary between image sub-block C and image sub-block B is a dividing line that runs through and is perpendicular to the boundary of the image block. However, the effective information density of image sub-block C and image sub-block B is greater than the threshold, and the cutting condition is not met.

[0094] Cutting terminated, such as Figure 8 As shown, the area within the dashed box represents the discarded image blocks. Ultimately, image block ACB is cut into image block A and image block CB, with image block A being discarded and not included in the stitched image.

[0095] Example 7

[0096] This invention provides an apparatus for processing image blocks in immersive media encoding.

[0097] like Figure 3 As shown, B, A, and C are three image sub-blocks of image block BAC. The colors of the image sub-blocks, from light to dark, represent the lowest to highest effective density of the corresponding image sub-blocks.

[0098] In this example, the initial threshold is between the effective information densities of image sub-block A and image sub-block B.

[0099] The module for calculating effective information density takes image sub-blocks A, B, and C as inputs and outputs the effective information density of each of them.

[0100] Threshold determination module: The input is the effective information density of each of image sub-block A, image sub-block B, and image sub-block C. The common boundary between image sub-block B and image sub-block A is a dividing line that runs through and is perpendicular to the boundary of the image block. The effective information density of image sub-block B is greater than the threshold, and the effective information density of image sub-block A is less than the threshold, thus satisfying the cutting condition. The common boundary between image sub-block A and image sub-block C is a dividing line that runs through and is perpendicular to the boundary of the image block. The effective information density of image sub-block A is less than the threshold, and the effective information density of image sub-block C is less than the threshold, thus satisfying the cutting condition.

[0101] The segmentation line decision module takes as input the common boundary between image sub-blocks B and A, the common boundary between image sub-blocks A and C, and the effective information density difference between the image sub-blocks on either side of A, and outputs the actual segmentation line. Since the effective information density difference between image sub-blocks A and C is larger, the common boundary between image sub-blocks A and C is used as the actual segmentation line.

[0102] The cutting module takes the common boundary of image sub-blocks A and C as input, makes the cutting decision, and outputs the cut image blocks. Image block BAC is cut into image block BA and image block C.

[0103] Assembly and stitching decision module: The input is that since image block BA contains image sub-block A with an effective information density less than the threshold, and image block BA has a dividing line that satisfies the cutting conditions, it is not discarded.

[0104] Threshold Update Module: Inputs are the effective information density and pixel rate limit of the undiscarded image blocks BA and C, and the output is the updated threshold. Remove image block BAC, add image blocks BA and C, recalculate the threshold, and update the threshold to the minimum effective information density of the undiscarded image blocks within the pixel rate limit of the stitched image.

[0105] In this example, the updated threshold lies between the effective information densities of image sub-block A and image sub-block B.

[0106] For the image block BA generated after cutting, repeat the above device process. The common boundary between image sub-block B and image sub-block A is a dividing line that runs through and is perpendicular to the boundary of the image block. The effective information density of image sub-block B is greater than the threshold, and the effective information density of image sub-block A is less than the threshold, thus satisfying the cutting condition.

[0107] Using the common boundary between image sub-block B and image sub-block A as the dividing line, image block BA is cut into image block B and image block A; since the effective information density of image block A is less than the threshold and there is no dividing line that satisfies the cutting conditions, it is discarded and not assembled into the stitched image.

[0108] Cutting terminated, such as Figure 9 As shown, the area within the dashed box represents the discarded image blocks. Ultimately, image block BAC is cut into image block B, image block A, and image block C, with image block A being discarded and not included in the stitched image.

[0109] Add image block B, recalculate the threshold, and update the threshold to the minimum effective information density of the image blocks and image sub-blocks that were not discarded within the pixel rate limit of the mosaic.

[0110] This device enables the segmentation of image sub-blocks at the granular level, accurately discarding image block A with low effective information density. In the process of multi-view image stitching, it can organize image blocks with higher effective information density under the limitation of limited pixel rate, improve the information utilization rate of the stitched image, and improve the rendering quality at the decoding end.

[0111] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method of processing image blocks in immersive media coding, the method comprising: The method comprises the following steps: For at least one image block in the multi-view image, the effective information density of the image sub-blocks on both sides of at least one split line which runs through and is perpendicular to the boundary of the image block is calculated respectively, the effective information being the additional information of the image sub-blocks in the multi-view image relative to other views; When the image sub-blocks on both sides of the split line satisfy the cutting condition, i.e., the effective information density of one side of the image sub-blocks is greater than a threshold value and the effective information density of the other side of the image sub-blocks is less than the threshold value, the image block is cut along the split line, and the image block is cut into two image blocks; For the image block containing the image sub-blocks with the effective information density less than the threshold value, if there is no split line satisfying the cutting condition, the image block is discarded and is not assembled into the mosaic image; Otherwise: No cutting is performed. For the image block containing the image sub-blocks with the effective information density less than the threshold value, if there is a split line satisfying the cutting condition, the image block is cut again.

2. The method of claim 1, wherein, For the image block having multiple split lines with the image sub-blocks on both sides satisfying the cutting condition, the split line with the maximum difference in the effective information density of the image sub-blocks on both sides is selected as the actual split line.

3. The method of claim 1, wherein, When the image sub-blocks on both sides of the split line satisfy the cutting condition, the threshold value is updated according to the pixel rate limit of the mosaic image and the effective information density of the non-discarded image block.

4. The method of claim 1, wherein, The method comprises the following modules:

5. An apparatus for processing a block of an image, characterized by An effective information density calculation module is configured to calculate the effective information density of the image sub-blocks on both sides of a split line which runs through and is perpendicular to the boundary of the image block in the multi-view image block, the effective information being the additional information of the image sub-blocks in the multi-view image relative to other views; Input: image block in the multi-view image; Output: effective information density of the image sub-blocks on both sides of the split line; A threshold value judgment module is configured to judge whether the effective information density of the image sub-blocks on both sides of the split line is greater than a threshold value and less than the threshold value, respectively, and if the condition is satisfied, the image block is cut, otherwise, the image block is not cut; Input: respective effective information density on both sides of the split line, threshold value; Output: cutting decision; A cutting module is configured to cut the image block along the split line if the cutting condition is satisfied; Input: split line, cutting decision; Output: cut image block; An assembly mosaic decision module is configured to discard the image block containing the image sub-blocks with the effective information density less than the threshold value and having no split line satisfying the cutting condition, and the image block is not assembled into the mosaic image; Input: image block; Output: assembly mosaic decision. For the image block containing the image sub-blocks with the effective information density less than the threshold value, if there is a split line satisfying the cutting condition, the image block is cut again.

6. The apparatus of claim 5, wherein, The method further comprises the following modules:

7. The apparatus of claim 5, wherein, A split line decision module is configured to select the split line with the maximum difference in the effective information density of the image sub-blocks on both sides as the actual split line for the image block having multiple split lines with the image sub-blocks on both sides satisfying the cutting condition; Input: multiple split lines satisfying the cutting condition, difference in effective information density; Output: actual split line. The method further comprises the following modules:

8. The apparatus of claim 5, wherein, ​ Threshold updating module: when the image sub-blocks on both sides of the split line satisfy the cutting condition, update the threshold according to the splicing image pixel rate limit and the effective information density of the non-discarded image block; Input: effective information density of the non-discarded image block, splicing image pixel rate limit; Output: updated threshold.

Images