Method and device for processing image blocks in immersive media coding

Abstract

The invention provides a method for processing an image block in immersive media coding, which comprises the following steps of: for at least one image block in a multi-view image, for at least one segmentation line penetrating through and vertical to the boundary of the image block in the image block, respectively calculating effective information densities of image sub-blocks at two sides of the segmentation line, the effective information is additional information of the image sub-block relative to other viewpoints in multiple viewpoints; when the image sub-blocks on the two sides of the segmentation line meet the segmentation condition, and the segmentation condition is that the effective information density of the image sub-block on one side is larger than a threshold value and the effective information density of the image sub-block on the other side is smaller than the threshold value; for the image blocks containing the image sub-blocks with the effective information density smaller than a threshold value, if the segmentation lines meeting the segmentation conditions do not exist, discarding the image blocks, and not assembling the image blocks into the spliced image; otherwise, cutting is not carried out.

Description

Technical Field

[0001] The present invention relates to the field of immersive media coding, and in particular to a method and device for processing multi-viewpoint image blocks. Background Art

[0002] "Immersion" as a subjective evaluation refers to the viewer's sense of involvement in the virtual scene established and displayed by the multimedia system. As the capabilities of acquisition and display devices are enhanced year by year, immersive media, as a visual multimedia that can bring viewers a strong sense of immersion, has become a research hotspot in the industry and science community in terms of encoding, transmission and rendering. Multi-viewpoint images plus depth information is an effective immersive media expression method, which consists of texture images from multiple viewpoints and depth images corresponding to each texture image. Using viewpoint synthesis technology based on depth images, this expression method can render the image of the target viewpoint based on the camera parameters of the target image and the positional relationship between the target viewpoint and the existing viewpoints.

[0003] Since there is generally a large amount of information redundancy between multiple viewpoints, encoding and transmitting multi-viewpoint sub-block stitching images can significantly save pixel rate. Before encoding and decoding, by analyzing the geometric texture relationship between multiple viewpoints, the main viewpoint (some images in the multi-viewpoint image that contain complete viewpoint information) image is used to remove redundant pixels from other viewpoints as much as possible, so that only unique valid information is retained in viewpoint images other than the main viewpoint. Considering the coding efficiency, the retained valid pixels are aggregated into rectangular sub-blocks, and finally multiple image blocks are assembled into a stitching image for encoding and transmission. At the decoding end, the decoded image block information is used to extract all block images from the decoded multi-viewpoint block stitching image, and finally the target image is synthesized in units of image blocks.

[0004] However, as the field of view of video content becomes larger and the number of source viewpoints increases, the coding and transmission scheme based on spliced ​​images is exposed to be insufficient in processing large amounts of data input. In many industrial scenarios, the pixel rate of spliced ​​images is limited, and the necessary sub-block images left after redundancy removal between viewpoints cannot be fully placed in a limited-size atlas. The loss of sub-block images leads to information loss during decoding, making it impossible to synthesize high-quality videos. Therefore, in order to measure the importance of image blocks and make decisions, the concept of effective information volume of pixels is introduced, which measures the degree of non-overlap between the current pixel and the pixels in other viewpoints. The higher the effective information volume, the lower the degree of representation by pixels in other viewpoints, the greater the unique information, and the more it should be retained in the spliced ​​image.

[0005] More specifically, effective information usually includes two channels: geometric information and texture information. The values ​​of each channel are independent of each other and are calculated by the logarithm of the difference in disparity and brightness between the current pixel and the reference pixel from other viewpoints at the current pixel position. If the current pixel has no reference pixel, the amount of information is the maximum value, which is determined by the bit width of disparity and brightness. The specific calculation method is as follows: Let the disparity difference between the current pixel and the reference pixel be d g , the brightness difference is d t , then the effective information ρ is:

[0006] ρ = log d g +logd t

[0007] The geometric width of the video g and brightness bit width b t , then the maximum value of effective information ρ max for:

[0008]

[0009] Since the retained pixels are aggregated into image sub-blocks, the effective information of an image sub-block is calculated by accumulating the effective information values ​​of all the valid pixels therein.

[0010] ρ block =∑ pixel∈block ρ pixel

[0011] The concept of effective information density μ is further proposed, which is the effective information of the sub-block divided by the number of effective pixels.

[0012]

[0013] In the process of sub-block image stitching, based on the effective information density μ of the sub-block, the image sub-block with large effective information density is retained preferentially under the restriction of the priority pixel rate, so as to improve the information utilization rate and the rendering quality at the decoding end.

[0014] However, the distribution of effective information in an image sub-block may be uneven. Selecting the image sub-blocks to be retained first based on the effective information density μ may discard local areas with high effective information density in some sub-blocks, or retain local areas with low effective information density in the retained image sub-blocks, wasting pixel rate. Therefore, a method for cutting image sub-blocks based on effective information distribution is needed, so that the effective information distribution in the same sub-block after cutting is as uniform as possible, and the information rate can be improved when organizing the splicing image, thereby improving the coding efficiency. However, how to make the cutting more complete during the cutting process, how to reduce invalid cutting, and not cutting when the effective information density of two sub-blocks is greater than the retention threshold, so as to prevent the continuity of the image area from being destroyed and thus affecting the coding efficiency, are all difficult problems and challenges. Summary of the invention

[0015] In order to solve the above technical problem, the present invention proposes a method and device for processing image blocks in immersive media coding.

[0016] For at least one image block in a multi-view image, for at least one dividing line in the image block that runs through and is perpendicular to the boundary of the image block, the effective information density of the image sub-blocks on both sides of the dividing line is calculated respectively, and the effective information is the additional information of the image sub-block in the multi-view relative to other viewpoints.

[0017] When the image sub-blocks on both sides of the dividing line meet the cutting condition, the cutting condition is that the effective information density of the image sub-block on one side is greater than the threshold and the effective information density of the image sub-block on the other side is less than the threshold, cutting is performed along the dividing line to cut the image block into two image blocks. For an image block containing an image sub-block with an effective information density less than the threshold, if there is no dividing line that meets the cutting condition, the image block is discarded and is not assembled into the spliced ​​image; otherwise, no cutting is performed.

[0018] Furthermore, for the image block including the image sub-block whose effective information density is less than the threshold after cutting, if there is a segmentation line satisfying the cutting condition, it is cut again.

[0019] Furthermore, for an image block having multiple segmentation lines whose two side image sub-blocks satisfy the cutting condition, a segmentation line having the largest difference in effective information density of the two side image sub-blocks of the segmentation line is selected as the actual segmentation line.

[0020] Furthermore, when the image sub-blocks on both sides of the segmentation line meet the segmentation condition, the threshold is updated according to the pixel rate limit of the mosaic image and the effective information density of the image blocks that are not discarded.

[0021] Furthermore, the present invention proposes a device for processing an image block, comprising the following modules:

[0022] Calculate effective information density module: used to calculate the effective information density of image sub-blocks on both sides of the dividing line along the dividing line that runs through and is perpendicular to the boundary of the image block in the multi-view image block. The effective information is the additional information of the image sub-block relative to other viewpoints in the multi-view.

[0023] Threshold judgment module: used to judge whether the effective information density of the image sub-blocks on both sides of the segmentation line is greater than the threshold and less than the threshold. If the condition is met, the image sub-blocks are cut; if the condition is not met, the image sub-blocks are not cut.

[0024] Cutting line decision module: for image blocks with multiple cutting lines whose two side image sub-blocks meet the cutting conditions, the cutting line with the largest difference in effective information density of the two side image sub-blocks is selected as the actual cutting line

[0025] Cutting module: When the cutting conditions are met, cutting is performed along the dividing line.

[0026] Assembly splicing decision module: used to discard image sub-blocks containing effective information density less than a threshold and image blocks without a segmentation line that meets the cutting condition, and not assemble them into the splicing image.

[0027] Threshold updating module: when the image sub-blocks on both sides of the segmentation line meet the cutting condition, the threshold is updated according to the pixel rate limit of the mosaic image and the effective information density of the image blocks that are not discarded.

[0028] Furthermore, the image block cutting device cuts the image block containing the image sub-block whose effective information density is less than the threshold value again if there is a segmentation line satisfying the cutting condition.

[0029] The advantages of the present invention are that, by judging the granularity of image sub-blocks, it is possible to fully cut image blocks and discard image blocks more accurately. In the process of multi-view image splicing, image blocks with higher effective information density can be organized under the limitation of limited pixel rate, thereby improving the information utilization rate of the spliced ​​image and the rendering quality at the decoding end. At the same time, the occurrence of cutting is controlled by thresholds, and invalid cutting is reduced in scenarios where no discarding is required, thereby saving encoding bit rate. After each cutting, the threshold is updated according to the limitation of pixel rate and the effective information density of the non-discarded image sub-blocks, so that the threshold is more accurate and the overall encoding efficiency is increased. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] The principles of the present invention are explained through the following embodiments in conjunction with the accompanying drawings.

[0031] The drawings described herein are used to provide further understanding of the present invention and constitute a part of this application. The embodiments of the present invention are only used to explain the present invention and are only some special cases therein. The scope of application of the present invention is not limited to these embodiments.

[0032] Figure 1 Schematic diagram of three image sub-blocks A, B, and C of image block ABC in one embodiment of the present invention.

[0033] Figure 2 Schematic diagram of three image sub-blocks A, B, and C of image block ABC in another embodiment of the present invention.

[0034] Figure 3 Schematic diagram of three image sub-blocks A, B, and C of image block ABC in another embodiment of the present invention.

[0035] Figure 4 It is a schematic diagram of the cutting result in one embodiment of the present invention, wherein the dashed box represents the discarded image blocks.

[0036] Figure 5 It is a schematic diagram of the cutting result in another embodiment of the present invention, wherein the dashed box represents the discarded image blocks.

[0037] Figure 6 It is a schematic diagram of the cutting result in another embodiment of the present invention, wherein the dashed box represents the discarded image blocks.

[0038] Figure 7 It is a schematic diagram of the cutting result in another embodiment of the present invention, wherein the dashed box represents the discarded image blocks.

[0039] Figure 8 It is a schematic diagram of the cutting result in another embodiment of the present invention, wherein the dashed box represents the discarded image blocks.

[0040] Fig. 9 It is a schematic diagram of the cutting result in another embodiment of the present invention, wherein the dashed box represents the discarded image blocks. DETAILED DESCRIPTION

[0041] Example 1

[0042] like Figure 1 As shown, A, B, and C are three image sub-blocks of image block ABC, and the colors of the image sub-blocks from light to dark represent the lowest to highest effective density of the corresponding image sub-blocks.

[0043] In this example, the initial threshold is located between the effective information densities of image sub-block A and image sub-block B.

[0044] The common boundary of image sub-block A and image sub-block B is a segmentation line that runs through and is perpendicular to the boundary of the image block, and the effective information density of image sub-block A is less than the threshold, and the effective information density of image sub-block B is greater than the threshold, and the cutting condition is met; the common boundary of image sub-block B and image sub-block C is a segmentation line that runs through and is perpendicular to the boundary of the image block, but the effective information density of image sub-block B and image sub-block C is greater than the threshold, and the cutting condition is not met;

[0045] Therefore, the common boundary between image sub-block A and image sub-block B is used as the dividing line, and image block ABC is cut into image block A and image block BC. Since the effective information density of image block A is less than the threshold and there is no dividing line that satisfies the cutting condition, it is discarded and not assembled into the spliced ​​image.

[0046] Remove image block ABC, add image block BC, recalculate the threshold, and update the threshold to the minimum value of the effective information density of the image blocks and image sub-blocks that are not discarded within the pixel rate limit of the mosaic image.

[0047] In this example, the updated threshold is less than the effective information density of the image sub-block B.

[0048] For the image block BC generated after cutting, the above process is repeated. The common boundary of the image sub-block B and the image sub-block C is a segmentation line that runs through and is perpendicular to the boundary of the image block. However, the effective information density of the image sub-block B and the image sub-block C is greater than the threshold value, and the cutting condition is not met.

[0049] Cutting termination, such as Figure 4 As shown, the dashed box is a discarded image block, and finally the image block ABC is cut into image block A and image block BC, wherein image block A is discarded and does not enter the spliced ​​image.

[0050] Example 2

[0051] like Figure 1 As shown, A, B, and C are three image sub-blocks of image block ABC, and the colors of the image sub-blocks from light to dark represent the lowest to highest effective density of the corresponding image sub-blocks.

[0052] In this example, the initial threshold is located between the effective information densities of image sub-block B and image sub-block C.

[0053] The common boundary of image sub-block A and image sub-block B is a segmentation line that runs through and is perpendicular to the boundary of the image block, but the effective information density of image sub-block A and image sub-block B is less than the threshold, and the cutting condition is not met; the common boundary of image sub-block B and image sub-block C is a segmentation line that runs through and is perpendicular to the boundary of the image block, and the effective information density of image sub-block B is less than the threshold, and the effective information density of image sub-block C is greater than the threshold, and the cutting condition is met;

[0054] Therefore, the common boundary of image sub-block B and image sub-block C is used as the dividing line, and image block ABC is cut into image block AB and image block C. Since image block AB contains image sub-block A whose effective information density is less than the threshold, but the effective information densities of image sub-blocks A and B are both less than the threshold, there is no dividing line that meets the cutting condition, so it is discarded and not assembled into the spliced ​​image.

[0055] Cutting termination, such as Figure 5 As shown, the dashed box is a discarded image block, and finally the image block ABC is cut into image block AB and image block C, wherein the image block AB is discarded and does not enter the spliced ​​image.

[0056] Remove image block ABC, add image block C, recalculate the threshold, and update the threshold to the minimum value of the effective information density of the image blocks and image sub-blocks that are not discarded within the pixel rate limit of the mosaic image.

[0057] Example 3

[0058] like Figure 1 As shown, A, B, and C are three image sub-blocks of image block ABC, and the colors of the image sub-blocks from light to dark represent the lowest to highest effective density of the corresponding image sub-blocks.

[0059] In this example, the initial threshold is less than the effective information density of image sub-block A

[0060] The common boundary of image sub-block A and image sub-block B is a segmentation line that runs through and is perpendicular to the boundary of the image block, but the effective information density of image sub-block A and image sub-block B is greater than the threshold, and the cutting condition is not met; the common boundary of image sub-block B and image sub-block C is a segmentation line that runs through and is perpendicular to the boundary of the image block, but the effective information density of image sub-block B and image sub-block C is greater than the threshold, and the cutting condition is not met;

[0061] Therefore, no cutting is performed. Cutting is terminated and the threshold is not updated.

[0062] Through this method, the invalid segmentation of the image block ABC can be reduced, and the encoding bit rate can be saved;

[0063] Example 4

[0064] like Figure 1 As shown, A, B, and C are three image sub-blocks of image block ABC, and the colors of the image sub-blocks from light to dark represent the lowest to highest effective density of the corresponding image sub-blocks.

[0065] In this example, the initial threshold is located between the effective information densities of image sub-block A and image sub-block B.

[0066] The common boundary of image sub-block A and image sub-block B is a dividing line that runs through and is perpendicular to the boundary of the image block, and the effective information density of image sub-block A is less than the threshold, and the effective information density of image sub-block B is greater than the threshold, and the cutting condition is met; the common boundary of image sub-block B and image sub-block C is a dividing line that runs through and is perpendicular to the boundary of the image block, but the effective information densities of image sub-block B and image sub-block C are both greater than the threshold, and the cutting condition is not met.

[0067] Therefore, the common boundary between image sub-block A and image sub-block B is used as the dividing line, and image block ABC is cut into image block A and image block BC. Since the effective information density of image block A is less than the threshold and there is no dividing line that satisfies the cutting condition, it is discarded and not assembled into the spliced ​​image.

[0068] Remove image block ABC, add image block BC, recalculate the threshold, and update the threshold to the minimum value of the effective information density of the image blocks and image sub-blocks that are not discarded within the pixel rate limit of the mosaic image.

[0069] In this example, the updated threshold is located between the effective information densities of image sub-block B and image sub-block C.

[0070] For the image block BC generated after cutting, the above process is repeated, the common boundary of the image sub-block B and the image sub-block C is a segmentation line that runs through and is perpendicular to the boundary of the image blocks, and the effective information density of the image sub-block B is less than the threshold, and the effective information density of the image sub-block C is greater than the threshold, and the cutting condition is met;

[0071] Therefore, the common boundary between image sub-block B and image sub-block C is used as the dividing line, and image block BC is cut into image block B and image block C. Since the effective information density of image block B is less than the threshold and there is no dividing line that satisfies the cutting condition, it is discarded and not assembled into the spliced ​​image.

[0072] Cutting termination, such as Figure 6 As shown, the dashed box is a discarded image block, and finally the image block ABC is cut into image block A, image block B and image block C, wherein image block A and image block B are discarded and do not enter the spliced ​​image.

[0073] Remove image block BC, add image block C, recalculate the threshold, and update the threshold to the minimum value of the effective information density of the image blocks and image sub-blocks that are not discarded within the pixel rate limit of the mosaic image.

[0074] By using this method, after each cutting, the threshold is updated according to the pixel rate limit and the effective information density of the non-discarded image sub-blocks, so that the threshold is more accurate, the cutting is more sufficient, and the overall coding efficiency is increased.

[0075] Example 5

[0076] like Figure 2 As shown, A, C, and B are three image sub-blocks of the image block ACB, and the colors of the image sub-blocks from light to dark represent the lowest to highest effective density of the corresponding image sub-blocks.

[0077] In this example, the initial threshold is located between the effective information densities of image sub-block B and image sub-block C.

[0078] The common boundary of image sub-block A and image sub-block C is a segmentation line that runs through and is perpendicular to the boundary of the image block, and the effective information density of image sub-block A is less than the threshold, and the effective information density of image sub-block C is greater than the threshold, and the cutting condition is met; the common boundary of image sub-block B and image sub-block C is a segmentation line that runs through and is perpendicular to the boundary of the image block, and the effective information density of image sub-block B is less than the threshold, and the effective information density of image sub-block C is greater than the threshold, and the cutting condition is met;

[0079] Since the difference in effective information density between image sub-block A and image sub-block C is larger, the common boundary of image sub-block A and image sub-block C is used as the actual dividing line, and image block ACB is cut into image block A and image block CB; since the effective information density of image block A is less than the threshold and there is no dividing line that meets the cutting condition, it is discarded and not assembled into the spliced ​​image.

[0080] Remove the image block ACB, add the image block CB, recalculate the threshold, and update the threshold to the minimum value of the effective information density of the image blocks and image sub-blocks that are not discarded within the pixel rate limit of the mosaic image.

[0081] In this example, the updated threshold is located between the effective information densities of the image sub-block C and the image sub-block B.

[0082] For the image block CB generated after cutting, the above process is repeated, the common boundary of the image sub-block C and the image sub-block B is a segmentation line that runs through and is perpendicular to the boundary of the image blocks, and the effective information density of the image sub-block B is less than the threshold, and the effective information density of the image sub-block C is greater than the threshold, and the cutting condition is met;

[0083] Therefore, the common boundary between image sub-block C and image sub-block B is used as the dividing line, and image block CB is cut into image block C and image block B. Since the effective information density of image block B is less than the threshold and there is no dividing line that satisfies the cutting condition, it is discarded and not assembled into the spliced ​​image.

[0084] Cutting termination, such as Figure 7 As shown, the dashed box is a discarded image block, and the final image block ACB is cut into image block A, image block B and image block C, wherein image block A and image block B are discarded and do not enter the splicing image.

[0085] Remove the image block CB, add the image sub-block C, recalculate the threshold, and update the threshold to the minimum value of the effective information density of the image blocks and image sub-blocks that are not discarded within the pixel rate limit of the mosaic image.

[0086] Example 6

[0087] like Figure 2 As shown, A, C, and B are three image sub-blocks of the image block ACB, and the colors of the image sub-blocks from light to dark represent the lowest to highest effective density of the corresponding image sub-blocks.

[0088] In this example, the initial threshold is located between the effective information densities of image sub-block A and image sub-block B.

[0089] The common boundary of image sub-block A and image sub-block C is a dividing line that runs through and is perpendicular to the boundary of the image block, and the effective information density of image sub-block A is less than the threshold, and the effective information density of image sub-block C is greater than the threshold, and the cutting condition is met; the common boundary of image sub-block C and image sub-block B is a dividing line that runs through and is perpendicular to the boundary of the image block, but the effective information densities of image sub-block C and image sub-block B are both greater than the threshold, and the cutting condition is not met.

[0090] Therefore, the common boundary between image sub-block A and image sub-block C is used as the dividing line, and image block ACB is cut into image block A and image block CB; since the effective information density of image block A is less than the threshold and there is no dividing line that satisfies the cutting condition, it is discarded and not assembled into the spliced ​​image.

[0091] Remove the image block ACB, add the image block CB, recalculate the threshold, and update the threshold to the minimum value of the effective information density of the image blocks and image sub-blocks that are not discarded within the pixel rate limit of the mosaic image.

[0092] In this example, the updated threshold is located below the threshold of the image sub-block B.

[0093] For the image block CB generated after cutting, the above process is repeated. The common boundary of the image sub-block C and the image sub-block B is a segmentation line that runs through and is perpendicular to the boundary of the image block. However, the effective information density of the image sub-block C and the image sub-block B is greater than the threshold value, and the cutting condition is not met.

[0094] Cutting termination, such as Figure 8 As shown, the dashed box is a discarded image block, and finally the image block ACB is cut into image block A and image block CB, wherein image block A is discarded and does not enter the spliced ​​image.

[0095] Example 7

[0096] An embodiment of the present invention provides a device for processing image blocks in immersive media coding

[0097] like Figure 3 As shown, B, A, and C are three image sub-blocks of the image block BAC, and the colors of the image sub-blocks from light to dark represent the lowest to highest effective density of the corresponding image sub-blocks.

[0098] In this example, the initial threshold is located between the effective information densities of image sub-block A and image sub-block B.

[0099] Calculate the effective information density module: the input is image sub-block A, image sub-block B and image sub-block C, and the output is the effective information density of image sub-block A, image sub-block B and image sub-block C respectively.

[0100] Threshold judgment module: input is the effective information density of each of the image sub-block A, image sub-block B and image sub-block C, the common boundary of the image sub-block B and the image sub-block A is a segmentation line that runs through and is perpendicular to the boundary of the image block, and the effective information density of the image sub-block B is greater than the threshold, and the effective information density of the image sub-block A is less than the threshold, and the cutting condition is met; the common boundary of the image sub-block A and the image sub-block C is a segmentation line that runs through and is perpendicular to the boundary of the image block, and the effective information density of the image sub-block A is less than the threshold, and the effective information density of the image sub-block C is less than the threshold, and the cutting condition is met;

[0101] Segmentation line decision module: The input is the common boundary between image sub-block B and image sub-block A, the common boundary between image sub-block A and image sub-block C, and the difference in effective information density between the image sub-blocks on both sides, and the output is the actual segmentation line. Since the difference in effective information density between image sub-block A and image sub-block C is larger, the common boundary between image sub-block A and image sub-block C is used as the actual segmentation line.

[0102] Cutting module: Input is the common boundary of the segmentation line image sub-block A and image sub-block C, cutting decision, and output is the cut image block. Image block BAC is cut into image block BA and image block C;

[0103] Assembly splicing decision module: The input is that since the image block BA contains an image sub-block A whose effective information density is less than a threshold, and the image block BA has a segmentation line that satisfies the cutting condition, it is not discarded.

[0104] Threshold update module: The input is the effective information density and pixel rate limit of the non-discarded image blocks BA and C, and the output is the updated threshold. Remove the image block BAC, add the image block BA and the image block C, recalculate the threshold, and update the threshold to the minimum value of the effective information density of the non-discarded image blocks within the pixel rate limit of the mosaic.

[0105] In this example, the updated threshold is between the effective information densities of image sub-block A and image sub-block B.

[0106] For the image block BA generated after cutting, the above device process is repeated, the common boundary of the image sub-block B and the image sub-block A is a segmentation line that runs through and is perpendicular to the boundary of the image blocks, and the effective information density of the image sub-block B is greater than the threshold, and the effective information density of the image sub-block A is less than the threshold, and the cutting condition is met;

[0107] Taking the common boundary between image sub-block B and image sub-block A as the dividing line, image block BA is cut into image block B and image block A. Since the effective information density of image block A is less than the threshold and there is no dividing line that satisfies the cutting condition, it is discarded and not assembled into the spliced ​​image.

[0108] Cutting termination, such as Fig. 9 As shown, the dashed box is a discarded image block, and the final image block BAC is cut into image block B, image block A and image block C, wherein image block A is discarded and does not enter the spliced ​​image.

[0109] Image block B is added, the threshold is recalculated, and the updated threshold is the minimum value of the effective information density of the image blocks and image sub-blocks that are not discarded within the pixel rate limit of the mosaic image.

[0110] Through this device, it is possible to segment the image into sub-blocks, accurately discard the image block A with low effective information density, and organize image blocks with higher effective information density under the limitation of limited pixel rate during the multi-view image stitching process, thereby improving the information utilization rate of the stitching image and the rendering quality at the decoding end.

[0111] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for processing image blocks in immersive media coding, characterized in that: include: For at least one image block in the multi-view image, for at least one dividing line in the image block that runs through and is perpendicular to a boundary of the image block, respectively calculate effective information density of image sub-blocks on both sides of the dividing line, where the effective information is additional information of the image sub-block relative to other viewpoints in the multi-view; When the image sub-blocks on both sides of the dividing line meet the cutting condition, the cutting condition is that the effective information density of the image sub-block on one side is greater than the threshold and the effective information density of the image sub-block on the other side is less than the threshold: Cutting is performed along the dividing line to cut the image block into two image blocks, For an image block containing an image sub-block whose effective information density is less than a threshold, if there is no segmentation line satisfying the cutting condition, the image block is discarded and is not assembled into the spliced ​​image; otherwise: No cutting is done.

2. The method according to claim 1, characterized in that For an image block including an image sub-block whose effective information density is less than a threshold value, if there is a segmentation line satisfying the segmentation condition, it is segmented again.

3. The method according to claim 1, characterized in that For an image block having multiple segmentation lines whose two side image sub-blocks satisfy the cutting condition, the segmentation line with the largest difference in effective information density of the two side image sub-blocks of the segmentation line is selected as the actual segmentation line.

4. The method according to claim 1, characterized in that: When the image sub-blocks on both sides of the segmentation line meet the cutting condition, the threshold is updated according to the pixel rate limit of the mosaic image and the effective information density of the image blocks that are not discarded.

5. A device for processing an image block, characterized in that: Includes the following modules: An effective information density calculation module is used to calculate the effective information density of the image sub-blocks on both sides of the dividing line along the dividing line that runs through and is perpendicular to the boundary of the image block in the multi-view image block, where the effective information is the additional information of the image sub-block relative to other viewpoints in the multi-view; Input: image blocks in multi-view images; Output: effective information density of image sub-blocks on both sides of the segmentation line; Threshold judgment module: used to judge whether the effective information density of the image sub-blocks on both sides of the segmentation line is greater than the threshold and less than the threshold, and if the condition is met, the image sub-blocks are cut, otherwise the image sub-blocks are not cut; Input: effective information density on both sides of the dividing line, threshold; Output: cutting decision; Cutting module: when the cutting conditions are met, cutting is performed along the cutting line; Input: split line, cutting decision; Output: the cut image block; An assembly and splicing decision module is used to discard image sub-blocks containing effective information density less than a threshold and image blocks without a segmentation line satisfying the cutting condition, and not assemble them into the splicing image; Input: image patch; Output: Assembly splicing decision.

6. The device according to claim 5, characterized in that For an image block including an image sub-block whose effective information density is less than a threshold value, if there is a segmentation line satisfying the segmentation condition, it is segmented again.

7. The device according to claim 5, characterized in that Also includes the following modules: A segmentation line decision module: for an image block having multiple segmentation lines whose two side image sub-blocks satisfy the cutting condition, a segmentation line having the largest difference in effective information density of the two side image sub-blocks of the segmentation line is selected as the actual segmentation line; Input: multiple segmentation lines that meet the cutting conditions, and effective information density difference; Output: actual dividing line.

8. The device according to claim 5, characterized in that Also includes the following modules: Threshold updating module: when the image sub-blocks on both sides of the segmentation line meet the cutting condition, the threshold is updated according to the pixel rate limit of the mosaic image and the effective information density of the image blocks that are not discarded; Input: effective information density of the image blocks that are not discarded, pixel rate limit of the mosaic image; Output: Updated threshold.

Images