Image decoding device

Abstract

To provide an image decoding device capable of suppressing discomfort of a decoding image even if both of an attention region and a non-attention region are compressed at a different compression ratio.SOLUTION: A low-compression image decoding part 3 decodes an image of an attention region and a circumference region of the attention region. A high-compression image decoding part 4 decodes an image of a non-attention region. An attention region circumference image composition part 6 composes a first decoding image in the circumference region of the attention region decoded by the low-compression image decoding part 3 with a second decoding image of the circumference region of the attention region decoded from the decoding image of the non-attention region decoded by the high-compression image decoding part 4 to generate a composition decoding image in the circumference region of the attention region. An image composition part 7 composes a third decoding image of the attention region decoded by the low-compression image decoding part 3, a composition decoding image generated by an attention region circumference image composition part 6, and a fourth decoding image excluding the decoding image of the circumference region of the attention region from the decoding image of the non-attention region decoded by the high-compression image decoding part 4, and the composition decoding image of each frame is generated.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an image decoding device. [Background technology]

[0002] Patent Document 1 describes a technique in which a region of interest within a frame is compressed at a low compression rate, and a non-region of interest other than the region of interest is compressed at a high compression rate. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2001-36901 A Summary of the Invention [Problem to be solved by the invention]

[0004] When the encoded image data described in Patent Document 1 is decoded, the difference in image quality between the region of interest and the region of non-interest may be conspicuous and may cause an unnatural feeling. It is not preferable that an unnatural difference in image quality exists within a frame.

[0005] An object of the present invention is to provide an image decoding device that can suppress the sense of incongruity in a decoded image even if a region of interest and a region of non-interest are compressed at different compression rates. [Means for solving the problem]

[0006] The present invention relates to a low-compression image decoding unit that decodes images of the region of interest and the surrounding region of the region of interest in encoded image data, the region of interest and a surrounding region of the region of interest that is adjacent to the region of interest and surrounds the region of interest, the low-compression image decoding unit decoding images of the region of interest and the surrounding region of the region of interest in encoded image data, the high-compression image decoding unit decoding images of the region of non-interest in the encoded image data, and a first decoded image of the surrounding region of the region of interest decoded by the low-compression image decoding unit and a second decoded image of the region of interest decoded by the high-compression image decoding unit. and an image synthesis unit that synthesizes a third decoded image of the area of ​​interest decoded by the low-compression image decoding unit, the synthetic decoded image generated by the image synthesis unit, and a fourth decoded image excluding the decoded image of the area surrounding the area of ​​interest among the decoded images of the area of ​​non-attention decoded by the high-compression image decoding unit, to generate a synthetic decoded image of the area surrounding the area of ​​interest. Effect of the Invention

[0007] According to the image decoding device of the present invention, it is possible to suppress the sense of incongruity in the decoded image even if the region of interest and the region of non-interest are compressed at different compression rates. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a block diagram showing an image decoding device according to an embodiment. [Diagram 2] FIG. 2 is a diagram showing an example of a frame of encoded image data input to an image decoding device according to an embodiment. [Figure 3A] FIG. 3A is a diagram showing the attention area and the surrounding areas of the attention area of ​​the frame shown in FIG. [Figure 3B] FIG. 3B is a diagram showing a non-attention region of the frame shown in FIG. [Figure 3C]FIG. 3C is a diagram showing the surrounding areas of the attention area in the frame shown in FIG. [Figure 4] FIG. 4 is a characteristics diagram showing a first example of a method of synthesizing two decoded images by the attention area surrounding image synthesis unit included in the image decoding device according to one embodiment. [Diagram 5] FIG. 5 is a characteristic diagram showing a second example of a method for synthesizing two decoded images by the attention area surrounding image synthesis unit included in the image decoding device according to one embodiment. [Figure 6] FIG. 6 is a characteristic diagram showing a third example of a method for synthesizing two decoded images by the attention area surrounding image synthesis unit included in the image decoding device according to an embodiment. [Figure 7A] FIG. 7A is a characteristic diagram showing a state in which the mixing ratio of the decoded images by the low-compression image decoding unit is reduced in the third example shown in FIG. [Figure 7B] FIG. 7B is a characteristic diagram showing a state in which the mixing ratio of the decoded images by the low-compression image decoding unit is increased in the third example shown in FIG. [Figure 8] FIG. 8 is a characteristic diagram showing a fourth example of a method for synthesizing two decoded images by the attention area surrounding image synthesis unit included in the image decoding device according to an embodiment. [Figure 9] FIG. 9 is a characteristic diagram showing a fifth example of a method for synthesizing two decoded images by the attention area surrounding image synthesis unit included in the image decoding device according to one embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] An image decoding device according to an embodiment will be described below with reference to the accompanying drawings. Encoded image data is input to an image decoding device 100 according to an embodiment shown in Fig. 1. The encoded image data may be image data encoded by MPEG2 or image data encoded by H.264 / MPEG-4 AVC. The encoding method for the encoded image data is arbitrary. Using frame F of image data shown in Fig. 2 as an example, how the image of each frame is encoded will be described.

[0010] As an example, image data called 4K, which is 3840 pixels horizontally and 2160 pixels vertically, is encoded to produce encoded image data. As shown in Fig. 2, a frame F of image data has an area surrounded by pixel addresses Pfa, Pfb, Pfc, and Pfd at the upper left end, upper right end, lower left end, and lower right end. An area surrounded by pixel addresses P1a, P1b, P1c, and P1d, for example, at the center in the frame F, is set as an attention area Fr1. The attention area Fr1 is a part of the frame F. An area of ​​the frame F other than the attention area Fr1 is set as a non-attention area Fr2.

[0011] Of the area surrounded by pixel addresses P3a, P3b, P3c, and P3d, the area excluding the attention area Fr1 is set as the attention area surrounding area Fr3. The attention area surrounding area Fr3 is an area adjacent to the attention area Fr1 and surrounding the attention area Fr1. The attention area Fr1 is an area equivalent to, for example, 1920 pixels horizontally and 1080 vertically, which is called full HD. However, the size of the attention area Fr1 is not limited to an area equivalent to full HD.

[0012] Of the image data having a frame structure as shown in Fig. 2, the region of interest Fr1 and the region surrounding the region of interest Fr3 shown in Fig. 3A are encoded at a first compression rate. Of the image data having a frame structure as shown in Fig. 2, the region of non-interest Fr2 shown in Fig. 3B is encoded at a second compression rate higher than the first compression rate. The region in the center of the non-interest region Fr2 that corresponds to the region of interest Fr1 shown in black is treated as no-signal data.

[0013] The region surrounding the region of interest Fr3 shown in FIG. 3C is encoded at a first compression rate with high image quality in FIG. 3A, and at a second compression rate with low image quality in FIG. 3B.

[0014] First encoded data in which the region of interest Fr1 and the region surrounding the region of interest Fr3 are encoded at a first compression rate, and second encoded data in which the region of non-interest Fr2 is encoded at a second compression rate are associated with each other to form encoded image data input to the image decoding device 100. For example, a common frame number is assigned to the frames of the first encoded data and the second encoded data, and the two frames are associated with each other.

[0015] In Fig. 1, the communication unit 1 receives the encoded image data as described above. The communication unit 1 receives the encoded image data via a network such as the Internet. A receiving unit that receives broadcast waves may be provided instead of the communication unit 1. The storage unit 2 temporarily stores the encoded image data received by the communication unit 1. Of the first and second encoded data in the encoded image data stored in the storage unit 2, the first encoded data is supplied to a low-compression image decoding unit 3, and the second encoded data is supplied to a high-compression image decoding unit 4.

[0016] The low-compression image decoding unit 3 decodes the images of the attention region Fr1 and the attention region surrounding region Fr3 by decoding the first encoded data. The high-compression image decoding unit 4 decodes the image of the non-attention region Fr2 by decoding the second encoded data. The image of the attention region surrounding region Fr3 decoded by the low-compression image decoding unit 3 is supplied to an attention region surrounding image determination unit 5 and an attention region surrounding image synthesis unit 6. The image of the attention region Fr1 decoded by the low-compression image decoding unit 3 is supplied to an image synthesis unit 7. Of the images of the non-attention region Fr2 decoded by the high-compression image decoding unit 4, the image of the attention region surrounding region Fr3 is supplied to the attention region surrounding image synthesis unit 6, and the image of the region excluding the attention region surrounding region Fr3 is supplied to the image synthesis unit 7.

[0017] The attention region surrounding image determination unit 5 detects the magnitude of the motion vector of the image based on the input image of the attention region surrounding area Fr3, and detects the complexity of the image based on the spatial frequency of the image. The attention region surrounding image synthesis unit 6 synthesizes a first decoded image of the attention region surrounding area Fr3 decoded by the low compression image decoding unit 3 and a second decoded image of the attention region surrounding area Fr3 among the decoded images of the non-attention region Fr2 decoded by the high compression image decoding unit 4, to generate a synthesized decoded image of the attention region surrounding area Fr3.

[0018] The attention area surrounding image synthesis unit 6 may vary the synthesis method of the first decoded image and the second decoded image depending on the size of the motion vector or the complexity of the image detected by the attention area surrounding image determination unit 5. Although it is not essential to provide the attention area surrounding image determination unit 5, it is preferable to provide it.

[0019] An example of a method for synthesizing the first decoded image and the second decoded image by the attention area surrounding image synthesizing unit 6 will be described with reference to Figures 4 to 9. In Figures 4 to 9, the solid lines indicate the mixing ratios of the first decoded image, and the dashed lines indicate the mixing ratios of the second decoded image. The region Fr20 shown in Figures 4 to 9 indicates the region excluding the attention area surrounding region Fr3 in the non-attention area Fr2.

[0020] As shown in Fig. 4, as a first example, the attention area surrounding image synthesis unit 6 synthesizes the first decoded image and the second decoded image at a uniform mixing ratio regardless of the position in the attention area surrounding area Fr3. For example, the mixing ratio of the first and second decoded images is 50%. The attention area surrounding image synthesis unit 6 may increase the mixing ratio of the first decoded image as the motion vector detected by the attention area surrounding image determination unit 5 increases (as the motion increases).

[0021] For example, if the motion vector is equal to or greater than a first threshold, the attention area peripheral image synthesis unit 6 synthesizes the first decoded image at 60% and the second decoded image at 40%. If the motion vector is equal to or greater than a second threshold that is greater than the first threshold, the attention area peripheral image synthesis unit 6 synthesizes the first decoded image at 70% and the second decoded image at 30%. If the motion vector is equal to or greater than a third threshold that is greater than the second threshold, the attention area peripheral image synthesis unit 6 synthesizes the first decoded image at 80% and the second decoded image at 20%.

[0022] As shown in Fig. 5, as a second example, the attention region surrounding image synthesis unit 6 synthesizes the first decoded image and the second decoded image such that the mixing ratio of the first decoded image gradually decreases and the mixing ratio of the second decoded image gradually increases from the inner peripheral edge to the outer peripheral edge of the attention region surrounding area Fr3. In Fig. 5, the inner peripheral edge of the attention region surrounding area Fr3 is the edge on the attention region Fr1 side, and the outer peripheral edge is the edge on the area Fr20 side.

[0023] As shown in Fig. 6, as a third example, the attention area surrounding image synthesis unit 6 synthesizes the first decoded image and the second decoded image such that the mixing ratio of the first decoded image decreases from 100% to, for example, 50% from the inner edge to the outer edge of the attention area surrounding area Fr3, and the mixing ratio of the second decoded image increases from 0% to, for example, 50%. At the outer edge of the attention area surrounding area Fr3, the mixing ratio of the first decoded image may be a predetermined mixing ratio that is greater than 0% and less than 100%. At the outer edge of the attention area surrounding area Fr3, the mixing ratio of the second decoded image may be a mixing ratio obtained by subtracting the predetermined mixing ratio from 100%.

[0024] When the motion vector detected by the attention region surrounding image determination unit 5 is equal to or greater than a predetermined threshold, the attention region surrounding image synthesis unit 6 may synthesize the first decoded image and the second decoded image with the characteristics shown in Fig. 6. The attention region surrounding image synthesis unit 6 may increase the mixing ratio of the first decoded image at the outer periphery of the attention region surrounding area Fr3 as the motion vector detected by the attention region surrounding image determination unit 5 increases.

[0025] Figures 7A and 7B show only the mixing ratio of the first decoded image in Figure 6. As shown in Figures 7A and 7B, the attention area surrounding image synthesis unit 6 may make the mixing ratio of the first decoded image at the outer periphery of the attention area surrounding area Fr3 smaller or larger than that in Figure 6 depending on the magnitude of the motion vector.

[0026] The attention region peripheral image synthesis unit 6 may decrease or increase the mixing ratio of the first decoded image at the outer peripheral edge of the attention region peripheral area Fr3 as shown in Figures 7A and 7B, depending on the complexity of the image in the attention region peripheral area Fr3 detected by the attention region peripheral image determination unit 5. If the image is complex, the attention region peripheral image synthesis unit 6 may decrease the mixing ratio of the first decoded image at the outer peripheral edge of the attention region peripheral area Fr3 as shown in Figure 7A. If the image is not complex, the attention region peripheral image synthesis unit 6 may increase the mixing ratio of the first decoded image at the outer peripheral edge of the attention region peripheral area Fr3 as shown in Figure 7B.

[0027] 8 and 9, in the region surrounding the attention region Fr3, there is a boundary between a complex part ArC, which is a complex image having a complexity equal to or greater than a first threshold, and a non-complex part ArU, which is a non-complex image having a complexity less than a second threshold that is smaller than the first threshold. This type of boundary is often clearly visible to the human eye due to processing such as edge enhancement in the image.

[0028] 8, the attention region surrounding image synthesis unit 6 synthesizes the first decoded image and the second decoded image with a mixing ratio of the first decoded image being 100% and the second decoded image being 0% from the inner peripheral edge to the boundary of the attention region surrounding area Fr3. Also, the attention region surrounding image synthesis unit 6 synthesizes the first decoded image and the second decoded image with a mixing ratio of the first decoded image being 0% and the second decoded image being 100% from the boundary to the outer peripheral edge of the attention region surrounding area Fr3.

[0029] The attention region surrounding image synthesis unit 6 may synthesize the first decoded image and the second decoded image as shown in Fig. 9 instead of Fig. 8. As shown in Fig. 9, the attention region surrounding image synthesis unit 6 synthesizes the first decoded image and the second decoded image from the inner peripheral edge to the boundary of the attention region surrounding area Fr3, for example, with a mixing ratio of the first decoded image being 90% and the mixing ratio of the second decoded image being 10%. Also, the attention region surrounding image synthesis unit 6 synthesizes the first decoded image and the second decoded image from the boundary to the outer peripheral edge of the attention region surrounding area Fr3, for example, with a mixing ratio of the first decoded image being 10% and the mixing ratio of the second decoded image being 90%.

[0030] In this way, when there is a boundary where the image complexity changes within the attention region surrounding area Fr3, the attention region surrounding image synthesis unit 6 synthesizes the first decoded image and the second decoded image such that the mixing ratio of the first decoded image is greater than the mixing ratio of the second decoded image from the inner edge of the attention region surrounding area Fr3 to the boundary. Also, the attention region surrounding image synthesis unit 6 synthesizes the first decoded image and the second decoded image such that the mixing ratio of the second decoded image is greater than the mixing ratio of the first decoded image from the boundary to the outer edge of the attention region surrounding area Fr3.

[0031] Returning to FIG. 1, the attention region surrounding image synthesis unit 6 supplies the synthetic decoded image of the attention region surrounding area Fr3 generated as described above to the image synthesis unit 7. The image synthesis unit 7 receives the third decoded image of the attention region Fr1 decoded by the low compression image decoding unit 3 and the fourth decoded image excluding the decoded image of the attention region surrounding area Fr3 among the decoded images of the non-attention region Fr2 decoded by the high compression image decoding unit 4. The timing is adjusted so that the third decoded image of the attention region Fr1, the synthetic decoded image of the attention region surrounding area Fr3, and the fourth decoded image excluding the decoded image of the attention region surrounding area Fr3 among the decoded images of the non-attention region Fr2 of the same frame are input to the image synthesis unit 7.

[0032] The image synthesis unit 7 synthesizes the third decoded image, the composite decoded image, and the fourth decoded image to generate a composite decoded image for each frame. The image synthesis unit 7 supplies the composite decoded image data to a display unit (not shown).

[0033] As described above, the image decoding device 100 can suppress an unnatural feeling in the decoded image of each frame even if the region of interest Fr1 and the region of non-interest Fr2 are compressed at different compression rates.

[0034] The present invention is not limited to the above-described embodiment, and various modifications are possible without departing from the scope of the present invention. The image decoding device 100 may be configured with hardware or software. The image decoding device 100 may be configured with a mixture of hardware and software. The use of hardware and software is arbitrary. [Explanation of symbols]

[0035] 1. Communications Department 2 Storage section 3. Low-compression image decoding section 4 Highly compressed image decoding section 5. Image determination unit for the area of ​​interest 6. Area of ​​interest surrounding image synthesis unit 7 Image synthesis section 100 Image decoding device

Claims

1. a low-compression image decoding unit that decodes images of the region of interest and the surrounding region of the region of interest in encoded image data in which a portion of the region of interest in each frame is set as a region of interest, the region of interest and a surrounding region of the region of interest adjacent to the region of interest and surrounding the region of interest are encoded at a first compression rate, and non-attention regions other than the region of interest in each frame are encoded at a second compression rate higher than the first compression rate; a high-compression image decoding unit that decodes an image of the non-attention area in the encoded image data; a region-of-interest surrounding image synthesis unit that synthesizes a first decoded image of the region surrounding the region of interest decoded by the low-compression image decoding unit and a second decoded image of the region surrounding the region of interest among the decoded images of the region of non-interest decoded by the high-compression image decoding unit to generate a synthesized decoded image of the region surrounding the region of interest; an image synthesis unit that synthesizes a third decoded image of the region of interest decoded by the low-compression image decoding unit, the synthesized decoded image generated by the region of interest peripheral image synthesis unit, and a fourth decoded image excluding the decoded image of the region of interest peripheral area among the decoded images of the region of non-interest decoded by the high-compression image decoding unit to generate a synthesized decoded image of each frame; An image decoding device comprising:

2. The image decoding device according to claim 1 , wherein the attention area surrounding image synthesis unit synthesizes the first decoded image and the second decoded image at a uniform mixing ratio regardless of a position within the attention area surrounding the attention area.

3. 2. The image decoding device according to claim 1, wherein the attention area surrounding image synthesis unit synthesizes the first decoded image and the second decoded image so that a mixing ratio of the first decoded image gradually decreases and a mixing ratio of the second decoded image gradually increases from the inner edge to the outer edge of the attention area surrounding the attention area.

4. 4. The image decoding device of claim 3, wherein the attention area surrounding image synthesis unit sets a mixing ratio of the first decoded image to 100% at the inner edge and a mixing ratio of the second decoded image to 0%, sets a mixing ratio of the first decoded image to a predetermined mixing ratio greater than 0% and less than 100% at the outer edge, and sets a mixing ratio of the second decoded image to a mixing ratio obtained by subtracting the predetermined mixing ratio from 100%.

5. 2. The image decoding device of claim 1, wherein, when a boundary where image complexity changes exists within the region surrounding the region of interest, the attention area surrounding image synthesis unit synthesizes the first decoded image and the second decoded image so that a mixing ratio of the first decoded image from the inner edge of the region surrounding the region of interest to the boundary is greater than a mixing ratio of the second decoded image, and a mixing ratio of the second decoded image from the boundary to the outer edge of the region surrounding the region of interest is greater than a mixing ratio of the first decoded image.

Images