Shrinkage device and program

Abstract

To generate a high-quality reduced image.SOLUTION: A reducing apparatus 1 includes: a frequency decomposition unit 10 which performs frequency band decomposition in a spatial direction until each frequency band becomes equal to or lower than a spatial resolution of a reduced image with respect to an original image, to generate a frequency band component; an encoding information extraction unit 13 which performs encoding processing on the reduced image to extract encoding information; a degeneration processing unit 14 which performs degeneration processing on the frequency band component using the encoding information to generate a degenerated frequency band component; a frequency reconfiguration unit 15 which performs frequency reconstruction on the degenerated frequency band component to generate a degenerated image of the same size as the original image; and a spatial resolution reduction unit 16 which generates a reduced image of the degenerated image. The degeneration processing unit 14 sets a degeneration rate of an encoding block position where a quantization parameter is equal to or larger than a threshold, in a frequency band exceeding a lowest frequency band and having a resolution equal to or lower than the spatial resolution of the reduced image, so as to become higher than a degeneration rate of the encoding block position where the quantization parameter is below the threshold.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a reduction device and a program.

Background Art

[0002] In Patent Document 1, when the decoded image itself is deteriorated when performing super-resolution processing on the decoded image, depending on the super-resolution parameters, the deterioration components themselves may be greatly emphasized. Therefore, by repeatedly performing super-resolution processing and reduction restoration processing until a predetermined end condition is satisfied, a technique for obtaining optimal super-resolution parameters is disclosed.

[0003] Also, in Patent Document 2, in a method of once reducing an input image and converting it to an intermediate resolution, performing existing encoding / decoding on this, and then returning to the original resolution, the amount of code corresponding to the encoder and the optimal resolution reduction rate for that amount of code are accumulated in advance, and a technique for selecting an optimal intermediate resolution is disclosed.

[0004] On the other hand, in encoding methods such as H.265 / HEVC (High Efficiency Video Coding) and H.266 / VVC (Versatile Video Coding), orthogonal transformation such as DCT (Discrete Cosine Transform) and motion compensation using motion vectors are performed. Details of the technology of H.265 / HEVC are described in detail in, for example, Non-Patent Document 1.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Patent Document 2

Non-Patent Documents

[0006]

Non-Patent Document 1

[0007] In the prior art disclosed in Patent Documents 1 and 2, when generating reduced-size images, artifacts such as block distortion occur in images that are difficult to encode, resulting in a deterioration of image quality. Generally, when a sufficient amount of encoding cannot be secured in images that are difficult to encode, image quality degradation due to encoding artifacts is more likely to occur.

[0008] In view of these circumstances, the object of the present invention is to provide a reduction device and program capable of generating high-quality reduced images. [Means for solving the problem]

[0009] To solve the above problems, a reduction device according to one embodiment is a reduction device that generates a reduced image of an original image, comprising: a frequency decomposition unit that generates frequency band components by performing spatial frequency band decomposition of the original image while retaining phase information until each frequency band is less than or equal to the spatial resolution of the reduced image; and an encoding information extraction unit that performs encoding processing on the reduced image and extracts encoding information including the quantization parameters used for encoding for each encoding block. Input from the frequency decomposition unit For the aforementioned frequency band components, The information input from the aforementioned coded information extraction unit A degeneration processing unit that performs degeneration processing using the aforementioned encoded information to generate a degenerate frequency band component in which the components have been degenerated, Input from the aforementioned degraded processing unit A frequency reconstruction unit performs frequency reconstruction on the aforementioned degenerate frequency band components to generate a degenerate image of the same size as the original image. Input from the frequency reconstruction unit The system includes a spatial resolution reduction unit that generates a reduced image by reducing the spatial resolution of the aforementioned degenerate image, The encoding information extraction unit receives the reduced image from the spatial resolution reduction unit,The degeneration processing unit does not perform degeneration processing in the lowest frequency band, and in the frequency band exceeding the lowest frequency band and below the spatial resolution of the reduced image, it sets the degeneration rate of the encoded block positions where the quantization parameter is equal to or greater than the first threshold to be higher than the degeneration rate of the encoded block positions where the quantization parameter is less than the first threshold.

[0010] Furthermore, in a reduction device according to one embodiment, the reduction processing unit may set the reduction rate of the frequency band component that exceeds the spatial resolution of the reduced image to be the highest.

[0011] Furthermore, in the reduction device according to one embodiment, the processing by the encoded information extraction unit, the degeneration processing unit, the frequency reconstruction unit, and the spatial resolution reduction unit may be repeated until the value of the quantization parameter becomes smaller than the second threshold.

[0012] Furthermore, in a reduction apparatus according to one embodiment, Input from the frequency decomposition unit The noise level setting unit further comprises a noise level setting unit that calculates the noise level using the highest diagonal frequency band component among the aforementioned frequency band components, and the degradation processing unit is Input from the aforementioned noise level setting unit The degeneration rate of the frequency band components below the noise level may be higher than the degeneration rate of the frequency band components above the noise level.

[0013] Furthermore, in a reduction device according to one embodiment, the degeneracy processing unit may set the degeneracy rate of spatiotemporal frequency band components whose absolute values ​​are included in a predetermined order from the highest to the lowest higher to be higher than the degeneracy rate of spatiotemporal frequency band components whose absolute values ​​are not included in the predetermined order.

[0014] Furthermore, in the reduction apparatus according to one embodiment, the frequency decomposition unit may also perform frequency band decomposition in the time direction with respect to the original image to generate the frequency band components.

[0015] Furthermore, in a reduction device according to one embodiment, the reduction processing unit may perform reduction processing when the time-direction frequency band component exceeds a third threshold in a frequency band that exceeds the lowest frequency band and is less than or equal to the spatial resolution of the reduced image.

[0016] Furthermore, the program according to one embodiment causes the computer to function as the above-mentioned reduction device. [Effects of the Invention]

[0017] According to the present invention, high-quality reduced images can be generated. [Brief explanation of the drawing]

[0018] [Figure 1] This is a block diagram showing an example configuration of a reduction device according to the first embodiment. [Figure 2] This figure shows the results of decomposing an 8K resolution original image into 4th wavelet packets in the spatial direction. [Figure 3] This figure shows the results of decomposing an 8K resolution original image into 4th wavelet packets in the spatial direction and 1st wavelet packets in the temporal direction. [Figure 4] This is a block diagram showing an example configuration of a reduction device according to the second embodiment. [Figure 5] This figure shows an example of a basic degeneracy function used in the reduction device according to the second embodiment. [Figure 6] This is a block diagram showing an example configuration of a reduction device according to the third embodiment. [Modes for carrying out the invention]

[0019] Embodiments of the present invention will be described in detail below with reference to the drawings.

[0020] (First embodiment) Figure 1 is a block diagram showing an example configuration of a reduction device 1 according to a first embodiment of the present invention. The reduction device 1 shown in Figure 1 comprises a frequency decomposition unit 10, an encoded information extraction unit 13, a degeneration processing unit 14, a frequency reconstruction unit 15, and a spatial resolution reduction unit 16.

[0021] The reduction device 1 takes an original image as input, generates a reduced image by reducing the spatial resolution of the original image (input image), and outputs it. The reduction device 1 can be used as a pre-processing step for image encoding, for example, when reducing an original image shot at 8K resolution to 4K resolution for broadcasting.

[0022] The frequency decomposition unit 10 comprises a spatial frequency band decomposition unit 11 and a time frequency band decomposition unit 12. In this embodiment, the frequency decomposition unit 10 is described as having a time frequency band decomposition unit 12, but it may also be configured without a time frequency band decomposition unit 12.

[0023] The spatial frequency band decomposition unit 11 performs spatial frequency band decomposition while retaining phase information with respect to the original image (i.e., without decimation) until each frequency band is less than or equal to the spatial resolution of the reduced image output by the reduction device 1, thereby generating components for each spatial frequency band (hereinafter referred to as "spatial frequency band components"). Each component exhibits a power spectrum. The spatial frequency band decomposition unit 11 then outputs the spatial frequency band components to the temporal frequency band decomposition unit 12. In this embodiment, wavelet packet decomposition is performed as frequency band decomposition. The wavelet filter and the number of spatial decomposition layers can be arbitrarily set by the user.

[0024] Figure 2 shows the result of decomposing an 8K resolution original image into 4th-order wavelet packets in the spatial direction, resulting in spatial frequency bands of 1K × 0.5K. To perform high-precision multi-resolution decomposition of the original image, it is desirable to use wavelet filters with linear phase properties, relatively long tap lengths, and steep cutoff characteristics (e.g., CDF (Cohen-Daubechies-Feauveau) 9 / 7, Biorthogonal (6,8), etc.). Furthermore, the power between each band is assumed to satisfy Parseval's identity.

[0025] Furthermore, in this embodiment, the spatial frequency band decomposition unit 11 performs spatial frequency band decomposition using wavelet packet decomposition without decimation. Therefore, the number of spatial elements within each frequency band is 8K × 4K. In Figure 2, the frequency bands indicated by XX are composed of four frequency bands: LL, LH, HL, and HH.

[0026] The time-frequency band decomposition unit 12 performs time-direction frequency band decomposition on the spatial frequency band components generated by the spatial frequency band decomposition unit 11, generating power spectral element components for each spatiotemporal frequency band (hereinafter referred to as "spatiotemporal frequency band components"). The time-frequency band decomposition unit 12 then outputs the spatiotemporal frequency band components to the degeneration processing unit 14. In this embodiment, wavelet decomposition is performed as frequency band decomposition. The wavelet filter and the number of time-decomposition levels can be arbitrarily set by the user.

[0027] Figure 3 shows the results of decomposing an 8K resolution source image into 4th wavelet packets in the spatial direction and 1st wavelet packets in the temporal direction. When a spatial frequency band component of 60 frames / second is decomposed into 1st wavelet packets, it is decomposed into a temporal frequency band of 1 to 30 frames / second and 31 to 60 frames / second. Here, the wavelet filter is the same as that of the spatial frequency band decomposition unit 11. Also, when wavelet packet decomposition is performed without decimation, the number of elements in the temporal direction within each frequency band is 60. The number of elements in the spatiotemporal direction within each frequency band is 8K × 4K × 60.

[0028] The symbolic information extraction unit 13 performs encoding processing on the reduced image generated by the spatial resolution reduction unit 16 described later, and extracts the encoding information used for encoding for each encoding block. Then, the encoding information extraction unit 13 outputs the encoding information to the degradation processing unit 14. Here, as an example, the encoding processing is performed using VVC / H.266, and as the encoding information, the division information of the encoding block (that is, the coding unit CU (Coding Unit)), and the quantization parameter QP (Quantization Parameter) and motion vector information for each CU are extracted. The division information of the CU is the size information and position information of the CU.

[0029] The degradation processing unit 14 acquires degradation setting information from the outside. The degradation setting information is information including reduced spatial resolution information, band-limited frequency information, and degradation coefficient information. First, after explaining these information, the degradation processing will be explained.

[0030] The reduced spatial resolution information is information that specifies the spatial resolution of the generated reduced image, and is set by the user. Let the horizontal reduced spatial resolution be H S and the vertical reduced spatial resolution be V S Then, for example, the reduced spatial resolution {H S , V S} is selected from the following 8 types. {H S , V S} = {1K, 0.5K}, {2K, 1K}, {3K, 1.5K}, {4K, 2K}, {5K, 2.5K}, {6K, 3K}, {7K, 3.5K}, {8K, 4K} (8 types)

[0031] The band-limited frequency information is information that specifies the lower limit frequency of the band limitation, and is set by the user. Let the horizontal band-limited lower frequency be H L and the vertical band-limited lower frequency be V L and the temporal band-limited lower frequency be T L Then, for example, the band-limited lower frequency {H L,V L ,T L The combinations of} are as follows. horizontal direction H L =1K,2K,3K,4K,5K,6K,7K,8K(8 types) Vertical V L =0.5K,1K,1.5K,2K,2.5K,3K,3.5K,4K(8 types) Time direction T L =30,60 (2 types)

[0032] If the reduction device 1 does not include a time-frequency band resolution unit 12, the lower limit frequency of the band limitation is {H L ,V L}. Additionally, the lower frequency limit for bandwidth restriction can be set to on / off. If off, {H L ,V L}={H S ,V S} and if the reduction device 1 is equipped with a time-frequency band resolution unit 12, then T L = 60.

[0033] Degeneracy coefficient information refers to information indicating a degeneracy coefficient between 0 and 1, and is set by the user. There may be multiple degeneracy coefficients. In this embodiment, the first degeneracy coefficient m1, which has the lowest degeneracy rate, the second degeneracy coefficient m2, which has the next lowest degeneracy rate, and the third degeneracy coefficient m3, which has the highest degeneracy rate, are used as the degeneracy coefficients. For example, the initial values ​​are m1=0.5 (3dB attenuation), m2=0.1 (10dB attenuation), and m3=0.001 (30dB attenuation).

[0034] The degeneration processing unit 14 uses the encoding information obtained from the encoding information extraction unit 13 and the degeneration setting information obtained from an external source to generate a degenerate spatiotemporal frequency band component (hereinafter referred to as the "degenerate frequency band component") from the spatiotemporal frequency band component obtained from the spatiotemporal frequency band decomposition unit 12. This degeneration process limits the frequency band. The degeneration processing unit 14 then outputs the degenerate frequency band component to the frequency reconstruction unit 15.

[0035] An example of applying the degeneracy coefficient will be explained. The frequency band to which no degeneracy processing is performed and the frequency band to which the third degeneracy coefficient m3 is applied can be defined as follows, for example. No degeneracy treatment: The lowest spatiotemporal frequency band containing the DC component (lowest frequency band LL) 1 In this case, no degeneracy process is performed (i.e., the components are not changed). Third degeneracy coefficient m3: Reduced spatial resolution {H S ,V S Applicable in frequency bands exceeding}.

[0036] Furthermore, exceeding the lowest frequency band and reducing spatial resolution {H S ,V S For the following frequency bands, the first degeneracy coefficient m1 and the second degeneracy coefficient m2 are applied by switching between them for each CU position or element position, depending on the spatial frequency power ratio for each CU position and the coding information (quantization parameter QP, magnitude of the motion vector, etc.). The switching of the degeneracy coefficient is performed, for example, based on the following criteria. Second degeneracy coefficient m2: Exceeds the lowest frequency band and reduces spatial resolution {H S ,V S This is applied to CU positions in the following frequency band where the quantization parameter QP is greater than or equal to the threshold Tq (first threshold). First degeneracy coefficient m1: Exceeds the lowest frequency band and reduces spatial resolution {H S ,V S This is applied to CU positions in the following frequency band where the quantization parameter QP is less than the threshold Tq.

[0037] Furthermore, the degeneracy processing unit 14 may also perform degeneracy processing when the frequency band component in the time direction exceeds the time frequency threshold (third threshold). For example, the following may be added to the application criteria for the first degeneracy coefficient m1. First degeneracy coefficient m1: Exceeds the lowest frequency band and reduces spatial resolution {H S ,V S In the frequency band below}, the lower limit frequency of the band limitation {H L ,V L ,T L Applicable to spatiotemporal high-frequency bands exceeding}.

[0038] Furthermore, the degeneracy coefficients m1, m2, m3, and the conditions under which each degeneracy coefficient is applied can be arbitrarily set by the user and are not limited to the examples above. For example, the criterion for applying the first degeneracy coefficient m1 could be "exceeding the lowest frequency band and reducing the spatial resolution {H S ,V S In the frequency band below}, among the CU positions where the quantization parameter QP is less than the threshold Tq, the element position where the proportion of spatial high-frequency power is greater than or equal to the threshold Tp, or the CU position where the magnitude of the motion vector is greater than or equal to the threshold Tm may be used. Furthermore, instead of applying all of the first degeneracy coefficient m1, second degeneracy coefficient m2, and third degeneracy coefficient m3, one or more of them may be applied.

[0039] The frequency reconstruction unit 15 performs frequency reconstruction (e.g., wavelet packet inverse decomposition) on the degenerate frequency band components generated by the degenerate processing unit 14 to generate a degenerate image of the same size as the original image. Frequency reconstruction is performed in the time and spatial directions if the frequency decomposition unit 10 includes a time frequency band decomposition unit 12, and in the spatial direction if the frequency decomposition unit 10 does not include a time frequency band decomposition unit 12. The frequency reconstruction unit 15 then outputs the degenerate image to the spatial resolution reduction unit 16.

[0040] The spatial resolution reduction unit 16 generates a reduced image by reducing the spatial resolution of the degenerate image generated by the frequency reconstruction unit 15. The spatial resolution reduction unit 16 reduces the spatial resolution {H} by, for example, downsampling pixels from the degenerate image. S ,V S A reduced image of} is generated. The reduction ratio can be set arbitrarily by the user. If the reduction ratio is not an integer fraction, the user can specify the interpolation filter, such as Nearest neighbor, Bicubic, Lanczos-3, etc. The spatial resolution reduction unit 16 then outputs the reduced image to the outside of the reduction device 1 and to the degeneration processing unit 14.

[0041] When the reduction device 1 first generates a reduced image, the encoding information is not extracted by the encoding information extraction unit 13, so an initial value is used as the encoding information. The reduction device 1 repeatedly performs the processing by the encoding information extraction unit 13, the degeneration processing unit 14, the frequency reconstruction unit 15, and the spatial resolution reduction unit 16 until the quantization parameter QP extracted by the encoding information extraction unit 13 becomes smaller than the QP threshold (second threshold). When repeating this process, any parameter among the encoding information and degeneration setting information can be changed. The reduction device 1 also repeatedly performs the processing by the frequency decomposition unit 10 until the value of the quantization parameter QP becomes smaller than the QP threshold, and the filter used for frequency decomposition may be changed during the repetition.

[0042] As explained above, the reduction device 1 decomposes the original image into frequency band components and performs bandwidth limiting. This makes it possible to generate high-quality reduced images. For example, when reducing an original image shot at 8K resolution to 4K resolution for broadcast, reducing the image with the reduction device 1 can suppress image degradation due to encoding artifacts that tend to occur with images that are difficult to encode, and enable high-quality encoding of the reduced image.

[0043] (Second embodiment) Next, a reduction device according to the second embodiment will be described. Figure 4 is a block diagram showing an example configuration of the reduction device 2 according to the second embodiment. The reduction device 2 shown in Figure 4 comprises a frequency decomposition unit 10, an encoded information extraction unit 13, a degeneration processing unit 14a, a frequency reconstruction unit 15, a spatial resolution reduction unit 16, and a noise level setting unit 17. The reduction device 2 according to the second embodiment differs from the reduction device 1 according to the first embodiment in that it includes a degeneration processing unit 14a instead of a degeneration processing unit 14, and further includes a noise level setting unit 17. The other components are the same as those of the first embodiment, so the same reference numerals are used and their description is omitted.

[0044] The noise level setting unit 17 receives spatiotemporal frequency band components from the frequency decomposition unit 10 and sets the noise level as a parameter for noise reduction. The noise level setting unit 17 then outputs the noise level to the degradation processing unit 14a. The system may offer two modes: an automatic setting mode in which the noise level is set automatically by the user, and a manual setting mode in which the noise level is set manually. In manual setting mode, the user can set any noise level. The user may also be able to specify whether or not to set a noise level.

[0045] In automatic setting mode, the noise level setting unit 17 calculates the noise level using the highest diagonal frequency band component among the spatiotemporal frequency band components. There may be multiple noise levels. For example, the noise level setting unit 17 calculates two noise levels {L1, L2} using the following equations (1) to (3). In the example shown in Figure 2, the highest diagonal frequency band (hereinafter referred to as the "diagonal highest frequency band") is frequency band XX when there is no time wavelet packet decomposition. 64 Therefore, if time wavelet packet decomposition is enabled, the frequency bandwidth is XX 64 This is a time-frequency bandwidth. Also, XX 64 Among LL 64 ,HH 64 Any of the following frequency bands may be used. In the following formula, p is the total number of elements in the diagonal highest frequency band, and x i γ represents the component values ​​of the diagonal highest frequency band, and μ is the arithmetic mean of the component values ​​of the diagonal highest frequency band. In practice, the square root of equation (1) is a constant, so the calculation of L1 is (standard deviation of the component values ​​of the diagonal highest frequency band) × (constant). Also, in L2, outliers are excluded, so the value of the top γ% (for example, γ=95) may be considered the maximum value.

[0046]

number

[0047] The noise level setting unit 17 may further multiply the noise levels {L1, L2} by an offset for each frequency band identifier n=1 to 64 shown in Figure 2. Whether or not to apply the offset and the offset value can be set for each frequency band. The offsets for each frequency band are, for example, as shown in Table 1 below. In this example, for frequency bands n=1 to 4, the noise levels {L1, L2} are multiplied by 1.4 for the time low frequency band and by 1.2 for the time high frequency band. The offsets shown in Table 1 are applied similarly to the other frequency bands. Note that if the frequency decomposition unit 10 does not have a time frequency band decomposition unit 12, only the time low frequency band offset is used.

[0048] [Table 1]

[0049] The degeneracy processing unit 14a sets the degeneracy rate of spatiotemporal frequency band components below the noise level set by the noise level setting unit 17 to be higher than the degeneracy rate of spatiotemporal frequency band components above the noise level. For example, the degeneracy processing unit 14a generates a degeneracy function from the noise levels {L1, L2} (L1, L2 after offset when offset is applied) set by the noise level setting unit 17, and based on the degeneracy function, performs noise removal and degeneracy processing on the spatiotemporal frequency band components obtained from the time-frequency band decomposition unit 12 to generate degenerate frequency band components. The component value x at the element position to which the degeneracy function is applied is converted to y according to the degeneracy function y = f(x). There may be multiple types of degeneracy functions. In this embodiment, a basic degeneracy function, a first degeneracy function, a second degeneracy function, and a third degeneracy function are used as degeneracy functions.

[0050] Figure 5 shows an example of the basic degeneracy function f0(x). As shown in Figure 5, the basic degeneracy function f0(x) is y=0 in the interval where the absolute value of x, |x|, is greater than or equal to 0 and less than L1. In the interval where |x| is greater than or equal to L1 and less than L2, if x is positive, y=(x-L1)L2 / (L2-L1), and if x is negative, y=(x+L1)L2 / (L2-L1). In the interval where |x| is greater than or equal to L2, y=x. Note that if the noise rejection setting is off, L1=0 and L2=1, in which case y=x and no degeneracy occurs.

[0051] The degeneracy processing unit 14a generates a degenerate function by changing the slope of the basic degeneracy function using the degeneracy coefficients indicated by the degeneracy coefficient information. As described above, when the first degeneracy coefficient m1 with the lowest degeneracy rate, the second degeneracy coefficient m2 with the next lowest degeneracy rate, and the third degeneracy coefficient m3 with the highest degeneracy rate are used as the degeneracy coefficients, the unit generates the first degeneracy function f1(x) with the lowest degeneracy rate, the second degeneracy function f2(x) with the next lowest degeneracy rate, and the third degeneracy function f3(x) with the highest degeneracy rate.

[0052] The first degenerate function f1(x) is defined as y=0 in the interval where |x| is greater than or equal to 0 and less than L1. In the interval where |x| is greater than or equal to L1 and less than L2, if x is positive, y=m1(x-L1)L2 / (L2-L1), and if x is negative, y=m1(x+L1)L2 / (L2-L1). In the interval where |x| is greater than or equal to L2, y=m1x. Note that if the noise rejection setting is off, L1=0 and L2=1, in which case y=m1x.

[0053] The second degenerate function f2(x) is defined as y=0 in the interval where |x| is greater than or equal to 0 and less than L1. In the interval where |x| is greater than or equal to L1 and less than L2, if x is positive, y=m2(x-L1)L2 / (L2-L1), and if x is negative, y=m2(x+L1)L2 / (L2-L1). In the interval where |x| is greater than or equal to L2, y=m2x. Note that if the noise rejection setting is off, L1=0 and L2=1, in which case y=m2x.

[0054] The third degenerate function f3(x) is defined as y=0 in the interval where |x| is greater than or equal to 0 and less than L1. In the interval where |x| is greater than or equal to L1 and less than L2, if x is positive, y=m3(x-L1)L2 / (L2-L1), and if x is negative, y=m3(x+L1)L2 / (L2-L1). In the interval where |x| is greater than or equal to L2, y=m3x. Note that if the noise reduction setting is off, L1=0 and L2=1, in which case y=m3x.

[0055] The degeneration processing unit 14a generates a degenerate frequency band component from the spatiotemporal frequency band component obtained from the time-frequency band decomposition unit 12, using the encoded information obtained from the encoded information extraction unit 13, the degeneration setting information obtained from an external source, and the degeneration function, and outputs it to the frequency reconstruction unit 15.

[0056] The application of the degenerate function is the same as in the first embodiment, but a degenerate function is used instead of a degenerate coefficient. The frequency band to which no degenerate processing is performed and the frequency band to which the third degenerate function f3(x) is applied can be, for example, as follows. No degeneracy treatment: The lowest spatiotemporal frequency band containing the DC component (lowest frequency band LL) 1 No degeneracy process is performed on this (i.e., the components are not changed). Third degenerate function f3(x): Reduced spatial resolution {H S ,V S Applicable to frequency bands exceeding}.

[0057] Furthermore, exceeding the lowest frequency band and reducing spatial resolution {H S ,V S For the frequency band below, the first degenerate function f1(x) and the second degenerate function f2(x) are switched and applied to each CU position or element position according to the spatial frequency power ratio for each CU position and the coding information (quantization parameter QP, magnitude of the motion vector, etc.). The switching of the degenerate function is performed, for example, according to the following criteria. Second degenerate function f2(x): Exceeds the lowest frequency bandwidth and reduces spatial resolution {H S ,V S This is applied to CU positions where the quantization parameter QP is greater than or equal to the threshold Tq in the following frequency band. First degenerate function f1(x): Exceeds the lowest frequency bandwidth and reduces spatial resolution {H S ,V S In the frequency band below {H}, the CU position where the quantization parameter QP is less than the threshold Tq, and the lower limit frequency of the band limitation {H}. L ,V L ,T L Applicable to spatiotemporal high-frequency bands exceeding}. The fundamental degeneracy function f0(x) is applied to the remaining frequency band.

[0058] Note that the degenerate functions f0(x), f1(x), f2(x), f3(x), and the conditions for applying each degenerate function can be arbitrarily set by the user and are not limited to the examples above. For example, the criterion for applying the first degenerate function f1(x) could be "exceeding the lowest frequency bandwidth and reducing spatial resolution {H S ,V S In the frequency band below}, among the CU positions where the quantization parameter QP is less than the threshold Tq, the element position where the proportion of spatial high-frequency power is greater than or equal to the threshold Tp, or the CU position where the magnitude of the motion vector is greater than or equal to the threshold Tm, may also be used. Furthermore, instead of applying all of the fundamental degeneracy functions f0(x), first degeneracy function f1(x), second degeneracy function f2(x), and third degeneracy function f3(x), one or more may be applied.

[0059] Thus, in this embodiment, the noise level is calculated using the highest frequency band component in the diagonal direction, and the degeneration rate of frequency band components below the noise level is made higher than the degeneration rate of frequency band components above the noise level. Therefore, it is possible to perform noise reduction simultaneously during the degeneration process and improve image quality.

[0060] (Third embodiment) Next, a reduction device according to the third embodiment will be described. Figure 6 is a block diagram showing an example configuration of a reduction device 3 according to the third embodiment of the present invention. The reduction device 3 shown in Figure 6 comprises a frequency decomposition unit 10, an encoded information extraction unit 13, a degeneration processing unit 14b, a frequency reconstruction unit 15, a spatial resolution reduction unit 16, a noise level setting unit 17, and an element reduction threshold setting unit 18. The reduction device 3 according to the third embodiment differs from the reduction device 2 according to the second embodiment in that it includes a degeneration processing unit 14b instead of a degeneration processing unit 14a, and further includes an element reduction threshold setting unit 18. The other components are the same as those of the second embodiment, so the same reference numerals are used and their description is omitted.

[0061] The element reduction threshold setting unit 18 receives spatiotemporal frequency band components from the frequency decomposition unit 10 and sets the element reduction threshold α. The element reduction threshold setting unit 18 then outputs the element reduction threshold α to the degeneration processing unit 14b. The user may specify an automatic setting mode in which the element reduction threshold is set automatically, and a manual setting mode in which the element reduction threshold is set manually. In manual setting mode, the user can set any element reduction threshold α. The user may also specify whether or not to set an element reduction threshold.

[0062] In automatic setting mode, the element reduction threshold setting unit 18 arranges the spatiotemporal frequency band components in descending order of absolute value and determines the top j% component value as the element reduction threshold α.

[0063] The degeneracy processing unit 14b generates a degeneracy function from the noise levels {L1, L2} (L1, L2 after offset if offset is applied) set by the noise level setting unit 17, similar to the second embodiment. Then, the degeneracy processing unit 14a generates a degenerate frequency band component using the spatiotemporal frequency band component obtained from the time-frequency band decomposition unit 12, the encoded information obtained from the encoded information extraction unit 13, the degeneracy setting information obtained from an external source, the degeneracy function, and the element reduction threshold α obtained from the element reduction threshold setting unit 18, and outputs it to the frequency reconstruction unit 15.

[0064] The degeneracy processing unit 14b uses an element reduction threshold α to make the degeneracy rate of spatiotemporal frequency band components whose absolute values ​​are included in the top j% (a predetermined rank from the top) higher than the degeneracy rate of spatiotemporal frequency band components whose absolute values ​​are not included in the top j%. For example, an element reduction threshold α can be used when applying the first degenerate function f1(x) and the second degenerate function f2(x) as shown below. The rest is the same as in the second embodiment, so the explanation is omitted. Second degenerate function f2(x): Exceeds the lowest frequency bandwidth and reduces spatial resolution {H S ,V S In the frequency band below {H}, (1) the CU position where the quantization parameter QP is greater than or equal to the threshold Tq, and (2) the lower limit frequency of the band limitation {H}. L ,V L ,T L This applies to components with an absolute value exceeding the element reduction threshold α in the spatiotemporal high-frequency band exceeding}. First degenerate function f1(x): Exceeds the lowest frequency bandwidth and reduces spatial resolution {H S ,V S In the frequency band below {H}, (1) the CU position where the quantization parameter QP is less than the threshold Tq, and (2) the lower limit frequency of the band limitation {H}. L ,V L ,T L This applies to components with an absolute value less than or equal to the element reduction threshold α in the spatiotemporal high-frequency band exceeding}.

[0065] Thus, in this embodiment, the degeneracy rate of spatiotemporal frequency band components whose absolute values ​​are included in a predetermined order from the highest to the lowest is made higher than the degeneracy rate of spatiotemporal frequency band components whose absolute values ​​are not included in the predetermined order. Therefore, it is possible to increase the degeneracy rate of strong edge and texture components and improve coding efficiency.

[0066] (program) To enable the aforementioned reduction devices 1, 2, and 3 to function, it is also possible to use computers capable of executing program instructions. Here, the computers may be general-purpose computers, dedicated computers, workstations, PCs (Personal Computers), electronic notepads, etc. The program instructions may be program code, code segments, etc., for executing the required tasks.

[0067] A computer comprises a processor, a memory unit, an input unit, an output unit, and a communication interface. The processor may be a CPU (Central Processing Unit), MPU (Micro Processing Unit), GPU (Graphics Processing Unit), DSP (Digital Signal Processor), SoC (System on a Chip), etc., and may be composed of multiple processors of the same or different types. The processor controls each of the above components and performs various calculations by reading and executing programs from the memory unit. At least a part of these processes may be implemented in hardware. The input unit is an input interface that receives user input operations and acquires information based on user operations, such as a pointing device, keyboard, or mouse. The output unit is an output interface that outputs information, such as a display or speaker. The communication interface is an interface for communicating with external devices, such as a LAN (Local Area Network) interface.

[0068] The program may be recorded on a computer-readable recording medium. Using such a medium, the program can be installed on the computer. The recording medium on which the program is recorded may be a non-transitory recording medium. Non-transitory recording media are not particularly limited, but may include, for example, CD-ROMs, DVD-ROMs, or USB (Universal Serial Bus) memory. Alternatively, the program may be downloaded from an external device via a network.

[0069] For example, a program to function as a reduction device 1 causes the computer to perform the following steps: generating frequency band components by spatially decomposing the original image while retaining phase information until each frequency band is less than or equal to the spatial resolution of the reduction image; performing encoding on the reduction image and extracting encoded information, including the quantization parameters used for encoding, for each encoded block; performing degeneracy processing on the frequency band components using the encoded information to generate degenerate frequency band components with degenerated components; performing frequency reconstruction on the degenerate frequency band components to generate a degenerate image of the same size as the original image; and generating a reduction image with reduced spatial resolution of the degenerate image.

[0070] Furthermore, the shrinking devices 1, 2, and 3 may be composed of one or more semiconductor chips, and the semiconductor chip may be equipped with a CPU that executes a program describing the processing content that realizes each of the functions of the shrinking devices 1, 2, and 3.

[0071] Although the embodiments described above are representative examples, it will be apparent to those skilled in the art that many modifications and substitutions are possible within the spirit and scope of the present invention. Therefore, the present invention should not be interpreted as being limited by the embodiments described above, and various modifications or changes are possible without departing from the scope of the claims. For example, it is possible to integrate multiple component blocks shown in the configuration diagram of the embodiments, or to divide a single component block. [Explanation of Symbols]

[0072] 1,2,3 reduction device 10 Frequency Resolution Section 11 Spatial frequency band resolution section 12-hour frequency band resolution section 13 Encoded information extraction unit 14,14a,14b Degraded Processing Unit 15. Frequency Reconstruction Section 16 Spatial resolution reduction section 17. Noise level setting section 18. Element Reduction Threshold Setting Section

Claims

1. A reduction device that generates a reduced image of the original image, A frequency decomposition unit generates frequency band components by performing spatial frequency band decomposition on the original image while retaining phase information until each frequency band is less than or equal to the spatial resolution of the reduced image, An encoding information extraction unit performs encoding processing on the reduced image and extracts encoding information, including the quantization parameters used for encoding, for each encoding block. A degeneration processing unit performs a degeneration process on the frequency band components input from the frequency decomposition unit using the encoded information input from the encoded information extraction unit to generate a degenerate frequency band component in which the components have been degenerated. A frequency reconstruction unit performs frequency reconstruction on the degenerate frequency band components input from the degenerate processing unit to generate a degenerate image of the same size as the original image, A spatial resolution reduction unit generates a reduced image by reducing the spatial resolution of the degenerate image input from the frequency reconstruction unit, Equipped with, The encoding information extraction unit receives the reduced image from the spatial resolution reduction unit, The reduction device comprises a reduction processing unit which does not perform reduction processing in the lowest frequency band, and in the frequency band exceeding the lowest frequency band and below the spatial resolution of the reduced image, the reduction rate of the encoded block position where the quantization parameter is equal to or greater than a first threshold is made higher than the reduction rate of the encoded block position where the quantization parameter is less than the first threshold.

2. The reduction device according to claim 1, wherein the reduction processing unit sets the reduction rate of the frequency band component that exceeds the spatial resolution of the reduced image to the highest value.

3. The reduction apparatus according to claim 1 or 2, wherein the processing by the encoded information extraction unit, the degeneration processing unit, the frequency reconstruction unit, and the spatial resolution reduction unit is repeated until the quantization parameter becomes smaller than a second threshold.

4. The noise level setting unit further comprises a noise level setting unit that calculates the noise level using the highest diagonal frequency band component among the frequency band components input from the frequency decomposition unit, The reduction device according to any one of claims 1 to 3, wherein the reduction processing unit makes the reduction rate of the frequency band components below the noise level input from the noise level setting unit higher than the reduction rate of the frequency band components above the noise level.

5. The reduction device according to any one of claims 1 to 4, wherein the reduction processing unit makes the reduction rate of spatiotemporal frequency band components whose absolute values ​​are included in a predetermined order from the highest to the lowest higher than the reduction rate of spatiotemporal frequency band components whose absolute values ​​are not included in the predetermined order.

6. The reduction device according to any one of claims 1 to 5, wherein the frequency decomposition unit performs frequency band decomposition in the time direction on the original image to generate the frequency band components.

7. The reduction device according to claim 6, wherein the reduction processing unit performs reduction processing when the frequency band component in the time direction exceeds a third threshold in a frequency band that exceeds the lowest frequency band and is less than or equal to the spatial resolution of the reduced image.

8. A program for causing a computer to function as a reduction device according to any one of claims 1 to 7.

Images