Reduction device and program

Abstract

To generate a reduced image with reduced artifacts such as block distortion.SOLUTION: A reduction device 1 includes a frequency decomposition unit 11 that performs frequency decomposition on the original image to generate a frequency band component, a degeneration processing unit 13 that generates a degenerated frequency band component by degenerating the frequency band components according to encoding information, a frequency reconstruction unit 14 that performs frequency reconstruction on the degenerated frequency band component to generate a degenerated image of the same size as the original image, a resolution reduction unit 15 that generates a reduced image by reducing the resolution of the reduced image, and an encoding information extraction unit 16 that performs encoding processing on the reduced image to extract the encoding information.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a reduction device and a program.

Background Art

[0002] In Patent Document 1, when the decoded image itself is deteriorated when performing super-resolution processing on the decoded image, depending on the super-resolution parameters, the deterioration component itself may be greatly emphasized. Therefore, until a predetermined end condition is satisfied, super-resolution processing and reduction restoration processing are repeatedly performed to obtain an optimal super-resolution parameter.

[0003] Further, Patent Document 2 discloses a technique in which an input image is once reduced and converted to an intermediate resolution, and after performing existing encoding / decoding on this, it is returned to the original resolution. In this method, the amount of code corresponding to the encoder and the optimal resolution reduction rate for that amount of code are accumulated in advance, and an optimal intermediate resolution is selected.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, in conventional image reduction techniques, when a sufficient bit rate cannot be ensured, in regions where encoding is difficult, such as regions with high high-frequency power, image quality degradation due to encoding artifacts such as block distortion is likely to occur.

[0006] An object of the present invention made in view of such circumstances is to provide a reduction device and a program capable of generating a reduced image in which artifacts such as block distortion are suppressed.

Means for Solving the Problems

[0007] The gist of the present invention for solving the above problems is as follows.

[0008] (1) A reduction device that generates a reduced image of an original image, comprising: a frequency decomposition unit that performs frequency decomposition on the original image to generate frequency band components; a degeneracy processing unit that generates degenerate frequency band components obtained by degenerating the frequency band components according to encoding information; a frequency reconstruction unit that performs frequency reconstruction on the degenerate frequency band components to generate a degenerate image having the same size as the original image; a resolution reduction unit that generates a reduced image with the resolution of the degenerate image reduced; and an encoding information extraction unit that encodes the reduced image to extract the encoding information.

[0009] (2) The reduction device according to (1), wherein generation of the reduced image of the original image is repeatedly performed until a change amount of the encoding information becomes equal to or less than a threshold value.

[0010] (3) The encoding information includes information indicating information indicating a motion vector, and the degeneracy processing unit increases a degeneracy rate as the motion vector is larger for a predetermined frequency band. The reduction device according to (1) or (2).

[0011] (4) The encoding information includes information indicating a prediction residual signal, and the degeneracy processing unit increases a degeneracy rate as the prediction residual signal is larger for a predetermined frequency band. The reduction device according to any one of (1) to (3).

[0012] (5) The encoding information includes information indicating a quantization parameter, and the degeneracy processing unit increases a degeneracy rate as the value of the quantization parameter is larger for a predetermined frequency band. The reduction device according to any one of (1) to (4).

[0013] The reduction device according to (5), wherein the degeneracy processing unit degenerates the frequency band component using a logarithmic function.

[0014] A program according to one embodiment causes a computer to function as the reduction device according to any one of (1) to (6). [Advantages of the Invention]

[0015] According to the present invention, it is possible to perform high-quality image reduction processing as pre-encoding processing. [Brief Description of the Drawings]

[0016]

Figure 1

Figure 2

[0017] Hereinafter, one embodiment will be described in detail with reference to the drawings.

[0018] FIG. 1 is a block diagram showing a configuration example of a reduction device according to one embodiment. The reduction device 1 shown in FIG. 1 includes a frequency decomposition unit 11, a degeneracy function setting unit 12, a degeneracy processing unit 13, a frequency reconstruction unit 14, a resolution reduction unit 15, an encoding information extraction unit 16, and an end determination unit 17.

[0019] The reduction device 1 inputs an original image and generates and outputs a reduced image with the resolution of the original image (input image) reduced. The reduction device 1 can be used as pre-processing for image encoding, for example, when reducing an original image taken at 8K resolution to 4K resolution for broadcasting.

[0020] The frequency decomposition unit 11 performs frequency decomposition in the spatial direction on the original image and generates power components for each frequency band (hereinafter referred to as "frequency band components"). Each component represents a power spectrum. Then, the frequency decomposition unit 11 outputs the generated frequency band components to the degeneracy processing unit 13.

[0021] In this embodiment, the frequency decomposition unit 11 performs wavelet packet decomposition as the frequency decomposition. In wavelet packet decomposition, frequency decomposition is performed evenly in the spatial direction. That is, frequency decomposition is performed not only on the low-frequency band but also on the high-frequency band. Note that the wavelet filter and the number of decomposition levels can be arbitrarily set by the user.

[0022] FIG. 2 is a diagram showing a state in which an original image with 8K resolution is subjected to 3-level wavelet packet decomposition in the spatial direction and decomposed into frequency bands of 1K×0.5K each. In order to perform high-precision multi-resolution decomposition of the original image, it is desirable to use a wavelet filter having linear phase, a relatively long tap length, and a sharp cutoff characteristic (for example, CDF (Cohen-Daubechies-Feauveau) 9 / 7, Biorthogonal (6,8), etc.). Also, the power between each band should satisfy Parseval's equation.

[0023] Also, in this embodiment, the frequency decomposition unit 11 performs frequency decomposition using wavelet packet decomposition while retaining phase information (that is, without decimation processing involving reduction of the image size). Therefore, when the size of the original image is 8K×4K, the number of elements in each frequency band is 8K×4K.

[0024] The degeneracy function setting unit 12 determines a degeneracy function based on the input degeneracy mode and outputs degeneracy function information, which is information indicating the determined degeneracy function, to the degeneracy processing unit 13. The degeneracy modes in this embodiment are the "linear function mode", the "logarithmic function mode", and the "temporary function + logarithmic function mode". The degeneracy coefficient is a coefficient indicating the degree of degeneracy. The degeneracy function information may be set by the user via the input I / F.

[0025] When the degeneracy mode is the linear function mode, the degeneracy function setting unit 12 determines a degeneracy function represented by a linear function. In this embodiment, with the input value being x, the output value being y, and the degeneracy coefficients being a, b, and c, the degeneracy function A with a large degeneracy rate is y = ax, the degeneracy function B with a medium degeneracy rate is y = bx, and the degeneracy function C with a small degeneracy rate is y = cx. For example, let a = 0.01, b = 0.1, and c = 0.5.

[0026] Also, when the degeneracy mode is the logarithmic function mode, the degeneracy function setting unit 12 determines a degeneracy function represented by a logarithmic function. In this embodiment, with the input value being x, the output value being y, and the degeneracy coefficients being α, β, and γ, the degeneracy function D with a large degeneracy rate is given by Equation (1), the degeneracy function E with a medium degeneracy rate is given by Equation (2), and the degeneracy function F with a small degeneracy rate is given by Equation (3). For example, let α = 0.01, β = 0.1, and γ = 0.5.

[0027] y = -α log10(-x) (when x < -2) y = α tanh(0.166x) (when -2 ≤ x ≤ 2) (1) y = α log10(x) (when x > 2)

[0028] y = -β log10(-x) (when x < -2) y = β tanh(0.166x) (when -2 ≤ x ≤ 2) (2) y = β log10(x) (when x > 2)

[0029] y = -γ log10(-x) (when x < -2) y = γ tanh(0.166x) (when -2 ≤ x ≤ 2) (3) y = γ log10(x) (when x > 2)

[0030] Also, when the degeneracy mode is the linear function + logarithmic function mode, the degeneracy function setting unit 12 determines the above degeneracy functions A, B, C, D, E, and F.

[0031] Note that when there is no degeneracy function information, the degeneracy function setting unit 12 selects the linear function mode and determines the degeneracy functions A, B, and C with the degeneracy coefficients a, b, and c being 0.

[0032] The degeneracy processing unit 13 receives, from the end determination unit 17, the encoding information used in the encoding process of the reduced image in the encoding information extraction unit 16, and generates a degenerate frequency band component in which the frequency band components are degenerate according to the encoding information. The encoding information in the present embodiment is three types, namely, "motion vector information", "quantization parameter information", and "prediction residual signal information", but it may be any one or two of them.

[0033] "Quantization parameter information" is information indicating a quantization parameter (QP). The quantization parameter is a parameter that controls the quantization step when quantizing an orthogonally transformed signal in an encoding method such as H.265 / HEVC (High Efficiency Video Coding) or H.266 / VVC (Versatile Video Coding). For example, the QP value ranges from 0 to 51, and every time the quantization parameter increases by 6, the quantization step doubles. For details of the quantization parameter, refer to, for example, the following reference. [Reference] Supervised by Sakae Okubo, "Impress Standard Textbook Series H.265 / HEVC Textbook", Impress Japan Co., Ltd., October 21, 2013

[0034] "Motion vector information" is information indicating the motion vector of an object between the target frame and the reference frame when performing inter-frame prediction in an encoding method such as H.265 / HEVC or H.266 / VVC. The motion vector is obtained by a block matching method or the like.

[0035] "Prediction residual signal information" is information indicating the difference between the input image and the motion compensation prediction image in an encoding method such as H.265 / HEVC or H.266 / VVC.

[0036] Based on the encoded information, the degeneracy processing unit 13 switches, at each frequency band and each element position, between the degeneracy functions A, B, C (in the case of the linear function mode) determined by the degeneracy function setting unit 12, the degeneracy functions D, E, F (in the case of the logarithmic function mode), and the degeneracy functions A, B, C, D, E, F (in the case of the transient function + logarithmic function mode), and uses a larger degeneracy function for higher frequency band components.

[0037] The processes of the degeneracy processing unit 13, the frequency reconstruction unit 14, the resolution reduction unit 15, the encoded information extraction unit 16, and the end determination unit 17 are repeatedly performed. Since there is no encoded information in the first process of the degeneracy processing unit 13, the degeneracy processing unit 13 sets a predetermined initial value as the encoded information during the first process. For example, it may be set to the most stringent level of conditions. In that case, at all pixel positions, the maximum value set by the codec for the motion vector information, the QP information is 51, and the prediction residual signal information is 255.

[0038] Specific examples of the switching conditions of the degeneracy function are shown below.

[0039] (1) Apply the degeneracy function A or D to all elements within the frequency band equal to or higher than the reduced resolution. In the example of FIG. 2, if the reduced resolution is 4K horizontally and 2K vertically, apply the degeneracy function A or D in the frequency band indicated by the diagonal lines in the figure.

[0040] (2) Do not apply the degeneracy function (do not perform the degeneracy process) to all elements within the lowest frequency band (the frequency band filled with black in the example of FIG. 2).

[0041] For frequency bands other than (3), (1), and (2) (the frequency band filled in gray in the example of Fig. 2), the degeneracy function is applied according to Table 1 below. For motion vectors, assuming that the larger the motion vector at each element position, the higher the encoding difficulty, a degeneracy function with a large degeneracy rate is used. For quantization parameters, assuming that the larger the value of the quantization parameter at each element position (hereinafter referred to as "QP value"), the higher the encoding difficulty, a degeneracy function with a large degeneracy rate is used. For prediction residual signals, assuming that the larger the prediction residual signal at each element position, the higher the encoding difficulty, a large degeneracy function is used.

[0042]

Table 1

[0043] (4) Further, in the frequency band less than the reduced resolution and equal to or higher than half of the reduced resolution, the criteria in Table 1 may be applied with one level strengthened. For example, for the magnitude of the motion vector, degeneracy function A or D is applied when it is 32 or more, and degeneracy function B or E is applied when it is 8 or more and less than 32. When it is less than 8, no degeneracy function is applied.

[0044] (5) In Table 1, when different degeneracy functions are applied to the magnitude of the motion vector, QP value, and magnitude of the prediction residual signal respectively, select the degeneracy function with the largest degeneracy rate. Alternatively, the degeneracy function with the smallest degeneracy rate may be selected.

[0045] (6) In the case of the linear function + logarithmic function mode, when the degeneracy function with the largest degeneracy rate is due to the QP value, select degeneracy functions D, E, F (logarithmic function mode). When the degeneracy function with the largest degeneracy rate is due to the magnitude of the motion vector or the magnitude of the prediction residual signal, select degeneracy functions A, B, C (linear function mode). Since the QP value is set logarithmically, the degeneracy function in the logarithmic function mode is preferentially used for the QP value.

[0046] The frequency reconstruction unit 14 performs frequency reconstruction on the degenerate frequency band components generated by the degeneracy processing unit 13 to generate a degenerate image of the same size as the original image. For example, when the frequency decomposition unit 11 performs a third-order wavelet packet decomposition, a third-order wavelet packet reconstruction is performed as its inverse transform. Then, the frequency reconstruction unit 14 outputs the generated degenerate image to the resolution reduction unit 15.

[0047] The resolution reduction unit 15 reduces the resolution of the degenerate image generated by the frequency reconstruction unit 14 to generate a reduced image with a resolution of {H S , V S}. For example, the resolution reduction unit 15 reduces the 8K resolution to 4K resolution by performing pixel decimation at a ratio of 2:1 in the horizontal and vertical directions of the degenerate image. Note that the reduction ratio can be arbitrarily set by the user. When the reduction ratio is not an integer multiple, pixel interpolation using a Lanczos-4 filter or the like is performed, and then pixel decimation is performed. The type of interpolation filter may be determined in advance or may be set by the user. Then, the resolution reduction unit 15 outputs the generated reduced image to the encoding information extraction unit 16.

[0048] The encoding information extraction unit 16 inputs the reduced image from the resolution reduction unit 15 and performs encoding processing at a required codec and required bit rate (for example, codec H.266 / VVC and bit rate 10 Mbps), and extracts information indicating the motion vector, quantization parameter, and prediction residual signal at each pixel position as encoding information. Then, the encoding information extraction unit 16 outputs the extracted encoding information to the end determination unit 17.

[0049] The end determination unit 17 determines whether to perform the generation process of the reduced image again on the original image or to end the generation process of the reduced image. When the end determination unit 17 determines to perform the generation process of the reduced image again, it outputs the encoded information to the degradation processing unit 13. Then, the processing from the degradation processing unit 13 to the encoded information extraction unit 16 is repeatedly performed. When the end determination unit 17 determines to end the generation process of the reduced image, it outputs the reduced image generated by the resolution reduction unit 15 to the outside of the reduction device 1. The end determination unit 17 compares the newly acquired encoded information by the encoded information extraction unit 16 with the previous encoded information, and determines to end the process when the amount of change in the encoded information becomes equal to or less than the threshold value. That is, the reduction device 1 repeatedly performs the generation of the reduced image for the original image until the amount of change in the encoded information becomes equal to or less than the threshold value. The end determination unit 17 may determine to end the process when the number of repetitions of the repetitive process reaches a predetermined number of times.

[0050] As described above, the reduction device 1 decomposes the original image into frequency band components and performs degradation processing in consideration of the encoding difficulty according to the encoded information. Therefore, for example, when reducing and broadcasting an original image taken at 8K resolution to 4K resolution, by reducing the image with the reduction device 1, it becomes possible to suppress image quality degradation due to encoding artifacts that are likely to occur in images that are difficult to encode.

[0051] <Program> In order to function as the above-described reduction device 1, it is also possible to use a computer capable of executing program instructions. Here, the computer may be a general-purpose computer, a dedicated computer, a workstation, a PC (Personal Computer), an electronic notebook pad, or the like. The program instructions may be program codes, code segments, etc. for executing necessary tasks.

[0052] A computer includes a processor, a memory unit, an input unit, an output unit, and a communication interface. The processor may be a CPU (Central Processing Unit), MPU (Micro Processing Unit), GPU (Graphics Processing Unit), DSP (Digital Signal Processor), SoC (System on a Chip), etc., and may be composed of a plurality of processors of the same type or different types. The processor reads a program from the memory unit and executes it to control each of the above components and perform various arithmetic processes. Note that at least a part of these processing contents may be realized by hardware. The input unit is an input interface that receives a user's input operation and acquires information based on the user's operation, such as a pointing device, a keyboard, a microphone, etc. The output unit is an output interface that outputs information, such as a display, a speaker, etc. The communication interface is an interface for communicating with an external device.

[0053] The program may be recorded on a computer-readable recording medium. By using such a recording medium, it is possible to install the program on the computer. Here, the recording medium on which the program is recorded may be a non-transitory recording medium. The non-transitory recording medium is not particularly limited, and may be, for example, a CD-ROM, a DVD-ROM, a USB (Universal Serial Bus) memory, etc. Also, this program may be in a form downloaded from an external device via a network.

[0054] For example, a program for causing the reduction device 1 to function performs frequency decomposition on the original image to generate frequency band components, generates degenerate frequency band components in which the frequency band components are degenerate according to the encoded information, performs frequency reconstruction on the degenerate frequency band components to generate a degenerate image having the same size as the original image, generates a reduced image in which the resolution of the degenerate image is reduced, and causes a computer to execute a step of encoding the reduced image to extract encoded information.

[0055] Further, the above-described reduction device 1 may be constituted by one or a plurality of semiconductor chips. This semiconductor chip may be equipped with a CPU that executes a program describing the processing content for realizing each function of the reduction device 1.

[0056] Although the above-described embodiments have been described as representative examples, it is obvious to those skilled in the art that many changes and substitutions can be made within the spirit and scope of the present invention. Therefore, the present invention should not be construed as being limited by the above-described embodiments, and various modifications or changes are possible without departing from the scope of the claims. For example, it is possible to integrate a plurality of constituent blocks described in the configuration diagram of the embodiment or to divide one constituent block.

Description of Signs

[0057] 1 Reduction device 11 Frequency decomposition unit 12 Degenerate function setting unit 13 Degenerate processing unit 14 Frequency reconstruction unit 15 Resolution reduction unit 16 Encoded information extraction unit 17 End determination unit

Claims

Claim 1 A reduction device for generating a reduced image of an original image, comprising: a frequency decomposition unit that performs frequency decomposition on the original image to generate frequency band components; a degeneracy processing unit that generates degenerate frequency band components obtained by degenerating the frequency band components according to encoding information; a frequency reconstruction unit that performs frequency reconstruction on the degenerate frequency band components to generate a degenerate image having the same size as the original image; a resolution reduction unit that generates a reduced image with the resolution of the degenerate image reduced; an encoding information extraction unit that encodes the reduced image and extracts the encoding information; A reduction device comprising the above components. Claim 2 The reduction device according to claim 1, wherein generation of a reduced image of the original image is repeatedly performed until a change amount of the encoding information becomes equal to or less than a threshold value. Claim 3 The encoding information includes information indicating information indicating a motion vector, The degeneracy processing unit increases a degeneracy rate as a motion vector is larger for a predetermined frequency band. The reduction device according to claim 1 or 2. Claim 4 The encoding information includes information indicating a prediction residual signal, The degeneracy processing unit increases a degeneracy rate as a prediction residual signal is larger for a predetermined frequency band. The reduction device according to claim 1 or 2. Claim 5 The encoding information includes information indicating a quantization parameter, The degeneracy processing unit increases a degeneracy rate as a value of the quantization parameter is larger for a predetermined frequency band. The reduction device according to claim 1 or 2. Claim 6 The reduction device according to claim 5, wherein the degeneracy processing unit degenerates the frequency band components using a logarithmic function. Claim 7 A program for causing a computer to function as the reduction device according to claim 1.

Images