Reduction device and program

Abstract

To generate a high-quality thumbnail image.SOLUTION: A reduction device 1 that generates a reduced image of an original image includes a frequency decomposition unit 10 that generates a frequency band component by performing frequency band decomposition in the spatial direction on the original image until each frequency band is equal to or less than the spatial resolution of the reduced image, a degeneration processing unit 13 that performs degeneration processing on frequency band components and generates degenerated frequency band components in which the components in each frequency band are degenerated, a frequency reconstruction unit 14 that performs frequency reconstruction on the degenerated frequency band components to generate a degenerated image having the same size as the original image, and a spatial resolution reduction unit 15 that generates a reduced image by reducing the spatial resolution of the degenerated image.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a reduction device and a program. [Background technology]

[0002] Patent Document 1 discloses a technology for obtaining optimal super-resolution parameters by repeatedly performing super-resolution processing and reduction restoration processing until a predetermined termination condition is met, since if the decoded image itself is degraded when super-resolution processing is performed on the decoded image, the degraded components themselves may be greatly emphasized depending on the super-resolution parameters.

[0003] Furthermore, Patent Document 2 discloses a method in which an input image is first reduced and converted to an intermediate resolution, and then the image is returned to the original resolution after undergoing conventional encoding and decoding. In this method, the amount of code corresponding to the encoder and the optimal resolution reduction ratio for that amount of code are stored in advance, and the optimal intermediate resolution is selected. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent No. 5262879 [Patent Document 2] Japanese Patent Application Laid-Open No. 2000-134618 Summary of the Invention [Problem to be solved by the invention]

[0005] However, the conventional techniques disclosed in Patent Documents 1 and 2 have a problem in that when a reduced image is generated, artifacts such as block distortion occur in images that are difficult to encode, resulting in degradation of image quality.

[0006] SUMMARY OF THE INVENTION In view of the above circumstances, an object of the present invention is to provide a reduction device and a program capable of generating a high-quality reduced image. [Means for solving the problem]

[0007] In order to solve the above problem, one embodiment of a reduction device is a reduction device that generates a reduced image of an original image, and includes: a frequency decomposition unit that performs frequency band decomposition in the spatial direction on the original image until each frequency band is equal to or less than the spatial resolution of the reduced image, thereby generating frequency band components; a degeneration processing unit that performs degeneration processing on the frequency band components, thereby generating degenerated frequency band components with degenerated components; a frequency reconstruction unit that performs frequency reconstruction on the degenerated frequency band components, thereby generating a degenerated image of the same size as the original image; and a spatial resolution reduction unit that generates a reduced image with a reduced spatial resolution of the degenerated image.

[0008] Furthermore, in the reduction device according to one embodiment, the degeneration processing unit may generate the degenerated frequency band components by multiplying components in a frequency band exceeding the spatial resolution of the reduced image by a first degeneration coefficient.

[0009] Furthermore, in one embodiment of the reduction device, the degeneration processing unit may detect edge components within a frequency band equal to or lower than the spatial resolution of the reduced image, and multiply the edge components by a second degeneration coefficient to generate the degenerated frequency band components.

[0010] Furthermore, in one embodiment of the reduction device, the degeneration processing unit may generate the degenerated frequency band components by multiplying components in a high-frequency band in a frequency band equal to or lower than the spatial resolution of the reduced image by a third degeneration coefficient.

[0011] Furthermore, in one embodiment of the reduction device, the reduction processing unit may multiply components in a frequency band exceeding the spatial resolution of the reduced image by a first reduction coefficient, detect edge components in a frequency band equal to or lower than the spatial resolution of the reduced image, multiply the edge components by a second reduction coefficient having a reduction rate lower than the first reduction coefficient, and multiply components in a high frequency band in a frequency band equal to or lower than the spatial resolution of the reduced image by a third reduction coefficient having a reduction rate lower than the second reduction coefficient to generate the reduced frequency band components.

[0012] Furthermore, in the reduction device according to one embodiment, the frequency decomposition section may also perform frequency band decomposition on the original image in the time direction to generate the frequency band components.

[0013] Furthermore, the reduction device according to one embodiment may further include a degeneration correction unit that calculates an average value for each band of the degenerate frequency band components and corrects the degenerate frequency band components corresponding to the reduced image so that the distribution of the average values ​​for each band of the degenerate frequency band components corresponding to the reduced image approaches the distribution of the average values ​​for each band of the degenerate frequency band components corresponding to the original image.

[0014] Moreover, a program according to one embodiment causes a computer to function as the reduction device. [Effects of the Invention]

[0015] According to the present invention, a high-quality reduced image can be generated. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a block diagram showing an example of the configuration of a reduction device according to a first embodiment. [Figure 2] FIG. 10 is a diagram showing the state of an original image with 8K resolution subjected to third-order wavelet packet decomposition in the spatial direction. [Figure 3] FIG. 10 is a diagram showing an original image with 8K resolution subjected to third-order wavelet packet decomposition in the spatial direction and first-order wavelet packet decomposition in the temporal direction. [Figure 4] 10 is a flowchart showing an example of processing by a reduction processing unit in the reduction device according to the first embodiment. [Figure 5] 5 is a diagram showing an example of coefficients set by a reduction processing unit in the reduction device according to the first embodiment. FIG. [Figure 6] FIG. 10 is a block diagram showing an example of the configuration of a reduction device according to a second embodiment. [Figure 7] 10A to 10C are diagrams illustrating an example of processing by a degeneration corrector in the reduction device according to the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0017] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings.

[0018] (First embodiment) Fig. 1 is a block diagram showing an example of the configuration of a reduction device 1 according to a first embodiment of the present invention. The reduction device 1 shown in Fig. 1 includes a frequency decomposition unit 10, a reduction processing unit 13, a frequency reconstruction unit 14, and a spatial resolution reduction unit 15.

[0019] The reduction device 1 receives an original image as input, generates a reduced image by reducing the spatial resolution of the original image (input image), and outputs the reduced image. The reduction device 1 can be used as a pre-processor for image coding, for example, when reducing an original image captured at 8K resolution to 4K resolution for broadcast.

[0020] The frequency decomposition unit 10 includes a spatial frequency band decomposition unit 11 and a time frequency band decomposition unit 12. In this embodiment, the frequency decomposition unit 10 is described as including the time frequency band decomposition unit 12, but the frequency decomposition unit 10 may not include the time frequency band decomposition unit 12.

[0021] The spatial frequency band decomposition unit 11 performs frequency band decomposition in the spatial direction on the original image until each frequency band is equal to or less than the spatial resolution of the reduced image output by the reduction device 1, and generates components for each spatial frequency band (hereinafter referred to as "spatial frequency band components"). Each component represents a power spectrum. The spatial frequency band decomposition unit 11 then outputs the spatial frequency band components to the time frequency band decomposition unit 12. In this embodiment, wavelet decomposition is performed as the frequency band decomposition. The wavelet filter and the number of spatial decomposition layers can be set arbitrarily by the user.

[0022] Figure 2 shows the state of an original 8K resolution image after spatially decomposing it into spatial frequency bands of 1K x 0.5K by third-order wavelet packet decomposition. To perform multi-resolution decomposition of an input image with high accuracy, it is desirable to use a wavelet filter with linear phase, a relatively long tap length, and steep cutoff characteristics (e.g., CDF (Cohen-Daubechies-Feauveau) 9 / 7, Biorthogonal (6,8), etc.). Furthermore, the power between each frequency band must satisfy Parseval's equation.

[0023] In this embodiment, the spatial frequency band decomposition unit 11 performs spatial frequency band decomposition using wavelet packet decomposition without decimation. "Without decimation" means that phase information is preserved. Therefore, in FIG. 2, the size of each frequency band is 1K x 0.5K, but the number of spatial elements within each frequency band is 8K x 4K.

[0024] The time-frequency band decomposition unit 12 performs frequency band decomposition in the time direction on the spatial frequency band components generated by the spatial frequency band decomposition unit 11, and generates power spectrum element components for each spatio-temporal frequency band (hereinafter referred to as "spatio-temporal frequency band components"). The time-frequency band decomposition unit 12 then outputs the spatio-temporal frequency band components to the degeneration processing unit 13. In this embodiment, wavelet decomposition is performed as the frequency band decomposition. Note that the wavelet filter and the number of time decomposition layers can be set arbitrarily by the user.

[0025] FIG. 3 is a diagram showing an original image with 8K resolution subjected to third-order wavelet packet decomposition in the spatial direction and first-order wavelet packet decomposition in the temporal direction. When a spatial frequency band component of 60 frames per second is subjected to first-order wavelet packet decomposition, it is decomposed into time frequency bands of 1 to 30 frames per second and 31 to 60 frames per second. Here, the wavelet filter is the same as that used in the spatial frequency band decomposition unit 11. Furthermore, when wavelet packet decomposition is performed without decimation, the number of elements in the time direction in each frequency band is 60. The number of elements in the time and space directions in each frequency band is 8K x 4K x 60.

[0026] The degeneration processing unit 13 performs degeneration processing on the spatio-temporal frequency band components generated by the time-frequency band decomposition unit 12 to generate spatio-temporal frequency band components (hereinafter referred to as "degenerate frequency band components") in which the components are degenerated. Then, the degeneration processing unit 13 outputs the degenerate frequency band components to the frequency reconstruction unit 14.

[0027] The processing of the degeneration processing unit 13 will be described with reference to Fig. 4. Fig. 4 is a flowchart showing an example of processing by the degeneration processing unit 13.

[0028] In step S11, it is determined whether or not the spatial resolution of the reduced image is exceeded for each spatiotemporal frequency band.

[0029] In step S12, components in the spatiotemporal frequency band exceeding the spatial resolution of the reduced image are multiplied by a first degeneracy coefficient having the highest degeneracy rate. The value of the first degeneracy coefficient is 0 or more and less than 1.

[0030] In step S13, it is determined whether each spatiotemporal frequency band is the lowest spatiotemporal frequency band.

[0031] In step S14, the components in the lowest spatiotemporal frequency band that include a DC component are not subjected to degeneration processing (that is, the values ​​of the elements are not changed).

[0032] In step S15, edge components are detected within a spatiotemporal frequency band equal to or less than the spatial resolution of the reduced image. For example, the degeneration processing unit 13 detects, as edge components, the same element positions that have values ​​greater than a predetermined threshold in all frequency bands within the spatiotemporal frequency band equal to or less than the spatial resolution of the reduced image. By using wavelet packet decomposition without decimation as the frequency band decomposition by the frequency decomposition unit 10, the number of elements in each frequency band becomes the same, thereby improving the accuracy of edge component detection.

[0033] In step S16, since there is a high possibility that aliasing distortion is large at element positions that have large power in all frequency bands, such as strong edge components, the component at that element position in the spatiotemporal high frequency band is multiplied by a second degeneracy coefficient whose degeneracy rate is lower than the first degeneracy coefficient. The value of the second degeneracy coefficient is greater than the first degeneracy coefficient but less than 1.

[0034] In step S17, it is determined whether or not a spatiotemporal frequency band equal to or lower than the spatial resolution of the reduced image is a high frequency band.

[0035] In step S18, because components in the high frequency band in the spatiotemporal frequency band equal to or lower than the spatial resolution of the reduced image may be difficult to encode, they are multiplied by a third degeneracy coefficient whose degeneracy rate is lower than that of the second degeneracy coefficient. The value of the third degeneracy coefficient is greater than that of the second degeneracy coefficient and less than 1.

[0036] 5 is a diagram showing an example of coefficients set by the degeneration processing unit 13. Here, the frequency band in the time direction is omitted. Also, the spatial resolution of the original image is 8K×4K, and the spatial resolution of the reduced image is 4K×2K.

[0037] When the first degeneration coefficient is 0.001, spatiotemporal frequency band components with horizontal frequencies of 4K to 8K and vertical frequencies of 2K to 4K are multiplied by 0.001 (power attenuated by 30dB). No degeneration processing is performed on the lowest spatiotemporal frequency band components. When the second degeneration coefficient is 0.1, after an edge component is detected, the component at that element position is multiplied by 0.1 (power attenuated by 10dB). When the third degeneration coefficient is 0.5, for example, spatiotemporal frequency band components with horizontal frequencies of 3K to 4K, vertical frequencies of 1.5K to 2K, and temporal frequencies of 30 to 60 are multiplied by 0.5 (power attenuated by 3dB).

[0038] The values ​​of the first degeneration coefficient, the second degeneration coefficient, the third degeneration coefficient, and the frequency band to which the third degeneration coefficient is applied can be set arbitrarily by the user and are not limited to the above example. Also, only one or two of the first degeneration coefficient, the second degeneration coefficient, the third degeneration coefficient, and the third degeneration coefficient may be applied.

[0039] The frequency reconstruction unit 14 performs frequency reconstruction (e.g., wavelet packet inverse decomposition) on the degenerated frequency band components generated by the degeneration processing unit 13, and generates a degenerated image of the same size as the original image. The frequency reconstruction is performed in the time direction and the space direction when the frequency decomposition unit 10 includes the time-frequency band decomposition unit 12, and is performed in the space direction when the frequency decomposition unit 10 does not include the time-frequency band decomposition unit 12. The frequency reconstruction unit 14 then outputs the degenerated image to the spatial resolution reduction unit 15.

[0040] The spatial resolution reduction unit 15 generates a reduced image by reducing the spatial resolution of the reduced image generated by the frequency reconstruction unit 14. The spatial resolution reduction unit 15 generates the reduced image by, for example, thinning out the pixels of the reduced image. The reduction ratio can be set arbitrarily by the user.

[0041] As described above, the reduction device 1 decomposes an original image into frequency band components and performs band limitation as necessary. This makes it possible to generate a high-quality reduced image. For example, when an original image captured at 8K resolution is reduced to 4K resolution for broadcast, by reducing the image using the reduction device 1, it becomes possible to encode the reduced image with high quality.

[0042] (Second embodiment) Next, a reduction device according to a second embodiment will be described. Fig. 6 is a block diagram showing an example of the configuration of a reduction device 2 according to the second embodiment. The reduction device 2 shown in Fig. 6 includes a frequency decomposition unit 10, a reduction processing unit 13, a reduction correction unit 16, a frequency reconstruction unit 14, and a spatial resolution reduction unit 15. The reduction device 2 according to the second embodiment differs from the reduction device 1 according to the first embodiment in that it further includes a reduction correction unit 16. As the other configurations are the same as those of the first embodiment, the same reference numerals are used and their description will be omitted.

[0043] The degeneration correction unit 16 receives the degenerate frequency band components from the degeneration processing unit 13. The degeneration correction unit 16 calculates the average value for each band of the degenerate frequency band components and corrects the degenerate frequency band components corresponding to the reduced image so that the distribution of the average values ​​for each band of the degenerate frequency band components corresponding to the reduced image approaches the distribution of the average values ​​for each band of the degenerate frequency band components corresponding to the original image. The degeneration correction unit 16 outputs the corrected degenerate frequency band components to the frequency reconstruction unit 14.

[0044] The processing of the degeneration correction unit 16 will be described with reference to FIG. 7. FIG. 7 is a diagram illustrating an example of processing by the degeneration correction unit 16. As in FIG. 5, the spatial resolution of the original image is 8K×4K, and the spatial resolution of the reduced image is 4K×2K. The degeneration correction unit 16 multiplies each element value of the degenerate frequency band component indicated by a by a constant so that the average value of the degenerate frequency band component indicated by a becomes equal to the average value of the four degenerate frequency band components indicated by A. Similarly, the degenerate correction unit 16 multiplies each element value of the degenerate frequency band components indicated by b to h by a constant so that the average values ​​of the degenerate frequency band components indicated by b to h become equal to the average values ​​of the degenerate frequency band components indicated by B to H, respectively. Note that the processing of the degeneration correction unit 16 is not limited to this, and any processing may be performed so that the distribution of the average values ​​for each band of the degenerate frequency band components corresponding to the reduced image approaches the distribution of the average values ​​for each band of the degenerate frequency band components corresponding to the original image.

[0045] As explained above, the reduction device 2 corrects the degenerated frequency band components. Therefore, it is possible to generate a reduced image of higher quality than the reduction device 1.

[0046] (program) A computer capable of executing program instructions can be used to function as the above-described reduction devices 1 and 2. Here, the computer may be a general-purpose computer, a special-purpose computer, a workstation, a PC (Personal Computer), an electronic notepad, etc. The program instructions may be program code, code segments, etc. for performing the necessary tasks.

[0047] The computer includes a processor, a storage unit, an input unit, an output unit, and a communication interface. The processor may be a CPU (Central Processing Unit), an MPU (Micro Processing Unit), a GPU (Graphics Processing Unit), a DSP (Digital Signal Processor), or an SoC (System on a Chip), and may be configured with multiple processors of the same or different types. The processor reads and executes programs from the storage unit to control the above components and perform various arithmetic processing. Note that at least a portion of these processing contents may be implemented by hardware. The input unit is an input interface that accepts user input operations and acquires information based on the user operations, such as a pointing device, keyboard, or mouse. The output unit is an output interface that outputs information, such as a display or speaker. The communication interface is an interface for communicating with external devices, such as a LAN (Local Area Network) interface.

[0048] The program may be recorded on a computer-readable recording medium. Using such a recording medium, the program can be installed on a computer. Here, the recording medium on which the program is recorded may be a non-transitory recording medium. The non-transitory recording medium is not particularly limited, and may be, for example, a CD-ROM, a DVD-ROM, or a USB (Universal Serial Bus) memory. Furthermore, the program may be downloaded from an external device via a network.

[0049] For example, a program for functioning as the reduction device 1 causes a computer to execute the following steps: performing frequency band decomposition in the spatial direction on the original image until each frequency band becomes a frequency band equal to or lower than the spatial resolution of the reduced image, thereby generating frequency band components; performing degeneration processing on the frequency band components, thereby generating degenerated frequency band components in which the components within each frequency band are degenerated; performing frequency reconstruction on the degenerated frequency band components, thereby generating a degenerated image of the same size as the original image; and generating a reduced image in which the spatial resolution of the degenerated image is reduced.

[0050] Furthermore, the reduction devices 1 and 2 may be configured with one or more semiconductor chips, and the semiconductor chip may be equipped with a CPU that executes a program that describes the processing content that realizes each function of the reduction devices 1 and 2.

[0051] Although the above-described embodiments have been described as typical examples, it will be apparent to those skilled in the art that many modifications and substitutions can be made within the spirit and scope of the present invention. Therefore, the present invention should not be construed as being limited by the above-described embodiments, and various modifications or alterations can be made without departing from the scope of the claims. For example, it is possible to integrate multiple building blocks shown in the block diagrams of the embodiments, or to divide one building block. [Explanation of symbols]

[0052] 1,2 Reduction device 10 Frequency decomposition section 11 Spatial frequency band decomposition section 12 Time-frequency band decomposition section 13 Degeneration processing unit 14 Frequency reconstruction unit 15 Spatial resolution reduction section 16 Degeneracy correction unit

Claims

1. A reduction device for generating a reduced image of an original image, comprising: a frequency decomposition unit that performs frequency band decomposition in the spatial direction on the original image until each frequency band becomes equal to or less than the spatial resolution of the reduced image, thereby generating frequency band components; a degeneration processing unit that performs a degeneration process on the frequency band components to generate degenerated frequency band components; a frequency reconstruction unit that performs frequency reconstruction on the degenerated frequency band components to generate a degenerated image of the same size as the original image; a spatial resolution reduction unit for generating a reduced image by reducing the spatial resolution of the degenerated image; A reduction device comprising:

2. The reduction device according to claim 1 , wherein the degeneration processing unit generates the degenerated frequency band components by multiplying components in a frequency band exceeding a spatial resolution of the reduced image by a first degeneration coefficient.

3. The reduction device according to claim 1 , wherein the degeneration processing unit detects edge components within a frequency band equal to or lower than a spatial resolution of the reduced image, and multiplies the edge components by a second degeneration coefficient to generate the degenerated frequency band components.

4. The reduction device according to claim 1 , wherein the degeneration processing unit generates the degenerated frequency band components by multiplying high-frequency band components in a frequency band equal to or lower than a spatial resolution of the reduced image by a third degeneration coefficient.

5. The degeneration processing unit multiplying a component in a frequency band exceeding the spatial resolution of the reduced image by a first degeneracy coefficient; detecting an edge component within a frequency band equal to or lower than a spatial resolution of the reduced image, and multiplying the edge component by a second degeneration coefficient having a degeneration rate lower than the first degeneration coefficient; 2. The reduction device according to claim 1, wherein the reduced frequency band components are generated by multiplying high frequency band components in a frequency band equal to or lower than the spatial resolution of the reduced image by a third degeneration coefficient having a degeneration rate lower than the second degeneration coefficient.

6. The reduction device according to claim 1 , wherein the frequency decomposition unit also performs frequency band decomposition on the original image in a time direction to generate the frequency band components.

7. 7. The reduction device according to claim 1, further comprising a degeneration correction unit that calculates an average value for each band of the degenerate frequency band components and corrects the degenerate frequency band components corresponding to the reduced image so that a distribution of the average values ​​for each band of the degenerate frequency band components corresponding to the reduced image approaches a distribution of the average values ​​for each band of the degenerate frequency band components corresponding to the original image.

8. A program for causing a computer to function as the reduction device according to any one of claims 1 to 7.

Images