Image conversion apparatus, image conversion method, and program
Patent Information
- Application Number
- JP2023010667
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-01-27
- Publication Date
- 2025-12-10
AI Technical Summary
Existing machine learning models for image super-resolution are limited to a fixed magnification and cannot flexibly handle different desired resolutions, restricting the ability to increase image resolution beyond the predefined setting.
An image conversion device that combines machine learning models with pixel interpolation, allowing flexible resolution magnification by using a machine learning model for high-resolution processing when magnifications match and performing interpolation when they differ, thereby generating high-resolution images at desired settings.
Enables flexible handling of resolution magnification, reduces memory requirements for storing multiple models, and improves image quality by prioritizing high-resolution processing with machine learning followed by interpolation, ensuring high-quality output images.
Smart Images
Figure 00000000_0000_ABST
Abstract
Description
[Technical field]
[0001] The present disclosure relates to an image conversion device, an image conversion method, and a program. [Background technology]
[0002] As described in Patent Document 1, a super-resolution technique is known that uses a machine learning model to increase the resolution of an image. [Prior art documents] [Patent documents]
[0003] [Non-Patent Document 1] Yulun Zhang et al., “Image Super-Resolution Using Very Deep Residual Channel Attention Networks”, [online], July 12, 2018,<URL: https: / / arxiv.org / abs / 1807.02758> Summary of the Invention [Problem to be solved by the invention]
[0004] A machine learning model for converting a low-resolution image into a high-resolution image performs image resolution processing at a magnification set during machine learning. For this reason, it is not possible to perform resolution processing at a magnification other than that set in the machine learning model that was generated in advance. Therefore, there has been a demand for technology that can flexibly accommodate the specification of the magnification for resolution processing. [Means for solving the problem]
[0005] The present disclosure can be realized in the following forms.
[0006] According to a first aspect of the present disclosure, there is provided an image conversion device for increasing the resolution of an image. The image conversion device includes a storage unit for storing at least one machine learning model generated by performing machine learning using learning data, the machine learning model increasing the resolution of an image by a predetermined set magnification, a reception unit for receiving a designation of a target magnification for the resolution increase process, a first processing unit for executing a resolution increase process at the set magnification on an image to be processed using the machine learning model, and a second processing unit for executing a pixel interpolation process on the image to be processed without using the machine learning model, in which, when the set magnification and the target magnification are different, the first processing unit executes the resolution increase process at the set magnification on the image to be processed, and the second processing unit executes the interpolation process according to the target magnification and the set magnification on the image to be processed before or after the resolution increase process by the first processing unit, and when the set magnification and the target magnification are the same, the first processing unit executes the resolution increase process at the set magnification on the image to be processed, and the second processing unit does not execute the interpolation process.
[0007] According to a second aspect of the present disclosure, there is provided an image conversion method for increasing the resolution of an image, the image conversion method including the steps of: preparing at least one machine learning model generated by performing machine learning using learning data, the machine learning model increasing the resolution of an image at a predetermined set magnification; receiving a designation of a target magnification for a process of increasing the resolution; performing the process of increasing the resolution at the set magnification using the machine learning model on an image to be processed; and, when the set magnification and the target magnification are different, performing pixel interpolation processing on the image to be processed according to the target magnification and the set magnification without using the machine learning model before or after the process of increasing the resolution; and, when the set magnification and the target magnification are the same, not performing the interpolation processing on the image to be processed before or after the process of increasing the resolution.
[0008] According to a third aspect of the present disclosure, there is provided a program for causing a computer to execute an image conversion process for increasing the resolution of an image. The program causes the computer to realize the following functions: reading at least one machine learning model generated by performing machine learning using learning data, the machine learning model for increasing the resolution of an image at a predetermined set magnification from a storage unit that stores the machine learning model; receiving a designation of a target magnification for the high-resolution process; executing the high-resolution process at the set magnification using the machine learning model on an image to be processed; executing pixel interpolation processing according to the target magnification and the set magnification without using the machine learning model before or after the high-resolution process on the image to be processed when the set magnification and the target magnification are different; and not executing the interpolation processing on the image to be processed before or after the high-resolution process when the set magnification and the target magnification are the same. [Brief description of the drawings]
[0009] [Figure 1] 1 is a block diagram showing a schematic configuration of an image conversion device according to a first embodiment. [Diagram 2] 11 is a flowchart showing an image conversion process. [Diagram 3] FIG. 11 is a block diagram relating to a memory of an image conversion device according to a second embodiment. [Figure 4] 10 is a flowchart showing image conversion processing in the second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0010] A. First embodiment: FIG. 1 is a block diagram showing a schematic configuration of an image conversion device 100 according to this embodiment. The image conversion device 100 generates an image by performing high-resolution processing on an input image at a specified magnification. High-resolution processing is processing for generating an image having a higher resolution than the resolution of the input image from the input image. Resolution means the density or number of pixels constituting an image. In high-resolution processing, for example, the pixel value of a pixel to be interpolated is calculated using the values of surrounding pixels to improve image quality. In this specification, an image means a digitized image and does not include an analog image.
[0011] The image conversion device 100 is a computer including a memory 110, an interface unit 120, an input device 130, a display device 140, and a processor 150. The memory 110 is also called a storage unit.
[0012] The memory 110 stores various programs and data used for various processes executed by the image conversion device 100. The memory 110 stores data representing a first learning model 11 and a second learning model 12, which are machine learning models. The first learning model 11 and the second learning model 12 are, for example, Convolutional Neural Networks (CNN). The first learning model 11 and the second learning model 12 are each generated by machine learning using learning data.
[0013] The first learning model 11 and the second learning model 12 convert a low-resolution image into a high-resolution image at a preset set magnification. Hereinafter, a low-resolution image means an image having a resolution lower than the resolution of the image before conversion. The set magnification for the first learning model 11 is 2 times. The set magnification for the first learning model 11 is not limited to this, and may be an integer of 2 or more. The set magnification for the second learning model 12 is 3 times. The set magnification for the second learning model 12 is not limited to this, and may be an integer of 2 or more. However, the set magnification for the second learning model 12 is different from the set magnification for the first learning model 11. Since two first learning models 11 and second learning models 12 having different set magnifications are prepared, it is possible to select a machine learning model according to a target magnification. The set magnification for the first learning model 11 is also referred to as magnification L. The set magnification for the second learning model 12 is also referred to as magnification M.
[0014] An input device 130 and a display device 140 are connected to the interface unit 120. The input device 130 is, for example, a keyboard or a mouse. The display device 140 is, for example, a liquid crystal display or an organic EL (Electro Luminescence) display.
[0015] The processor 150 realizes various functions by executing the programs stored in the memory 110. The processor 150 executes the programs stored in the memory 110 to function as a reception unit 210, a first processing unit 220, and a second processing unit 230.
[0016] The reception unit 210 receives a designation of a target magnification for the high-resolution processing. The first processing unit 220 performs high-resolution processing on the image to be processed using the first learning model 11 or the second learning model 12. The second processing unit 230 performs pixel interpolation processing on the image to be processed without using a machine learning model. In this specification, the pixel interpolation processing includes a process of interpolating pixels to increase the resolution of the image and a process of thinning out the pixels of the image to decrease the resolution of the image. The pixel interpolation processing used by the second processing unit 230 is, for example, a bilinear method, a bicubic method, or a Lanczos method.
[0017] Fig. 2 is a flowchart showing the image conversion process. The process shown in Fig. 2 is executed by the processor 150 functioning as the reception unit 210, the first processing unit 220, and the second processing unit 230. It is assumed that data representing the first learning model 11 and the second learning model 12 are stored in the memory 110 before the process in Fig. 2 is executed. For example, when the user instructs the start of the process via the input device 130, the processor 150 starts the process shown in Fig. 2.
[0018] In step S100, the processor 150, for example, displays an input screen on the display device 140 and receives an input image and a specified magnification. The user inputs a path indicating an area in the memory 110 where the input image IM is stored, and the specified magnification using the input device 130. The specified magnification is a magnification for the high resolution desired by the user. The specified magnification is also called a target magnification. Note that before the process of FIG. 2 starts, the user stores the input image IM in a desired area of the memory 110.
[0019] In step S105, processor 150 determines whether the specified magnification is the same as the set magnification of first learning model 11 or the set magnification of second learning model 12. If the specified magnification is the same as the set magnification of first learning model 11 or the set magnification of second learning model 12 (step S105; YES), processor 150 executes processing of step S110. On the other hand, if the specified magnification is different from either set magnification (step S105; NO), processor 150 executes processing of step S115.
[0020] In step S110, the processor 150 performs a high-resolution process on the input image to be processed, using one of the first learning model 11 and the second learning model 12, a machine learning model having the same set magnification as the specified magnification. For example, when the specified magnification is 2x, the processor 150 performs a high-resolution process on the input image IM using the first learning model 11. For example, when the resolution of the input image IM is 300 (dpi) x 300 (dpi), the high-resolution process generates an image having a resolution of 600 (dpi) x 600 (dpi), which is twice the number of pixels per inch.
[0021] In step S115, the processor 150 selects a machine learning model that satisfies a predetermined specified condition from among the first learning model 11 and the second learning model 12, and executes a process of increasing the resolution of the input image to be processed using the selected machine learning model. The predetermined specified condition is that the machine learning model has a set magnification closest to the specified magnification. The processor 150 selects, among the machine learning models stored in the memory 110, a machine learning model that has the smallest difference between the set magnification and the specified magnification as the machine learning model that satisfies the predetermined specified condition. When there are two machine learning models that satisfy the specified condition, the machine learning model that has a set magnification greater than the specified magnification is selected. When there are two machine learning models that satisfy the specified condition, the specified magnification is an intermediate value between the two set magnifications. The image quality of the generated image tends to be improved by using a process of increasing the resolution using a machine learning model rather than by using an interpolation process that does not use a machine learning model. For this reason, in this embodiment, the image conversion device 100 generates an intermediate image that is as close to the specified magnification as possible by a process of increasing the resolution using a machine learning model that has a set magnification closest to the specified magnification.
[0022] For example, when the specified magnification is 2.5 times, the processor 150 selects the second learning model 12 with a set magnification of 3 times. The processor 150 executes a high-resolution process on the input image IM using the second learning model 12. For example, when the resolution of the input image IM is 300 (dpi) x 300 (dpi), an intermediate image having a resolution of 900 (dpi) x 900 (dpi), which is three times the number of pixels per inch, is generated by the high-resolution process.
[0023] Also, for example, assume that three machine learning models are prepared, and their set magnifications are 2x, 4x, and 8x. Assume that the specified magnification is 3.8x. In this case, the machine learning model with the set magnification of 4x, which is closest to the specified magnification, is the machine learning model that satisfies the specified condition.
[0024] In step S120, the processor 150 executes pixel interpolation processing on the image to be processed. The processing target is an intermediate image generated by the high-resolution processing using the machine learning model in step S115.
[0025] In step S115, if the resolution increasing process is performed at a set magnification rate larger than the designated magnification rate, in step S120, downsampling process is performed as an interpolation process to reduce the resolution. For example, assume that the resolution of the input image IM is 300 (dpi) x 300 (dpi) and the designated magnification rate is 2.5 times. In step S115, if the resolution of the input image IM is converted to 3 times, an intermediate image having a resolution of 900 (dpi) x 900 (dpi) is generated. The resolution of the output image to be finally output is 750 (dpi) x 750 (dpi). In this case, the processor 150 performs pixel interpolation process on the intermediate image to reduce the resolution by 0.83 times the number of pixels per inch. By the interpolation process, an image having a resolution of 750 (dpi) x 750 (dpi) is generated from the intermediate image having a resolution of 900 (dpi) x 900 (dpi).
[0026] Furthermore, if the process of increasing the resolution is executed at a set magnification smaller than the designated magnification in step S115, then in step S120, upsampling, which is an interpolation process for increasing the resolution, is executed.
[0027] In step S125, processor 150 stores the converted image as an output image in memory 110. Then, the process shown in FIG.
[0028] As described above, in this embodiment, the image conversion device 100 generates an output image with a resolution increased at a specified magnification by combining a process of increasing resolution using a machine learning model and a process of interpolating pixels of an image without using a machine learning model. The image conversion device 100 can increase resolution at a desired magnification even if a machine learning model having the same set magnification as the specified magnification is not available. Therefore, a user can obtain an image with a resolution increased at a desired magnification even if a machine learning model having the same set magnification as the specified magnification is not available. In this way, it is possible to flexibly respond to the specification of the magnification for increasing resolution.
[0029] In addition, a user may wish to perform resolution enhancement at a designated magnification of a decimal value. Even if the value of the set magnification of the prepared machine learning model is an integer, the image conversion device 100 can perform resolution enhancement at a designated magnification of a decimal value by combining a resolution enhancement process using a machine learning model and an image pixel interpolation process not using a machine learning model. In this way, it is possible to flexibly respond to the designation of the resolution enhancement magnification.
[0030] Also, it is not necessary to prepare machine learning models for all set magnifications that the user may specify. Therefore, the capacity of memory 110 for storing the machine learning models can be reduced compared to an embodiment in which a machine learning model is prepared for each of a plurality of set magnifications. Also, compared to an embodiment in which a machine learning model is prepared for each of a plurality of set magnifications, the effort required for performing machine learning to generate a machine learning model can be saved.
[0031] In addition, in this embodiment, the process of increasing the resolution using a machine learning model is performed first, and then the interpolation process without using a machine learning model is performed. The interpolation process without using a machine learning model tends to generate images with lower image quality than the process of increasing the resolution using a machine learning model. When the interpolation process without using a machine learning model is performed on an input image with a low resolution, it is expected that the image quality of the generated image will be lower than that of the input image. For this reason, the image conversion device 100 first generates an intermediate image with a higher image quality than the image quality of the input image by first performing the process of increasing the resolution using a machine learning model, which tends to increase the accuracy of the improvement of the image quality that can be generated. Then, the image conversion device 100 performs the interpolation process without using a machine learning model. Compared to the mode in which the process of increasing the resolution using a machine learning model is performed after the interpolation process without using a machine learning model is performed, the image quality of the finally generated image can be improved.
[0032] In addition, in this embodiment, when the specified magnification and the set magnification of the machine learning model prepared in advance are not the same, the resolution enhancement process is performed using a machine learning model having a set magnification closest to the specified magnification. This is because the image quality of the generated image tends to be improved by using a resolution enhancement process using a machine learning model rather than by using an interpolation process that does not use a machine learning model. For this reason, in this embodiment, the image conversion device 100 selects a machine learning model having a set magnification closest to the specified magnification, and generates an intermediate image as close to the specified magnification as possible by a resolution enhancement process using the machine learning model.
[0033] B. Second embodiment: In the first embodiment, the method of selecting the machine learning model used for the high resolution processing was uniform. In the second embodiment, a machine learning model that meets the conditions desired by the user is used. The following will mainly describe the configuration different from the first embodiment. Descriptions of the same configuration as the first embodiment will be omitted.
[0034] FIG. 3 is a block diagram of the memory 110 of the image conversion device 100 according to this embodiment. In this embodiment, the image conversion device 100 executes a process of increasing the resolution of an image to be processed using any one of the first learning model 11, the second learning model 12, the third learning model 13, and the fourth learning model 14. The memory 110 stores data representing the first learning model 11 to the fourth learning model 14 that have been generated in advance by machine learning. The setting magnifications of the first learning model 11 to the fourth learning model 14 are all powers of 2 (values equal to or greater than the first power of 2), but are different from each other. The setting magnification of the first learning model 11 is 2 times. The setting magnification of the second learning model 12 is 4 times. The setting magnification of the third learning model 13 is 8 times. The setting magnification of the fourth learning model 14 is 16 times.
[0035] FIG. 4 is a flowchart showing the image conversion process in this embodiment. The process shown in FIG. 4 is executed by the processor 150 functioning as the reception unit 210, the first processing unit 220, and the second processing unit 230. It is assumed that the first learning model 11 to the fourth learning model 14 are stored in the memory 110 before the process in FIG. 4 is executed. In FIG. 4, the same processes as those in the first embodiment are denoted by the same reference numerals. For example, when the user instructs the start of the process via the input device 130, the processor 150 starts the process shown in FIG. 4.
[0036] The process of step S100 is the same as that of the first embodiment. In this embodiment, for ease of understanding of the technology, it is assumed that the specified magnification is greater than 2 times, which is the set magnification of the first learning model 11, and less than 16 times, which is the set magnification of the fourth learning model 14. The process of step S105 is the same as that of the first embodiment. The process of step S110 is the same as that of the first embodiment.
[0037] In step S200, processor 150 accepts designation of which of image quality improvement and processing speed improvement should be prioritized, for example, by displaying an input screen on display device 140. The user inputs the designation of which should be prioritized using input device 130.
[0038] In step S205, processor 150 determines whether or not it has been specified that improvement in image quality should be prioritized. If it has been specified that improvement in image quality should be prioritized (step S205; YES), processor 150 executes the process of step S210. On the other hand, if it has not been specified that improvement in image quality should be prioritized, that is, if it has been specified that improvement in processing speed should be prioritized (step S205; NO), processor 150 executes the process of step S220.
[0039] In step S210, processor 150 selects a machine learning model that satisfies a predetermined first specified condition from among the multiple machine learning models stored in memory 110. Processor 150 executes a high-resolution process on the input image to be processed using the selected machine learning model. The predetermined first specified condition is that the set magnification is greater than the specified magnification. When there are multiple machine learning models that satisfy the first specified condition, processor 150 selects from among them the machine learning model that has a set magnification closest to the specified magnification.
[0040] In step S215, processor 150 performs pixel interpolation processing, i.e., downsampling processing, on the image to be processed in order to reduce the resolution. This is because in step S210, the process of increasing the resolution is performed at a set magnification ratio that is larger than the specified magnification ratio.
[0041] In step S220, processor 150 selects a machine learning model that satisfies a predetermined second specified condition from among the multiple machine learning models stored in memory 110. Processor 150 executes a high-resolution process at a set magnification on the input image to be processed using the selected machine learning model. The predetermined second specified condition is that the set magnification is smaller than the specified magnification. When there are multiple machine learning models that satisfy the second specified condition, processor 150 selects from among them the machine learning model that has a set magnification closest to the specified magnification.
[0042] In a mode in which a machine learning model having a set magnification smaller than the specified magnification is used, the number of pixels to be interpolated by the high resolution processing can be reduced compared to a mode in which a machine learning model having a set magnification larger than the specified magnification is used in order to prioritize image quality improvement. As a result, the processing speed in S220 and S225 can be improved compared to S210 and S215 in which image quality improvement is prioritized.
[0043] For example, when the specified magnification is 2.5 times, the processor 150 performs a high-resolution process on the input image IM using the first learning model 11 with a set magnification of 2 times. For example, when the resolution of the input image IM is 300 (dpi) x 300 (dpi), an intermediate image having a resolution of 600 (dpi) x 600 (dpi), which is twice the number of pixels per inch, is generated.
[0044] In step S225, the processor 150 performs an interpolation process to increase the resolution of the image to be processed, that is, an upsampling process. This is because in step S220, the resolution increase process is performed at a set magnification smaller than the specified magnification. For example, assume that the resolution of the input image IM is 300 (dpi) x 300 (dpi) and the specified magnification is 2.5 times. When the resolution of the input image is converted to 2 times in step S220, an intermediate image having a resolution of 600 (dpi) x 600 (dpi) is generated. The resolution of the output image to be finally output is 750 (dpi) x 750 (dpi). In this case, the processor 150 performs a pixel interpolation process on the intermediate image to increase the resolution by 1.25 times the number of pixels per inch. Through the interpolation process, an image having a resolution of 750 (dpi) x 750 (dpi) is generated from the intermediate image having a resolution of 600 (dpi) x 600 (dpi).
[0045] In step S125, processor 150 stores the converted image as an output image in memory 110. Then, the process shown in FIG.
[0046] As described above, similarly to the first embodiment, the image conversion device 100 can generate an image with a desired resolution by combining a resolution enhancement process using a machine learning model and an image pixel interpolation process without using a machine learning model. In this way, it is possible to flexibly respond to the specification of the resolution enhancement magnification. In addition, in this embodiment, it is possible to execute a resolution enhancement process according to which of the improvement in image quality and the improvement in processing speed the user desires.
[0047] In addition, in this embodiment, whether prioritizing image quality improvement or processing speed improvement, the high-resolution processing using a machine learning model is performed first, and the interpolation processing without using a machine learning model is performed. The image conversion device 100 first generates an intermediate image with a higher image quality than the image quality of the input image by first performing the high-resolution processing using a machine learning model that tends to increase the accuracy of the improvement of the generated image quality. Then, the image conversion device 100 performs the interpolation processing without using a machine learning model. In this way, the image quality of the finally generated image can be improved compared to the mode in which the high-resolution processing using a machine learning model is performed after the interpolation processing without using a machine learning model is performed.
[0048] C. Other embodiments: C1. Alternative embodiment 1: The interpolation process may be performed after performing the high-resolution process using the machine learning model two or more times. For example, assume that the specified magnification is 7.5 times. In this case, a first intermediate image is generated by performing the high-resolution process on the input image using a machine learning model with a set magnification of 2 times. Then, a second intermediate image is generated by performing the high-resolution process on the first intermediate image using a machine learning model with a set magnification of 3 times. Furthermore, a pixel interpolation process that increases the number of pixels per inch by 1.25 times is performed on the second intermediate image to generate a final output image. In this way, a high-resolution image with the resolution of the input image increased by the specified magnification of 8 times can be generated.
[0049] C2. Alternative embodiment 2: In the first and second embodiments, an example has been described in which the interpolation process is performed after the high-resolution process using the machine learning model. However, the interpolation process may be performed first, and the high-resolution process using the machine learning model may be performed at least once on the image that has been subjected to the interpolation process.
[0050] C3. Alternative embodiment 3: In the first and second embodiments, examples have been described in which two or more machine learning models are prepared in advance. However, only one machine learning model may be prepared in advance.
[0051] Also, three or more machine learning models may be prepared in advance. One of the three or more machine learning models with different set magnifications can be selected according to the target magnification. In this case, all the set magnifications can be set to odd numbers (odd numbers equal to or greater than 3). Alternatively, all the set magnifications may be set to even numbers. Alternatively, the set magnifications of at least some of the multiple machine learning models may be decimals rather than integers.
[0052] Furthermore, the means for realizing the functions of the image conversion device 100 is not limited to software, and may be realized in part or in whole by dedicated hardware. For example, the dedicated hardware may be a circuit such as a field programmable gate array (FPGA) or an application specific integrated circuit (ASIC).
[0053] D. Other forms: The present disclosure is not limited to the above-mentioned embodiment, and can be realized in various configurations without departing from the spirit of the present disclosure. For example, the technical features in the embodiments corresponding to the technical features in each aspect described in the Summary of the Invention column can be appropriately replaced or combined to solve some or all of the above-mentioned problems or to achieve some or all of the above-mentioned effects. Furthermore, if the technical feature is not described as essential in this specification, it can be appropriately deleted.
[0054] (1) According to a first aspect of the present disclosure, there is provided an image conversion device for increasing the resolution of an image, the image conversion device including: a storage unit for storing at least one machine learning model generated by performing machine learning using learning data, the machine learning model increasing the resolution of an image by a predetermined set magnification; a reception unit for receiving a designation of a target magnification for the resolution increase process; a first processing unit for executing a resolution increase process at the set magnification on an image to be processed using the machine learning model; and a second processing unit for executing a pixel interpolation process on the image to be processed without using the machine learning model, in which, when the set magnification and the target magnification are different, the first processing unit executes the resolution increase process at the set magnification on the image to be processed, and the second processing unit executes the interpolation process according to the target magnification and the set magnification on the image to be processed before or after the resolution increase process by the first processing unit; and, when the set magnification and the target magnification are the same, the first processing unit executes the resolution increase process at the set magnification on the image to be processed, and the second processing unit does not execute the interpolation process. According to the above embodiment, a high-resolution process using a machine learning model is combined with an interpolation process of pixels of an image that does not use a machine learning model, so that the magnification of the high-resolution process can be flexibly specified.
[0055] (2) In the above embodiment, when the set magnification is greater than the target magnification, the second processing unit may, after the first processing unit has performed the high-resolution processing on the image to be processed, perform the interpolation processing on the image that has been subjected to the high-resolution processing in accordance with the target magnification and the set magnification to reduce the resolution. In a mode in which pixel interpolation processing is performed first and then resolution enhancement processing is performed, the image quality of the image obtained by performing pixel interpolation processing on a low-resolution image tends to decrease. In this case, it is expected that the image quality of the image that is the target of resolution enhancement processing will be lower than the image quality of the image before pixel interpolation processing is performed. According to the above mode, it is possible to avoid the occurrence of such a problem.
[0056] (3) In the above embodiment, when the set magnification is smaller than the target magnification, the second processing unit may perform the interpolation process to increase the resolution of the image that has been subjected to the high-resolution process after the first processing unit has performed the high-resolution process at least once on the image to be processed. In a mode in which pixel interpolation processing is performed first and then resolution enhancement processing is performed, the image quality of the image obtained by performing pixel interpolation processing on a low-resolution image tends to decrease. In this case, it is expected that the image quality of the image that is the target of resolution enhancement processing will be lower than the image quality of the image before pixel interpolation processing is performed. According to the above mode, it is possible to avoid the occurrence of such a problem.
[0057] (4) In the above embodiment, the machine learning models stored in the memory unit may include a first learning model in which the set multiplier is a multiplier L (L is an integer greater than or equal to 2) and a second learning model in which the set multiplier is a multiplier M (M is an integer greater than or equal to 2 and is a value different from L). A machine learning model corresponding to a target magnification can be selected from among a plurality of machine learning models each having a different set magnification.
[0058] (5) In the above aspect, the reception unit may receive a designation of which of improving image quality and improving processing speed should be given priority. According to the above aspect, it is possible to execute a high resolution process that satisfies either the desired improvement in image quality or the desired improvement in processing speed.
[0059] (6) In the above embodiment, the magnification L and the magnification M may both be a power of two.
[0060] (7) Both the magnification L and the magnification M may be odd numbers.
[0061] (8) According to a second aspect of the present disclosure, there is provided an image conversion method for increasing the resolution of an image, the image conversion method including the steps of: preparing at least one machine learning model generated by performing machine learning using learning data, the machine learning model increasing the resolution of an image at a predetermined set magnification; receiving a designation of a target magnification for a high-resolution process; executing the high-resolution process at the set magnification using the machine learning model on an image to be processed; and, when the set magnification and the target magnification are different, executing pixel interpolation processing according to the target magnification and the target magnification without using the machine learning model before or after the high-resolution process on the image to be processed, when the set magnification and the target magnification are the same, not executing the interpolation processing on the image to be processed before or after the high-resolution process. According to the above embodiment, a high-resolution process using a machine learning model is combined with an interpolation process of pixels of an image that does not use a machine learning model, so that the magnification of the high-resolution process can be flexibly specified.
[0062] (9) According to a third aspect of the present disclosure, there is provided a program for causing a computer to execute an image conversion process for increasing the resolution of an image. The program causes the computer to realize the following functions: reading at least one machine learning model generated by performing machine learning using learning data, the machine learning model for increasing the resolution of an image at a predetermined set magnification, from a storage unit that stores the machine learning model; receiving a designation of a target magnification for the high-resolution process; executing the high-resolution process at the set magnification using the machine learning model on an image to be processed; executing pixel interpolation processing according to the target magnification and the set magnification without using the machine learning model before or after the high-resolution process on the image to be processed when the set magnification and the target magnification are different; and not executing the interpolation processing on the image to be processed before or after the high-resolution process when the set magnification and the target magnification are the same. According to the above embodiment, a high-resolution process using a machine learning model is combined with an interpolation process of pixels of an image that does not use a machine learning model, so that the magnification of the high-resolution process can be flexibly specified.
[0063] The present disclosure may be realized in various forms other than those described above. For example, the present disclosure may be realized in the form of a non-transitory storage medium having a computer program recorded thereon. [Explanation of symbols]
[0064] 11: first learning model, 12: second learning model, 100: image conversion device, 110: memory, 120: interface unit, 130: input device, 140: display device, 150: processor, 210: reception unit, 220: first processing unit, 230: second processing unit, IM: input image
Claims
1. An image conversion device for increasing the resolution of an image, comprising: a storage unit that stores at least one machine learning model generated by performing machine learning using learning data, the machine learning model increasing the resolution of an image at a predetermined setting magnification; a receiving unit that receives a designation of a target magnification for the resolution enhancement process; a first processing unit that uses the machine learning model to perform a resolution enhancement process on an image to be processed at the set magnification; a second processing unit that performs pixel interpolation processing on the image to be processed without using the machine learning model; Equipped with When the set magnification and the target magnification are different, the first processing unit performs the resolution enhancement process on the image to be processed at the set magnification; the second processing unit performs the interpolation processing according to the target magnification and the set magnification on the image to be processed before or after the resolution enhancement processing by the first processing unit, When the set magnification and the target magnification are the same, the first processing unit performs the resolution enhancement process on the image to be processed at the set magnification; The second processing unit does not perform the interpolation process. Image conversion device.
2. 2. The image conversion device according to claim 1, When the set magnification is greater than the target magnification, the second processing unit, after the first processing unit has performed the resolution enhancement process on the image to be processed, performs the interpolation process on the image that has been subjected to the resolution enhancement process in accordance with the target magnification and the set magnification for lowering the resolution; Image conversion device.
3. 3. The image conversion device according to claim 2, When the set magnification is smaller than the target magnification, the second processing unit performs the interpolation process to increase the resolution of the image that has been subjected to the resolution enhancement process after the first processing unit has performed the resolution enhancement process on the image to be processed at least once; Image conversion device.
4. 4. The image conversion device according to claim 3, The machine learning models stored in the storage unit include a first learning model in which the set magnification is a magnification L (L is an integer equal to or greater than 2) and a second learning model in which the set magnification is a magnification M (M is an integer equal to or greater than 2 and is a value different from L). Image conversion device.
5. 5. The image conversion device according to claim 4, the receiving unit receives a designation of which of improving image quality and improving processing speed should be prioritized; Image conversion device.
6. 6. The image conversion device according to claim 5, The magnification L and the magnification M are both powers of 2. Image conversion device.
7. 6. The image conversion device according to claim 5, The magnification L and the magnification M are both odd numbers. Image conversion device.
8. An image conversion method for increasing the resolution of an image by a computer equipped with at least one machine learning model generated by performing machine learning using training data, the machine learning model increasing the resolution of an image at a predetermined setting magnification, comprising: A step of receiving a designation of a target magnification for the high resolution processing; performing the resolution enhancement process on the image to be processed at the set magnification using the machine learning model; When the set magnification and the target magnification are different, before or after the high-resolution processing, performing pixel interpolation processing on the image to be processed according to the target magnification and the set magnification without using the machine learning model; Including, When the set magnification and the target magnification are the same, The interpolation process is not performed on the image to be processed before or after the resolution enhancement process. Image conversion methods.
9. A program for causing a computer to execute an image conversion process for increasing the resolution of an image, The computer, a function of reading at least one machine learning model generated by performing machine learning using learning data, the machine learning model increasing the resolution of an image at a predetermined setting magnification, from a storage unit that stores the machine learning model; a function of receiving a designation of a target magnification for the resolution enhancement process; a function of executing the resolution enhancement process at the set magnification using the machine learning model on the image to be processed; When the set magnification and the target magnification are different, a function of executing pixel interpolation processing on the image to be processed according to the target magnification and the set magnification without using the machine learning model before or after the resolution enhancement processing; When the set magnification and the target magnification are the same, a function of not performing the interpolation process on the image to be processed before or after the resolution enhancement process; A program to make this happen.