Image processing apparatus and control method, storage medium, and program product

Abstract

The invention provides an image processing apparatus, a control method, a storage medium, and a program product. The image processing apparatus includes: a first generation unit configured to generate first conversion information used in generating an image having a different dynamic range from a first image; an image resizing processing unit configured to perform resizing processing on the first image; and a control unit configured to control whether to perform a resizing process on the first conversion information added to the first image when the resizing process is performed on the first image.

Description

TECHNICAL FIELD

[0001] The present application relates to an image processing apparatus, a control method, a storage medium, and a program product, and particularly to a size adjustment process of an image. BACKGROUND

[0002] There is known practice of using conversion information called a gain map to mutually convert a high dynamic range (HDR) image and a standard dynamic range (SDR) image to be compatible with a dynamic range that a display apparatus can display (Japanese Patent Application Laid-Open No. 2018-530281). A gain map is generated from a RAW image (master image), and the gain map is stored in an image file. Japanese Patent Application Laid-Open No. 2007-180851 describes a method of reducing a processing load when a gain map is generated by reducing a RAW image.

[0003] In Japanese Patent Application Laid-Open No. 2007-180851, by adding a gain map to a master image and storing the master image to an image file, an HDR image can be converted to an SDR image and the SDR image can be displayed on a display apparatus compatible with SDR.

[0004] However, Japanese Patent Application Laid-Open No. 2007-180851 does not mention a process of a gain map when a master image to which a gain map is added is size-adjusted (enlarged or reduced). SUMMARY

[0005] According to a first aspect of the present disclosure, an image processing apparatus includes a first generation unit configured to generate first conversion information used when generating an image having a different dynamic range from a first image, an image size adjustment processing unit configured to perform a size adjustment process on the first image, and a control unit configured to control whether to perform a size adjustment process on the first conversion information added to the first image when performing the size adjustment process on the first image.

[0006] According to a second aspect of the present disclosure, a control method of an image processing apparatus includes generating conversion information used to generate an image having a different dynamic range from a first image, performing a size adjustment process on the first image, and controlling whether to perform a size adjustment process on the conversion information added to the first image when performing the size adjustment process on the first image.

[0007] According to a third aspect of the embodiments, the present application provides a computer-readable storage medium storing a program for causing a computer to function as the image processing apparatus described above.

[0008] According to a fourth aspect of the embodiments, the present application provides a computer program product for causing a computer to function as the image processing apparatus described above.

[0009] Further features of the present disclosure will become apparent from the following description of example embodiments (with reference to the accompanying drawings). BRIEF DESCRIPTION OF DRAWINGS

[0010] Figure 1 is a block diagram illustrating a hardware configuration of an imaging device to which an image processing device according to the present embodiment is applied.

[0011] Figure 2 is a block diagram illustrating a functional configuration of an image processing unit according to the present embodiment.

[0012] Figures 3A to 3D is a diagram describing a method of generating a gain map according to the present embodiment.

[0013] Figure 4A and Figure 4B is a diagram illustrating a region integration process of a gain map according to the present embodiment.

[0014] Figure 5A and Figure 5B is a diagram illustrating a region division process of a gain map according to the present embodiment.

[0015] Figure 6A and Figure 6B is a diagram illustrating a data configuration of a main image to which a gain map is added according to the present embodiment.

[0016] Figure 7 is a flowchart showing a control process at the time of imaging according to the first embodiment.

[0017] Figure 8 is a flowchart showing a control process at the time of image reduction according to the second embodiment.

[0018] Figure 9A and Figure 9B is a diagram describing a gain map generation process at the time of image enlargement according to the third embodiment. DETAILED DESCRIPTION

[0019] Hereinafter, the embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments are not intended to limit the scope of the present disclosure. A plurality of features are described in the embodiments, but the disclosure is not limited to require all such features, and a plurality of such features can be appropriately combined. Furthermore, in the drawings, the same or similar configurations are given the same reference signs, and redundant descriptions thereof are omitted.

[0020] In the present embodiment, an example of resizing a gain map in a case where the image processing apparatus of the present disclosure is applied to an imaging apparatus such as a digital camera, a gain map is generated from a RAW image captured by the digital camera, and the RAW image is resized (enlarged or reduced) will be described.

[0021] Note that the imaging apparatus according to the present embodiment is not limited to a digital camera, and can be a personal computer (laptop PC or tablet PC), a smart phone, a network camera such as a surveillance camera, or a medical camera, or the like.

[0022] Apparatus Configuration

[0023] First, the configuration and functions of the imaging apparatus 100 according to the present embodiment will be described with reference to Figure 1 and Figure 2

[0024] Figure 1 is a block diagram illustrating a hardware configuration of the imaging apparatus 100 according to the present embodiment. Figure 2 is a block diagram illustrating a functional configuration of the image processing unit 104 of the imaging apparatus 100 according to the present embodiment.

[0025] The imaging apparatus 100 includes an optical unit 101, an imaging unit 102, an A / D conversion unit 103, an image processing unit 104, a display unit 105, a storage unit 106, a recording medium 107, a system control unit 108, and an operation unit 109.

[0026] The optical unit 101 includes a lens group including a zoom lens and a focus lens, and a shutter having an aperture function. The optical unit 101 adjusts the magnification, the focus position, and the light amount of an object image. The magnification, the focus state, and the light amount of the object image reaching the imaging unit 102 are adjusted, and an image is formed on an imaging surface of the imaging unit 102.

[0027] The imaging unit 102 includes an image sensor including a CCD and a CMOS, which converts an optical image of an object formed by the optical unit 101 into an electric signal. The imaging unit 102 generates still image data and moving image data including an analog signal.

[0028] The A / D conversion unit 103 converts an analog signal generated by the imaging unit 102 into a digital signal. The A / D conversion unit 103 generates still image data or moving image data including a digital signal from still image data or moving image data including an analog signal.

[0029] ​The image processing unit 104 performs various types of image processing on the image data output from the A / D conversion unit 103. The various types of image processing include, for example, development processing such as pixel interpolation, gamma conversion processing for generating a gain map, and color matrix processing. The image processing unit 104 generates an image file by compressing and encoding the still image data subjected to the image processing in a JPEG format or the like, or encoding the moving image data in a moving image compression method such as an MP4 format, and records the image file in the recording medium 107. The image processing unit 104 decodes the still image file read from the recording medium 107, and decodes the moving image file read from the recording medium 107. The image processing unit 104 can perform similar image processing not only on the image data output from the A / D conversion unit 103 but also on the image data read from the recording medium 107. Details of the detailed configuration and functions of the image processing unit 104 will be described later with reference to Figure 2 The detailed configuration and functions of the image processing unit 104 will be described later with reference to

[0030] The display unit 105 performs display of the image data (live view) captured by the imaging unit 102, display of the image data read from the recording medium 107, display of a graphical user interface (GUI) for interactive operation, and the like. The display unit 105 includes, for example, a display device such as a liquid crystal display or an organic EL display. The display unit 105 includes a display device that can express a luminance range (dynamic range) compatible with a standard dynamic range (SDR). The image processing unit 104 converts a high dynamic range (HDR) image that is a main image read from the recording medium 107 into an SDR image by applying a gain map added to the HDR image. The display unit 105 displays the SDR image converted from the HDR image by applying the gain map in the image processing unit 104. This can reduce a case where the HDR image is displayed on the display unit 105 that is not compatible with the HDR, and prevent a situation where the image is displayed with an image quality not assumed by the user.

[0031] The storage unit 106 is a volatile memory for storing various types of information such as an image processing program and a gain map required for the image processing by the image processing unit 104, and storing image data during the image processing, and the like.

[0032] The recording medium 107 is a non-volatile memory for recording an image file to which a gain map is added. The recording medium 107 is, for example, a memory card or a hard disk, and is built in the imaging device 100 or detachable from the imaging device 100.

[0033] The system control unit 108 includes a processor (CPU) for performing arithmetic processing and control processing of the imaging device 100, a volatile memory (ROM) for storing a program executed by the processor, and a work memory (RAM) loaded with a program read from a nonvolatile memory, constants, and variables used for executing the program, and the like. The system control unit 108 controls the components of the imaging device 100 by loading the program stored in the ROM into the RAM and executing the program.

[0034] The operation unit 109 is an operation member such as a switch, a button, or a touch panel for receiving various operations from a user and notifying the system control unit 108 of the operations. The operation unit 109 includes at least a still image capturing button, a moving image capturing button, a mode dial, and a power switch.

[0035] The still image capturing button is an operation member for instructing the system control unit 108 to perform a capturing process of a still image. The moving image capturing button is an operation member for instructing the system control unit 108 to perform a capturing process of a moving image.

[0036] The mode dial is an operation member for switching an operation mode of the imaging device 100. The mode dial can switch the operation mode of the imaging device 100 to any of a still image capturing mode, a moving image capturing mode, and a reproduction mode.

[0037] The power switch is an operation member for switching on / off of a power supply of the imaging device 100.

[0038] When the still image capturing button is half-pressed, an instruction of a capturing preparation process is notified to the system control unit 108. In the still image capturing mode, when the still image capturing button is half-pressed, the system control unit 108 starts a capturing preparation process (AE processing and AF processing) of a still image. When the still image capturing button is fully pressed, an instruction of a capturing process is notified to the system control unit 108. When the still image capturing button is fully pressed, the system control unit 108 performs a still image capturing process of recording image data captured by the imaging unit 102 on the recording medium 107.

[0039] In the moving image capturing mode, the system control unit 108 performs a capturing preparation process (AE processing and AF processing) of image data (frames) captured by the imaging unit 102 in response to a first pressing of the moving image capturing button, continues a moving image capturing process for recording a moving image of a predetermined time on the recording medium 107, and stops the moving image capturing process in response to a second pressing of the moving image capturing button.

[0040] In Figure 1In the example, the optical unit 101 is configured as a part of the image pickup device 100 including the image pickup unit 102, but is not limited to this configuration. For example, as in a single-lens reflex camera, a replaceable optical unit (replaceable lens) can be configured to be detachable from the image pickup device 100.

[0041] Figure 2 is a block diagram illustrating a functional configuration of the image processing unit 104 according to the present embodiment.

[0042] The image (RAW image) input to the image processing unit 104 is data generated by the A / D conversion unit 103 converting an analog image signal generated by the image pickup unit 102 into a digital image signal, and is a Bayer image including three components of red (R), green (G), and blue (B).

[0043] The development processing unit 211 generates an image in a predetermined format such as YUV422 by performing various types of image processing (development processing) on the RAW image input to the image processing unit 104. The development processing unit 211 includes a white balance processing unit 205, a color matrix processing unit 206, and a gamma processing unit 207.

[0044] The white balance processing unit 205 calculates a white balance gain based on the RAW image input to the image processing unit 104, and adjusts the white balance by applying the gain to the signal value of each pixel of red (R), green (G), and blue (B).

[0045] The color matrix processing unit 206 performs color matrix processing for converting the color gamut of the image data output from the white balance processing unit 205 on the image data. The color matrix processing can change the conversion coefficient in accordance with the spectral characteristics of the image sensor of the image pickup unit 102, the color gamut of the output image such as BT.601 or BT.2020, and the target value of color reproduction, and the like.

[0046] The gamma processing unit 207 performs gamma processing on the image data output from the color matrix processing unit 206, the gamma processing being for converting the signal value in accordance with a gamma characteristic (opto-electric transfer function: OETF) to generate an image signal that is compatible with the gamma characteristic of the output destination. For example, in order to generate an SDR image, gamma processing based on the gamma of the sRGB standard is performed, and in order to generate an HDR image, gamma processing based on the gamma characteristic of the OETF defined in ITU-R BT.2100 is performed.

[0047] The image resizing processing unit 208 resizes (reduces or enlarges) the RAW image input to the image processing unit 104 and the YUV image output from the development processing unit 211. The image resizing processing unit 208 generates a resized image using a known method such as bilinear interpolation or bicubic interpolation. Note that in the enlargement processing, a super-resolution process can be performed by machine learning such as deep learning.

[0048] The gain map generating unit 220 generates a gain map based on the RAW image input to the image processing unit 104. The gain map is conversion information used when correcting the signal value of each pixel of a main image to generate an image having different luminance. The gain map has a data configuration in which a gain value for correcting the signal value for each pixel is two-dimensionally arranged in correspondence with each pixel. In the present embodiment, an example of a gain map for converting an HDR image to an SDR image will be described.

[0049] Figures 3A to 3D is a diagram describing a method of generating a gain map for converting an HDR image to an SDR image.

[0050] Figure 3A The relationship between the signal value and the display luminance of the output image in a case where the SDR image and the HDR image (output image) output from the imaging device 100 are displayed on an HDR-compatible display device will be exemplified. The broken line 304 exemplifies the relationship between the signal value and the display luminance of the SDR image. The broken line 305 exemplifies the relationship between the signal value and the display luminance of the HDR image. The display luminance has a characteristic such that the display luminance of the SDR image and the display luminance of the HDR image coincide with each other in a case where the signal value of the output image is small, and there is a difference in the display luminance as the signal value increases. Such a display luminance characteristic enables the HDR image and the SDR image to be displayed in a manner that does not give the viewer a strange feeling even in a case where the HDR image and the SDR image are mixed.

[0051] In order to realize the display luminance characteristic as exemplified in Figure 3A , a gamma characteristic based on an electro-optical transfer function (EOTF) of an HDR-compatible display device can be provided in the gamma processing unit 207. Figure 3BThe gamma characteristics set in the gamma processing unit 207 when generating the SDR image and the HDR image are exemplified. The gamma characteristic 301 is a gamma characteristic of the sRGB standard. The gamma characteristic 302 is an SDR gamma set in the gamma processing unit 207 when the SDR image is generated in the imaging device 100. The SDR gamma 302 has a higher contrast than the contrast of the gamma characteristic 301 of the sRGB standard, and the tone is adjusted to improve the appearance of the image captured by the imaging device 100. The gamma characteristic 303 is an HDR gamma set in the gamma processing unit 207 when the HDR image is generated in the imaging device 100.

[0052] Figure 3C Exemplification Figure 3A of the ratio between the signal value and the display luminance of the HDR image and the SDR image in Figure 3D Exemplification of an example of the conversion of the ratio in Figure 3B to the relationship of the ratio of the display luminance of the HDR image and the SDR image to the signal value of the RAW image according to the gamma characteristic in Figure 3C

[0053] By storing the gain information for the signal value of the RAW image exemplified in Figure 3D in the storage unit 106, the base gain map generation unit 200 can calculate a gain value for converting the HDR image (main image) to the SDR image according to the input RAW image. The gain value generated for each pixel or each pixel in a plurality of pixels (2x2 pixels, etc.) of the RAW image is referred to as a base gain map. Note that the base gain map is not limited to information for converting the HDR image to the SDR image, and can be information for converting the SDR image to the HDR image, or information for converting the HDR image to an HDR image having a different dynamic range.

[0054] The gain map size adjustment processing unit 201 performs a size adjustment process (reduction or enlargement) for changing the resolution of all regions of the base gain map generated by the base gain map generation unit 200. The size adjustment processing method can be a known method such as bilinear interpolation or bicubic interpolation, or can be a size adjustment process performed by machine learning such as deep learning.

[0055] The gain map region integration unit 202 performs integration (gain map region integration process) of the maps of adjacent regions to reduce the data amount of the gain map generated by the base gain map generation unit 200 or the gain map size adjustment processing unit 201. Figure 4A and Figure 4B Exemplification of the region integration process of the gain map performed by the gain map region integration unit 202. Figure 4A ​The gain map generated by the example base gain map generation unit 200 or the gain map size adjustment processing unit 201 is illustrated. For example, the data amount of the gain map is reduced by bringing the gain map of Figure 4B into data gathered in a 2x2 region as Figure 4A . Note that in the examples of Figure 4A and Figure 4B , the case where the gains of the adjacent regions are the same is illustrated, but in a case where the gains in the regions can be considered to be the same (such as a case where all the peripheral regions of the region of interest have the same gain), the gains can be integrated. Resolutions other than 1x1 and 2x2 can be employed.

[0056] Figure 5A and Figure 5B Examples of the region division of the gain map performed by the gain map region division unit 203 are illustrated. The gain map region division unit 203 performs region division that partially or entirely increases the resolution of the gain map with respect to the gain map having regions that have been integrated by the gain map region integration unit 202 or the like.

[0057] The gain map encoding unit 204 performs a data amount-reducing process on the gain map output from the gain map region integration unit 202. For example, the bit depth of the gain map can be reduced by taking the logarithm of the gain map, or the data amount of the gain map can be reduced by performing lossless compression or lossy compression. The base gain map generated by the base gain map generation unit 200 can be input to the gain map encoding unit 204 as is.

[0058] The image comparison unit 212 calculates the difference between two images with respect to each pixel or each region. Based on the difference information calculated by the image comparison unit 212, the gain map region integration unit 202 performs gain map region integration processing, or the gain map region division unit 203 performs gain map region division processing.

[0059] The file storage unit 210 generates a file in which the main image generated by the development processing unit 211, the sub image (reduced image or enlarged image) having a resolution different from that of the main image generated by the image size adjustment processing unit 208, and the gain map of the main image and the sub image are combined into one. The format of the file is defined as CIPA DC-007 Multi-Picture Format (MPF) that is a standard of the Camera & Imaging Products Association (CIPA).

[0060] Figure 6A and Figure 6B The data configuration of the main image to which the gain map is added according to the present embodiment is illustrated.

[0061] Figure 6A Data configuration of an example RAW image file.

[0062] The RAW image file includes a main image RAW, a reduced RAW that has been resized by the image resizing processing unit 208 from the main image RAW, a monitor display JPEG used for display on a display device or the like, and a thumbnail JPEG used at the time of multi-display in which a plurality of images are simultaneously displayed. Gain maps (main image gain map, reduced RAW gain map, monitor display JPEG gain map, thumbnail gain map) corresponding to each image are also recorded in the RAW image file. An image other than the main image such as the reduced RAW, the monitor display JPEG, and the thumbnail JPEG is referred to as a sub image, and a gain map other than the main image gain map is referred to as a sub image gain map. Note that the number of gain maps can be reduced by using both the main image gain map and the reduced RAW gain map or by using the reduced RAW gain map and the monitor display JPEG gain map. Furthermore, various combinations such as a configuration that does not include a sub image and does not include a sub image gain map and a configuration that includes a sub image and does not include a sub image gain map are possible.

[0063] Figure 6B Data configuration of an example JPEG image file.

[0064] The JPEG image file is similar in data configuration to the RAW image file of Figure 6A except that the main image RAW is replaced by the main image JPEG and there is no reduced RAW and reduced RAW gain map.

[0065] The image processing control unit 209 controls Figure 2 the components of the image processing unit 104 of to generate and regenerate a main image, sub images, and gain maps and the like from an input RAW image, and to generate an output image to a display device.

[0066] [First Embodiment]

[0067] In the following, a gain map generation process at the time of photographing by the imaging device 100 according to the first embodiment will be described. Figure 7

[0068] Figure 7 is a flowchart showing the control process at the time of photographing according to the first embodiment. Figure 7 The process of is realized by the system control unit 108 executing a program stored in the ROM and controlling the image processing unit 104.

[0069] When the imaging device 100 is powered on and the system control unit 108 receives a photographing instruction from a shutter switch included in the operation unit 109, the process shown in Fig. 10 is started.Figure 7 The system control unit 108 controls the optical unit 101, the imaging unit 102, and the A / D conversion unit 103 to generate a RAW image.

[0070] In step S701, the image processing control unit 209 controls the development processing unit 211 to generate a master image (an HDR image) based on the RAW image input to the image processing unit 104.

[0071] In step S702, the image processing control unit 209 outputs the master image generated in step S701 to the image size adjustment processing unit 208. The image size adjustment processing unit 208 performs a size adjustment process on the master image to generate a sub image. Note that the sub image can be generated by outputting a reduced RAW on which the RAW image has been size-adjusted by the image size adjustment processing unit 208 to the development processing unit 211, and performing a development process on the reduced RAW by the development processing unit 211.

[0072] In step S703, the image processing control unit 209 controls the gain map generation unit 220 to generate a master image gain map from the master image generated in step S701. The master image gain map can have a configuration in which the resolution is the same as that of the master image, or a configuration in which the resolution is reduced to 1 / 4 or the like by the gain map size adjustment processing unit 201. The gain map region integration processing can be performed by the gain map region integration unit 202.

[0073] In step S704, the image processing control unit 209 controls the gain map size adjustment processing unit 201 and the gain map region integration unit 202 to generate a sub image gain map from the master image gain map generated in step S703. Note that a reduced RAW on which the RAW image has been size-adjusted by the image size adjustment processing unit 208 can be output to the gain map generation unit 220, and the gain map generation unit 220 can generate the sub image gain map.

[0074] In step S705, the image processing control unit 209 outputs the images and the gain maps generated in steps S701 to S704 to the file storage unit 210, and the file storage unit 210 generates a file in which the images and the gain maps are integrated. The system control unit 108 stores the file generated by the file storage unit 210 in the recording medium 107, and ends the process.

[0075] [Second Embodiment]

[0076] Hereinafter, the gain map generation process at the time of image reduction by the imaging device 100 according to the second embodiment will be described with reference to Figure 8 In step S703, the image processing control unit 209 controls the gain map generation unit 220 to generate a master image gain map from the master image generated in step S701. The master image gain map can have a configuration in which the resolution is the same as that of the master image, or a configuration in which the resolution is reduced to 1 / 4 or the like by the gain map size adjustment processing unit 201. The gain map region integration processing can be performed by the gain map region integration unit 202.

[0077] In the second embodiment, it is assumed that the user has Figure 6B The image file of the illustrated configuration is stored in the recording medium 107.

[0078] Figure 8 is a flowchart showing the control processing at the time of image reduction according to the second embodiment.

[0079] The user selects an image to be subjected to reduction processing from the images stored in the recording medium 107 via the operation unit 109, thereby starting the processing (step S800).

[0080] When the processing is started in step S800, the system control unit 108 reads the main image file selected by the user from the recording medium 107 and stores the main image file into the storage unit 106.

[0081] In step S801, the image processing control unit 209 controls the image size adjustment processing unit 208 to perform reduction processing on the main image stored in the storage unit 106 according to the reduction rate specified by the user via the operation unit 109.

[0082] In step S802, the image processing control unit 209 judges whether or not to regenerate the main image gain map. When judged not to regenerate the main image gain map, the image processing control unit 209 skips the main image gain map regeneration processing in step S803, and causes the processing to proceed to step S804.

[0083] In step S804, the image processing control unit 209 outputs the main image file stored in the storage unit 106 in step S800 and the reduced image generated in step S801 to the file storage unit 210, generates a file in which the main image is replaced with the reduced image generated in step S801 with respect to the main image file stored in the storage unit 106 in step S800, and stores the file into the recording medium 107, and ends the processing.

[0084] When judged to regenerate the main image gain map in step S802, the image processing control unit 209 causes the processing to proceed to step S803.

[0085] Here, the judgment processing in step S802 will be described. For example, when the size adjustment rate specified by the user satisfies a predetermined condition, that is, when the reduction rate is close to 1, that is, when the difference in the number of pixels between the main image and the size-adjusted image is small, the image processing control unit 209 judges that the gain map of the main image does not need to be newly generated. In a case where the gain map of the sub image can be used instead of the gain map of the main image (such as a case where the reduction rate specified by the user is close to the ratio of the number of pixels between the main image and the sub image), the image processing control unit 209 judges that the gain map of the main image does not need to be newly generated. In contrast, for example, in a case where a sufficiently large gain map (a ratio of the number of pixels of 1:1 or the like) for the main image is included, or in a case where a gain map that has not been subjected to region integration is stored, there is a high possibility that the data size can be greatly reduced by the regeneration of the gain map, and therefore, the image processing control unit 209 judges that the regeneration of the gain map is needed.

[0086] In step S803, the image processing control unit 209 performs the regeneration of the main image gain map. The regeneration of the main image gain map is achieved by a combination of one or more of the data compression processes such as the resolution change process by the gain map size adjustment processing unit 201, the region division process by the gain map region division unit 203, the region integration process by the gain map region integration processing 202, and the bit depth change process by the gain map encoding unit 204. In a case where the regeneration of the gain map is completed in step S803, the image processing control unit 209 causes the process to proceed to step S804, outputs the main image file stored in the storage unit 106 in step S800, the reduced image generated in step S801, and the main image gain map regenerated in step S803 to the file storage unit 210, generates a file in which the main image is replaced with the reduced image generated in step S801 and the main image gain map is replaced with the main image gain map regenerated in step S803 with respect to the main image file stored in the storage unit 106 in step S800, and stores the file in the recording medium 107, and ends the process.

[0087] In the second embodiment, the image file having the configuration as exemplified in Figure 6B However, similar processing can also be performed for an image file that does not have a sub image and does not have a sub image gain map.

[0088] According to the second embodiment, by also adjusting the gain map added to the main image file when the main image is size-adjusted, both the size reduction and the precision retention of the gain map can be achieved.

[0089] [Third Embodiment]

[0090] Hereinafter, the Figure 9A and Figure 9B The gain map generation processing at the time of image enlargement by the imaging device 100 of the third embodiment will be described.

[0091] The image enlargement processing of the third embodiment is similar to the case where the reduction processing of the second embodiment is rephrased as enlargement. Figure 8

[0092] Figure 9A and Figure 9B are diagrams that describe the gain map generation processing at the time of image enlargement according to the third embodiment.

[0093] As exemplified in Figure 9A , the image processing control unit 209 performs known super-resolution processing on the main image by the image size adjustment processing unit 208. In this case, as exemplified in Figure 9B , it is assumed that the gain map corresponding to the main image includes a region that has not undergone region integration and a region that has undergone region integration. In the gain map, it is considered that there is no problem even if super-resolution processing is performed only on the region that has not undergone region integration, and thus the super-resolution processing is partially performed to save the time required for the super-resolution processing. Along the same lines, it is possible to perform super-resolution processing only on the gain map of a region whose resolution is greatly improved by the super-resolution processing. As to whether the resolution is greatly improved, the image comparison unit 212 can determine the difference between the main image that has undergone the super-resolution processing and an image that is enlarged by known interpolation techniques.

[0094] According to the third embodiment, in the case where the gain map added to the main image file is also size-adjusted in response to the size adjustment of the main image, by partially size-adjusting the gain map, it is possible to save the time required for the size adjustment of the gain map.

[0095] Other Embodiments

[0096] Embodiments of the present application can also be embodied by a method in which a computer (Central Processing Unit (CPU), Micro Processing Unit (MPU)) of a system or an apparatus reads out and executes computer program codes of the software (computer program product including the computer program) provided by a network or various storage media to perform the functions of the above-described embodiments.

[0097] While the present disclosure has been described with reference to exemplary embodiments, it is to be understood that the present disclosure is not limited to the disclosed exemplary embodiments. The scope of the appended claims will meet the most broad interpretation so as to encompass all such modifications and equivalent structures and functions.​

Claims

1. An image processing apparatus, comprising: The first generation unit is configured to generate first transformation information used when generating an image with a different dynamic range from the first image; An image resizing processing unit is configured to perform resizing processing on the first image; as well as A control unit is configured to control whether to perform resizing processing on the first transformation information added to the first image when performing resizing processing on the first image.

2. The image processing apparatus according to claim 1, wherein, When the resizing rate of the first image does not meet the predetermined conditions, the control unit performs control to resize the first conversion information.

3. The image processing apparatus according to claim 2, further comprising a file storage unit configured to generate a first file storing the first image and the first conversion information generated from the first image. in, When performing the resizing process of the first conversion information, the file storage unit generates a file for the first file in which the first image is replaced with a resized second image and the first conversion information is replaced with resized second conversion information.

4. The image processing apparatus according to claim 2, wherein, The resizing process includes shrinking, and The predetermined condition is that the resolution difference between the first image and the resized second image is less than a predetermined value, in which case the resized second conversion information can be used to replace the first conversion information.

5. The image processing apparatus according to claim 1, wherein, If the resizing rate of the first image meets a predetermined condition, the control unit controls the process to prevent the resizing of the first conversion information from being performed.

6. The image processing apparatus according to claim 3, wherein, Without performing the resizing process on the first conversion information, the file storage unit generates a file for the first file in which the first image is replaced by a resized second image.

7. The image processing apparatus according to claim 5, wherein, The resizing process includes shrinking, and The predetermined condition is that the resolution difference between the first image and the resized second image is less than a predetermined value, in which case the resized second conversion information can be used to replace the first conversion information.

8. The image processing apparatus according to any one of claims 1 to 7, wherein, The first conversion information forms a gain map, in which the gain value for the signal value of each pixel is determined, and The gain map resizing process includes any one or more of the following processes: Resolution change processing is used to change the resolution of all regions of the gain map; Region integration processing is used to integrate some regions of the gain map; as well as Region segmentation processing is used to segment some regions of the gain map.

9. The image processing apparatus according to claim 8, wherein, The control unit determines which of the following processes should be performed—resolution change processing, region integration processing, and region segmentation processing—based on the size adjustment rate of the first image.

10. The image processing apparatus according to any one of claims 1 to 7 and 9, wherein, The image resizing processing unit generates one or more second images that have been resized from the first image, and When performing resizing processing on the first image, the control unit controls the resizing processing of the first transformation information added to the first image, or controls the resizing processing of the second transformation information added to the second image.

11. The image processing apparatus according to claim 10, wherein, The first generation unit generates the second transformation information to be added to the second image from the first transformation information.

12. The image processing apparatus according to any one of claims 1 to 6, wherein, The resizing process for the first image includes magnification, and The control unit performs amplification processing on the first conversion information in part.

13. The image processing apparatus according to claim 12, wherein, The magnification process includes super-resolution processing using machine learning.

14. The image processing apparatus according to any one of claims 1 to 7, 9, 11 and 13, wherein, The first conversion information is any one of the following: information for converting an image in a first dynamic range to an image in a second dynamic range, information for converting an image in a second dynamic range to an image in a first dynamic range, or information for converting an image in a first dynamic range to an image in a first dynamic range having a different dynamic range.

15. The image processing apparatus according to any one of claims 1 to 7, 9, 11 and 13, further comprising: Camera unit; as well as The second generation unit is configured to generate the first image from the RAW image generated by the camera unit.

16. A method for controlling an image processing device, comprising the following steps: Generate transformation information for generating an image with a different dynamic range than the first image; Perform resizing processing on the first image; as well as When performing resizing processing on the first image, control whether to perform resizing processing on the transformation information added to the first image.

17. A computer-readable storage medium storing a program for enabling a computer to function as an image processing apparatus according to any one of claims 1 to 15.

18. A computer program product for enabling a computer to function as an image processing device according to any one of claims 1 to 15.

Images