Image processing device, image processing method, and program

Abstract

This technology generates conversion information suitable for a composite image created by combining multiple images, each containing conversion information used when generating images in different formats. [Solution] The image processing device includes: acquisition means for acquiring a plurality of first images and a plurality of second images which have a different format from the plurality of first images and correspond to each of the plurality of first images; synthesis means for combining images with the same format from the plurality of first images and the plurality of second images to generate a first composite image and a second composite image; and generation means for generating composite image conversion information for mutually converting the first composite image and the second composite image from the first composite image and the second composite image.

Description

Technical Field

[0001] The present invention relates to a technique for generating conversion information used when acquiring images of different formats.

Background Art

[0002] Conventionally, in order to conform to the dynamic range that a display device can display, images in HDR (High Dynamic Range) format and images in SDR (Standard Dynamic Range) format have been mutually converted using conversion information called a gain map (Non-Patent Document 1).

[0003] Also, in Patent Document 1, it is described that an image with a wide dynamic range and a wide color gamut can be generated with a low processing load based on base image data and difference data used in processing for expanding the dynamic range and color gamut of the image data.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Non-Patent Documents

[0005]

Non-Patent Document 1

[0006] When combining images with gain maps, the gain map that was present in the original image disappears from the combined image, making it difficult to convert between images of different formats. Patent Document 1 does not mention gain maps that take into account the image combining process of images with gain maps.

[0007] The present invention has been made in view of the above problems, and its objective is to realize a technology for generating conversion information suitable for a composite image obtained by combining multiple images to which conversion information used when generating images of different formats has been added. [Means for solving the problem]

[0008] To solve the above problems and achieve the objective, the image processing apparatus of the present invention includes: acquisition means for acquiring a plurality of first images and a plurality of second images having a different format from the plurality of first images and corresponding to each of the plurality of first images; synthesis means for generating a first composite image and a second composite image by combining images with the same format from the plurality of first images and the plurality of second images; and generation means for generating composite image conversion information for mutually converting the first composite image and the second composite image from the first composite image and the second composite image. [Effects of the Invention]

[0009] According to the present invention, it is possible to generate conversion information suitable for a composite image obtained by combining multiple images to which conversion information used when generating images of different formats has been added. [Brief explanation of the drawing]

[0010] [Figure 1] A block diagram illustrating the device configuration of Embodiment 1. [Figure 2] An explanatory diagram of the image file for Embodiment 1. [Figure 3] An explanatory diagram of the image conversion process in Embodiment 1. [Figure 4] An explanatory diagram of the image synthesis process in Embodiment 1. [Figure 5] A diagram illustrating the gain map generation process of Embodiment 1. [Figure 6] An explanatory diagram of the composite image file of Embodiment 1. [Figure 7] A block diagram illustrating the device configuration of Embodiment 2. [Figure 8] An explanatory diagram of the image file for Embodiment 2. [Figure 9] An explanatory diagram of the image conversion process in Embodiment 2. [Figure 10] An explanatory diagram of the image synthesis process in Embodiment 2. [Modes for carrying out the invention]

[0011] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the invention according to the claims. Although a plurality of features are described in the embodiments, not all of these plurality of features are essential to the invention, and the plurality of features may be arbitrarily combined. Further, in the accompanying drawings, the same or similar configurations are denoted by the same reference numerals, and redundant descriptions are omitted.

[0012] In this embodiment, an embodiment in which the image processing apparatus of the present invention is applied to an information processing apparatus such as a personal computer (PC) or an information processing terminal such as a smart device or a tablet PC will be described. However, the present invention is not limited thereto, and it may be applied to an imaging apparatus such as a digital camera capable of capturing images.

[0013] [Embodiment 1] First, referring to FIG. 1, the configuration and functions of the image processing apparatus 10 according to Embodiment 1 will be described.

[0014] FIG. 1 is a block diagram illustrating the functional configuration of the image processing apparatus 10 according to Embodiment 1. FIG. 2 is an explanatory diagram of the image file according to Embodiment 1.

[0015] Each functional block of the image processing apparatus 10 according to Embodiment 1 can be implemented by software or a combination of software and hardware, except for parts that can be realized only by hardware. For example, the functional block may be realized by dedicated hardware such as an ASIC. Also, the functional block may be realized by a processor such as a CPU executing a program stored in a memory. Note that a plurality of functional blocks may be realized by a common configuration (for example, one ASIC). Also, the hardware that realizes a part of the functions of a certain functional block may be included in the hardware that realizes other functional blocks.

[0016] The image processing apparatus 10 inputs a plurality of image files including image data and a gain map attached to the image data, performs a synthesis process on the input image data, and generates synthesized image data. Then, the image processing apparatus 10 generates a gain map for the synthesized image, and outputs the one attached to the synthesized image data as an image file.

[0017] The gain map is conversion information for mutually converting an image in HDR (High Dynamic Range) format (HDR image) and an image in SDR (Standard Dynamic Range) format (SDR image) so as to conform to the dynamic range that the display device can display. Also, the format of the gain map includes a multi-channel or color channel (3-channel) format and a monochrome channel (1-channel) format. The number of channels of the gain map is not limited to 1 channel or 3 channels.

[0018] The image processing apparatus 10 includes an image input unit 101, a processing information input unit 102, an image conversion unit 103, an image synthesis unit 104, a gain map generation unit 105, and a file output unit 106. The image input unit 101 is a functional block that inputs the image file 1000 and the image file 1001 from an external device such as a camera. The image file 1000 includes the image data 10000 and the gain map 10001 attached to the image data 10000. The image file 1001 includes the image data 10010 and the gain map 10011 attached to the image data 10010. The image data 10000 and the image data 10010 are image data of HDR images. The gain map 10001 and the gain map 10011 are gain maps used for processes for mutually converting HDR images and SDR images.

[0019] In this embodiment, an HDR image is an image with a wider dynamic range than an SDR image, and is, for example, a YUV image to which the Electronic-Opto Transfer Function (EOTF) characteristics described in ST2084, an HDR standard to which a display device capable of displaying HDR images conforms, are applied. An SDR image is a YUV image with a narrower dynamic range than an HDR image, and is, for example, a YUV image to which sRGB gamma characteristics are applied. In this embodiment, the gamma characteristics of the HDR image are assumed to be HDR gamma, and the gamma characteristics of the SDR image are assumed to be SDR gamma, and the color gamut is assumed to be common.

[0020] Furthermore, in this embodiment, the gain map is a single-channel data with gain information for the same number of pixels as the assigned image. The gain map in this embodiment is obtained by degammaring the HDR image and the SDR image, respectively, and then recording the ratio of the Y component of the HDR image to the SDR image for each pixel. To convert from an HDR image to an SDR image, the HDR image is degammared, the Y component of each pixel is changed based on the gain of the gain map, and then the SDR gamma is applied to convert it to an SDR image. Conversely, to convert from an SDR image to an HDR image, the SDR image is degammared, the Y component of each pixel is changed based on the gain of the gain map, and then the HDR gamma is applied to convert it to an HDR image.

[0021] In this embodiment, the input image files are defined as files containing HDR image data and gain maps, but they may also be files containing SDR image data and gain maps. Furthermore, while the gain map in this embodiment has the same number of pixels as the image, its configuration is not limited to this. For example, it may have a reduced number of pixels compared to the image and be scaled to the same number of pixels as the image when used. Also, in this embodiment, image data 10000 and image data 10010 have the same resolution, but they may have different resolutions. In addition, the gain of each pixel in the gain map in this embodiment is created from the ratio of Y values ​​after degammaring the HDR image and SDR image, but the creation method is not limited to this. For example, the HDR image may be converted to an RGB image with HDR gamma applied, and the SDR image to an RGB image with SDR gamma applied, and the gain may be calculated from the ratio of any of the RGB values. Furthermore, while the gain map in this embodiment is data with 1 channel, it is not limited to this. For example, it may be data with 3 channels, each containing gain information corresponding to the R (red), G (green), and B (blue) color components of the RGB image. Finally, in this embodiment, there are two input image files, but there may be three or more.

[0022] The image input unit 101 outputs image data 10000 contained in image file 1000 and image data 10010 contained in image file 1001 to the image conversion unit 103 and the image synthesis unit 104. The image input unit 101 also outputs gain map 10001 contained in image file 1000 and gain map 10011 contained in image file 1001 to the image conversion unit 103.

[0023] Figure 2 illustrates the image file 1000, image data 10000, gain map 10001, image file 1001, image data 10010, and gain map 10011 of Embodiment 1. In Figure 2, each region of image data 10000 and image data 10010 indicates that the closer to white the region is, the higher the brightness, and the closer to black the region is, the lower the brightness. In addition, the white areas of gain map 10001 and gain map 10011 indicate regions where there is a difference in the Y component between the HDR image and the SDR image, while the shaded areas indicate regions where the Y component of the HDR image and the SDR image are equivalent.

[0024] The processing information input unit 102 is a functional block that receives user instructions 1004 from the user specifying the processing content.

[0025] The processing information input unit 102 includes, for example, a display that shows the user the type of image synthesis process and the file format of the synthesized image after the synthesis process, and a mouse or touch panel that accepts user instructions. The user instructs the user via the processing information input unit 102 on the type of image synthesis process and the file format of the synthesized image after the synthesis process. The processing information input unit 102 outputs processing information 1020 corresponding to the user instruction 1004 to the image synthesis unit 104 and the gain map generation unit 105.

[0026] The image conversion unit 103 is a functional block that uses image data and a gain map to acquire image data that corresponds to the given image data but has a different format. If the input image is an HDR image, it is converted to an SDR image, and if the input image is an SDR image, it is converted to an HDR image.

[0027] The image conversion unit 103 degammaps the input HDR image, then converts the Y component for each pixel of the image based on the information described in the gain map, and applies SDR gamma to convert it into an SDR image.

[0028] In this embodiment, converted image data 1030, which is an SDR image, is generated using image data 10000, which is an HDR image, and a gain map 10001. Additionally, converted image data 1031, which is an SDR image, is generated using image data 10010, which is an HDR image, and a gain map 10011.

[0029] The image conversion unit 103 outputs the generated converted image data 1030 and converted image data 1031 to the image synthesis unit 104. In this embodiment, each converted image data also includes information indicating which image data it was converted from.

[0030] Figure 3 is an explanatory diagram of the image conversion process in Embodiment 1.

[0031] In this embodiment, the regions where there is a difference in image color between the HDR image data 10000 and the converted image data 1030, which is an SDR image, indicate the portion where the Y component changed when the conversion between the HDR image and the SDR image was performed. Similarly, the regions where there is a difference in image color between the HDR image data 10010 and the converted image data 1031, which is an SDR image, indicate the portion where the Y component changed when the conversion between the HDR image and the SDR image was performed.

[0032] The image synthesis unit 104 is a functional block that generates a composite image based on the information described in the processing information 1020, using the input image data 10000, image data 10010, converted image data 1030, and converted image data 1031.

[0033] The image synthesis unit 104 identifies images with the same format from the input image data 10000, image data 10010, converted image data 1030, and converted image data 1031. In this embodiment, the image format is identified as either an HDR image or an SDR image. In this embodiment, image data 10000 and image data 10010 are identified as having the same image format as HDR images, and converted image data 1030 and converted image data 1031 are identified as having the same image format as SDR images.

[0034] Furthermore, the image synthesis unit 104 selects an image synthesis method based on the information described in the processing information 1020. In this embodiment, comparative brightness synthesis is selected as the synthesis method, and each pixel of the images to be synthesized is compared, with the brighter pixels being adopted as pixels in the synthesized image. Also, in Embodiment 1, the synthesized image has the same resolution and format as the images used for synthesis.

[0035] Next, the image synthesis unit 104 compares the Y values ​​for each pixel at the same coordinate in both the image data 10000 and the image data 10010. It then selects the pixel value of the pixel with the higher Y value as the pixel value of that coordinate in the synthesized image, thereby generating the synthesized image data 1040, which is a synthesized HDR image.

[0036] Furthermore, the image synthesis unit 104 compares the Y values ​​for each pixel at the same coordinate in both the converted image data 1030 and the converted image data 1031. It then selects the pixel value of the pixel with the higher Y value as the pixel value of that coordinate in the synthesized image, thereby generating synthesized image data 1041, which is a synthesized image of the SDR images.

[0037] Figure 4 is an explanatory diagram of the image synthesis process in Embodiment 1.

[0038] In Figure 4, each region of image data 10000, image data 10010, composite image data 1040, converted image data 1030, converted image data 1031, and composite image data 1041 shows that the closer to white the region is, the higher the brightness, and the closer to black the region is, the lower the brightness.

[0039] The image synthesis unit 104 outputs the generated synthesized image data 1040 and synthesized image data 1041 to the gain map generation unit 105.

[0040] The gain map generation unit 105 is a functional block that generates a gain map based on the information described in the processing information 1020, using the input composite image data 1040 and composite image data 1041.

[0041] In Embodiment 1, the processing information 1020 includes information that the image format of the composite image data to be output is HDR, and the format of the gain map to be output is 1-channel data for converting the Y component of each pixel.

[0042] The gain map generation unit 105 degammaifies the composite image data 1040, which is an HDR image, and the composite image data 1041, which is an SDR image, and then records the ratio of the Y component of each pixel, generating it as a gain map 1051.

[0043] Figure 5 is an explanatory diagram of the gain map generation process in Embodiment 1.

[0044] In Figure 5, the regions where there is a difference in color between the composite image data 1040, which is an HDR image, and the composite image data 1041, which is an SDR image, are regions where there is a difference in Y values ​​when comparing the degamma values ​​of each. Also, in Figure 5, the white areas of the gain map 1051 indicate regions where there is a difference in the Y components of the HDR image and the SDR image, while the shaded areas indicate regions where the Y components of the HDR image and the SDR image are equivalent.

[0045] The gain map generation unit 105 outputs the composite image data 1050 to the file output unit 106, selecting from the input composite image data 1040 and composite image data 1041 that match the image format of the composite image data described in the processing information 1020. The gain map generation unit 105 also outputs the gain map 1051 to the file output unit 106.

[0046] The file output unit 106 is a functional block that outputs an image file to the outside of the image processing device 10. The file output unit 106 outputs a file as a composite image file 1060, which is the input composite image data 1050 with a gain map 1051 added to it. Figure 6 is an explanatory diagram of the composite image file 1060 in Embodiment 1.

[0047] In Embodiment 1, an example was described in which the input image files are of the same format (HDR image) and the format (number of channels) of the gain map attached to each image file is also the same. However, the same applies when the input image files are of different formats (HDR image and SDR image) and the format (number of channels) of the gain map attached to each image file is different. In that case, a composite image is generated by combining the input HDR image and the HDR image converted from the input SDR image, a composite image is generated by combining the input SDR image and the SDR image converted from the input HDR image, and a gain map for the composite image is generated.

[0048] As described above, according to Embodiment 1, when synthesizing images to which gain maps have been added, it becomes possible to generate a gain map suitable for the synthesized image.

[0049] [Embodiment 2] Next, Embodiment 2 will be described with reference to Figures 7 to 10.

[0050] First, with reference to Figure 7, the configuration and functions of the image processing apparatus 20 of Embodiment 2 will be described.

[0051] Figure 7 is a block diagram illustrating the functional configuration of the image processing device 20 of Embodiment 2. Figure 8 is an explanatory diagram of the image file of Embodiment 2.

[0052] The image processing device 20 of Embodiment 2 receives multiple image files, each containing image data and a gain map applied to the image data, and performs a synthesis process on the input images to generate a composite image. The image processing device 20 then generates a gain map for the composite image and outputs an image file with the gain map applied to the composite image data.

[0053] The image processing device 20 includes an image input unit 201, a processing information input unit 202, an image conversion unit 203, an image synthesis unit 204, a gain map generation unit 205, and a file output unit 206.

[0054] The image input unit 201 is a functional block that inputs image files 2000 and 2001 from an external device such as a camera. Image file 2000 includes image data 20000 and a gain map 20001 attached to image data 20000. Image file 2001 includes image data 20010 and a gain map 20011 attached to image data 20010.

[0055] In Embodiment 2, image file 2000 is a file containing HDR image data and a gain map, and image file 2001 is a file containing SDR image data and a gain map. Gain map 20001 is a one-channel data with gain information for the Y component for the same number of pixels as the assigned image. Gain map 20011 is a three-channel data with gain information for the R, G, and B components for the same number of pixels as the assigned image.

[0056] The image input unit 201 outputs image data 20000 contained in image file 2000 and image data 20010 contained in image file 2001 to the image conversion unit 203 and the image synthesis unit 204. The processing information input unit 202 outputs gain map 20001 contained in image file 2000 and gain map 20011 contained in image file 2001 to the image conversion unit 203. The processing information input unit 202 also outputs information regarding the format of image data 20000, image data 20010, gain map 20001, and gain map 20011 as input file information 2010 to the gain map generation unit 205.

[0057] Figure 8 illustrates the image file 2000, image data 20000, gain map 20001, image file 2001, image data 20010, and gain map 20011 of Embodiment 2. In Figure 8, each region of the image data 20000 and image data 20010 shows that the closer to white the image, the higher the brightness, and the closer to black the image, the lower the brightness. In gain map 20001, the white areas indicate regions where there is a difference in the Y component between the HDR image and the SDR image, and the shaded areas indicate regions where the Y components of the HDR image and the SDR image are equivalent. Furthermore, gain map 20011 shows that it includes three gain maps for the R component, G component, and B component, and the white areas indicate regions where there is a difference in at least one of the R, G, and B components between the HDR image and the SDR image.

[0058] The processing information input unit 202 is a functional block that receives user instructions 2004 from the user specifying the processing content.

[0059] The processing information input unit 202 includes, for example, a display that shows the user the type of image synthesis process and the file format of the synthesized image after the synthesis process, as well as a mouse or touch panel that accepts user instructions. The user uses the processing information input unit 202 to specify the type of image synthesis process.

[0060] The processing information input unit 202 outputs processing information 2020 corresponding to the user instruction 2004 to the image synthesis unit 204.

[0061] The image conversion unit 203 is a functional block that uses image data and a gain map to acquire image data that corresponds to the given image data but has a different format. If the input image is an HDR image, it is converted to an SDR image, and if the input image is an SDR image, it is converted to an HDR image.

[0062] In this embodiment, image data 20000 is an HDR image, and gain map 20001 is one-channel data containing gain information for the Y component. In this embodiment, after degammaring of image data 20000, the Y component is converted for each pixel of the image based on the information described in the gain map, and a DR gamma is applied to generate converted image data 2030, which is an SDR image.

[0063] Furthermore, image data 20010 is an SDR image, and gain map 20011 is 3-channel data containing gain information for the R, G, and B components. In this embodiment, first, image data 20010 is degamma-processed, and then the YUV component of each pixel is converted to RGB components. Then, the R, G, and B components are converted for each pixel of the image based on the information described in the gain map. After the converted RGB components are converted back to YUV components, HDR gamma is applied to generate converted image data 2031, which is an HDR image.

[0064] The image conversion unit 203 outputs the generated converted image data 2030 and converted image data 2031 to the image synthesis unit 204. In this embodiment, each converted image data also includes information indicating which image data it was converted from.

[0065] Figure 9 is an explanatory diagram of the image conversion process in Embodiment 2.

[0066] In Figure 9, the areas where there is a difference in image color between the HDR image data 20000 and the converted SDR image data 2030 indicate the parts where the Y component changed during the conversion between the HDR and SDR images. The areas where there is a difference in image color between the SDR image data 20010 and the converted HDR image data 2031 indicate the parts where the Y component changed due to a change in at least one of the R, G, or B components during the conversion between the SDR and HDR images.

[0067] The image synthesis unit 204 is a functional block that generates a composite image based on the information described in processing information 2020, using the input image data 20000, image data 20010, converted image data 2030, and converted image data 2031.

[0068] The image synthesis unit 204 identifies images with the same format from the input image data 20000, image data 20010, converted image data 2030, and converted image data 2031. In this embodiment, the image format is identified as either an HDR image or an SDR image. In this embodiment, image data 20000 and converted image data 2031 are identified as having the same image format as HDR images, and image data 20010 and converted image data 2031 are identified as having the same image format as SDR images.

[0069] Furthermore, the image synthesis unit 204 selects an image synthesis method based on the information described in the processing information 2020. In this embodiment, comparative brightness synthesis is selected as the synthesis method, and each pixel of the images to be synthesized is compared, with the brighter pixels being adopted as pixels in the synthesized image. In this embodiment, the synthesized image is assumed to have the same resolution and image format as the images used for synthesis.

[0070] Furthermore, the image synthesis unit 204 compares the Y values ​​for each pixel at the same coordinate in both the image data 20000 and the converted image data 2031. It then selects the pixel value of the pixel with the higher Y value as the pixel value of that coordinate in the synthesized image, thereby generating the synthesized image data 2040, which is a synthesized HDR image.

[0071] Furthermore, the image synthesis unit 204 compares the Y values ​​for each pixel at the same coordinate in both the image data 20010 and the converted image data 2031. It then selects the pixel value of the pixel with the higher Y value as the pixel value of that coordinate in the synthesized image, thereby generating the synthesized image data 2041, which is a synthesized image of the SDR images.

[0072] Figure 10 is an explanatory diagram of the image synthesis process in Embodiment 2.

[0073] In Figure 10, each region of image data 20000, image data 20010, composite image data 2040, converted image data 2030, converted image data 2031, and composite image data 2041 shows that the closer to white the region, the higher the brightness, and the closer to black the region, the lower the brightness.

[0074] The image synthesis unit 204 outputs the generated synthesized image data 2040 and synthesized image data 2041 to the gain map generation unit 205.

[0075] The gain map generation unit 205 is a functional block that generates a gain map using the input composite image data 2040 and composite image data 2041, based on the information described in the input file information 2010 and processing information 2020.

[0076] In this embodiment, the processing information 2020 includes information that the image format of the composite image data to be output is HDR, and the gain map generation unit 205 selects an HDR image as the image format after the composite image. In this embodiment, the gain map generation unit 205 refers to the file information included in the input file information 2010 and selects a 1-channel data with gain information for the Y component as the format of the gain map to be generated, as the format of the gain map attached to the same image format as the image format after the composite image. In this embodiment, the method of selecting the image file format after the composite image is not limited to the above example. For example, an HDR image may be selected if higher image quality can be expected, or an SDR image may be selected if greater versatility can be expected. Furthermore, the method of selecting the format of the gain map to be generated is not limited to the above example. For example, among the input file formats, a 1-channel data with gain information for the Y component may be selected as the method that results in a smaller file size, or a 3-channel data with gain information for the R, G, and B components may be selected as the method that results in a larger file size, so that higher image quality can be expected.

[0077] The gain map generation unit 205 degammaifies the composite image data 2040, which is a composite image of HDR images, and the composite image data 2041, which is a composite image of SDR images, and then records the ratio of the Y component of each pixel, generating it as a gain map 2051.

[0078] The gain map generation unit 205 outputs the composite image data 2050 to the file output unit 206, selecting from the input composite image data 2040 and composite image data 2041 that match the image format of the composite image data described in the processing information 2020. The gain map generation unit 205 also outputs the gain map 2051 to the file output unit 206.

[0079] The file output unit 206 is a functional block that outputs an image file to the outside of the image processing device 20, and outputs a file as an image file 2060 in which a gain map 2051 has been added to the input composite image data 2050.

[0080] In Embodiment 2, an example was described where the input image file and the gain map are in different formats (an HDR image with a 1-channel gain map, and an SDR image with a 3-channel gain map). However, the same applies when the input image files are in the same format (HDR image or SDR image) but the gain map format is different. In that case, an SDR image or HDR image is generated from the input HDR image or SDR image, a composite image is generated by combining the input HDR images, a composite image is generated by combining the SDR images converted from the input HDR images, and a gain map for the composite image is generated.

[0081] As described above, according to Embodiment 2, when synthesizing images to which gain maps have been applied, it becomes possible to generate a gain map suitable for the synthesized image.

[0082] [Other embodiments] The present invention may be applied to a system consisting of multiple devices (such as a host computer, interface devices, imaging devices, and web applications), or to a device consisting of a single device.

[0083] This embodiment can also be implemented by supplying a program that implements one or more of the functions of the above-described embodiment to a system or device via a network or storage medium, and by having one or more processors in the computer of that system or device read and execute the program. It can also be implemented by a circuit (e.g., an ASIC) that implements one or more functions.

[0084] The invention is not limited to the embodiments described above, and various modifications and variations are possible without departing from the spirit and scope of the invention. Accordingly, claims are attached to disclose the scope of the invention.

[0085] The disclosures herein include the following image processing apparatus, image processing methods, and programs. [Item 1] Acquisition means for acquiring multiple first images and multiple second images which have a different format from the multiple first images and correspond to each of the multiple first images, A synthesis means that combines images with the same format from among the plurality of first images and the plurality of second images to generate a first composite image and a second composite image, An image processing apparatus characterized by comprising: generation means for generating composite image conversion information for mutually converting the first composite image and the second composite image from the first composite image and the second composite image. [Item 2] The image processing apparatus according to claim 1, wherein the acquisition means is characterized in that the plurality of first images are provided with conversion information used when generating images of different formats, and the plurality of second images are acquired by generating the plurality of second images of different formats from the plurality of first images. [Item 3] The image processing apparatus according to item 1 or 2, further comprising a selection means for selecting the format of the conversion information of the composite image. [Item 4] The image processing apparatus according to item 3, characterized in that the selection means selects the format of the conversion information of the composite image based on the format of the composite image. [Item 5] The image processing apparatus according to item 3, characterized in that the selection means selects the format that results in a larger file size from among the format of the conversion information attached to the first image and the format of the conversion information attached to the second image as the format of the conversion information of the composite image. [Item 6] The image processing apparatus according to item 3, characterized in that the selection means selects the format with the smaller file size from among the format of the conversion information attached to the first image and the format of the conversion information attached to the second image as the format of the conversion information of the composite image. [Item 7] The image processing apparatus according to item 3, characterized in that the selection means selects a format specified by the user as the format of the conversion information of the composite image. [Item 8] The image processing apparatus according to item 7, characterized in that the selection means selects the same format as the format of the composite image specified by the user as the format of the conversion information of the composite image. [Item 9] The image processing apparatus according to any one of items 3 to 8, characterized in that the format of the conversion information includes the number of channels corresponding to the color components. [Item 10] The format of the aforementioned multiple first images is the same, The image processing apparatus according to item 9, characterized in that the format of the conversion information attached to the plurality of first images has the same number of channels. [Item 11] The formats of the aforementioned multiple first images are different. The image processing apparatus according to item 9, characterized in that the format of the conversion information attached to the plurality of first images has a different number of channels. [Item 12] The aforementioned plurality of first images include third and fourth images having different formats. The image processing apparatus according to item 11, characterized in that the format of the conversion information assigned to the third image has a larger number of channels than the format of the conversion information assigned to the fourth image. [Item 13] The image processing apparatus according to item 12, characterized in that the format of the conversion information attached to the third image is 3 channels, and the format of the conversion information attached to the fourth image is 1 channel. [Item 14] The image processing apparatus according to any one of items 1 to 13, characterized in that the plurality of first images are HDR (High Dynamic Range) format images, and the plurality of second images are SDR (Standard Dynamic Range) format images. [Item 15] The image processing apparatus according to any one of items 1 to 13, characterized in that the plurality of first images are in SDR (Standard Dynamic Range) format, and the plurality of second images are in HDR (High Dynamic Range) format. [Item 16] The image processing apparatus according to any one of items 1 to 13, characterized in that the plurality of first images are HDR (High Dynamic Range) format images and SDR (Standard Dynamic Range) format images, and the plurality of second images are SDR format images generated from HDR format images among the first images and HDR format images generated from SDR format images. [Item 17] The image processing apparatus according to any one of items 1 to 13, characterized in that the conversion information is a gain map for mutually converting between an HDR (High Dynamic Range) format image and an SDR (Standard Dynamic Range) format image. [Item 18] An image processing method performed by an image processing device, The steps include obtaining a plurality of first images and a plurality of second images, each of which has a different format from the plurality of first images and is in a corresponding relationship to each of the plurality of first images, The steps include generating a first composite image and a second composite image by combining images with the same format from among the plurality of first images and the plurality of second images, An image processing method characterized by comprising the step of generating composite image conversion information for mutually converting between the first composite image and the second composite image from the first composite image and the second composite image. [Item 19] A program for causing a computer to function as one of the means of an image processing apparatus as described in any of items 1 through 17. [Explanation of Symbols]

[0086] 10, 20...Image processing unit, 101...Image input unit, 102...Processing information input unit, 103...Image conversion unit, 104...Image synthesis unit, 105...Gain map generation unit, 106...File output unit

Claims

1. Acquisition means for acquiring multiple first images and multiple second images which have a different format from the multiple first images and correspond to each of the multiple first images, A synthesis means that combines images with the same format from among the plurality of first images and the plurality of second images to generate a first composite image and a second composite image, An image processing apparatus characterized by having a generation means for generating composite image conversion information for mutually converting the first composite image and the second composite image from the first composite image and the second composite image.

2. The image processing apparatus according to claim 1, wherein the acquisition means is characterized in that the plurality of first images are provided with conversion information used when generating images of different formats, and the plurality of second images are acquired by generating the plurality of second images of different formats from the plurality of first images.

3. The image processing apparatus according to claim 1, further comprising a selection means for selecting the format of the conversion information of the composite image.

4. The image processing apparatus according to claim 3, wherein the selection means selects the format of the conversion information of the composite image based on the format of the composite image.

5. The image processing apparatus according to claim 3, characterized in that the selection means selects the format that results in a larger file size from among the format of the conversion information assigned to the first image and the format of the conversion information assigned to the second image as the format of the conversion information of the composite image.

6. The image processing apparatus according to claim 3, wherein the selection means selects the format with the smaller file size from among the format of the conversion information assigned to the first image and the format of the conversion information assigned to the second image as the format of the conversion information of the composite image.

7. The image processing apparatus according to claim 3, characterized in that the selection means selects a format specified by the user as the format of the conversion information of the composite image.

8. The image processing apparatus according to claim 7, characterized in that the selection means selects the same format as the format of the composite image specified by the user as the format of the conversion information of the composite image.

9. The image processing apparatus according to claim 3, characterized in that the format of the conversion information includes the number of channels corresponding to the color components.

10. The format of the aforementioned plurality of first images is the same, The image processing apparatus according to claim 9, characterized in that the format of the conversion information assigned to the plurality of first images has the same number of channels.

11. The formats of the aforementioned multiple first images are different. The image processing apparatus according to claim 9, characterized in that the format of the conversion information assigned to the plurality of first images has a different number of channels.

12. The aforementioned plurality of first images include third and fourth images having different formats. The image processing apparatus according to claim 11, characterized in that the format of the conversion information assigned to the third image has more channels than the format of the conversion information assigned to the fourth image.

13. The image processing apparatus according to claim 12, characterized in that the format of the conversion information assigned to the third image is 3 channels, and the format of the conversion information assigned to the fourth image is 1 channel.

14. The image processing apparatus according to claim 1, characterized in that the plurality of first images are in HDR (High Dynamic Range) format and the plurality of second images are in SDR (Standard Dynamic Range) format.

15. The image processing apparatus according to claim 1, characterized in that the plurality of first images are in SDR (Standard Dynamic Range) format and the plurality of second images are in HDR (High Dynamic Range) format.

16. The image processing apparatus according to claim 1, characterized in that the plurality of first images are HDR (High Dynamic Range) format images and SDR (Standard Dynamic Range) format images, and the plurality of second images are SDR format images generated from HDR format images among the first images and HDR format images generated from SDR format images.

17. The image processing apparatus according to claim 1, characterized in that the conversion information is a gain map for mutually converting between an image in HDR (High Dynamic Range) format and an image in SDR (Standard Dynamic Range) format.

18. An image processing method performed by an image processing device, The steps include obtaining a plurality of first images and a plurality of second images, each of which has a different format from the plurality of first images and is in a corresponding relationship to each of the plurality of first images, The steps include: generating a first composite image and a second composite image by combining images with the same format from among the plurality of first images and the plurality of second images; An image processing method characterized by comprising the step of generating composite image conversion information for mutually converting between the first composite image and the second composite image from the first composite image and the second composite image.

19. A program for causing a computer to function as one of the means of an image processing apparatus according to any one of claims 1 to 17.

Images