Image processing device, image processing method, and program

Abstract

To suppress wasteful generation of a first output image and a second output image with a poor reproductivity.SOLUTION: An image processor includes: an input unit for inputting a plurality of input images for a plurality of viewpoints; an image generation unit for generating, using the input images, a first output image and a second output image which at least partially reproduce the input images for each viewpoint of observing the first output image and the second output image in an overlapping state when changing the viewpoints; and a determination unit for determining the reproductivity of a reproduced image reproduced by the generated first output image and the generated second output image before the first output image and the second output image are generated.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present invention relates to an image processing device, an image processing method, and a program. [Background technology]

[0002] Light field printing is a well-known technology for producing printed materials in which the image observed changes depending on the viewpoint. For example, a technology is known in which a first target pattern and a second target pattern are generated based on content including multiple scene views, and the first target pattern is printed on a transparent front layer and the second target pattern is printed on a transparent back layer (see, for example, Patent Document 1). Summary of the Invention [Problem to be solved by the invention]

[0003] The technology disclosed in Patent Document 1 makes it possible to generate a first output image and a second output image that reproduce the input image corresponding to each viewpoint when the observation viewpoint is changed by superimposing the first output image and the second output image based on a plurality of input images corresponding to a plurality of viewpoints.

[0004] However, with this method, the reproducibility of the generated images, whose appearance changes depending on the viewpoint, cannot be determined until the first and second output images are generated, and therefore, first and second output images with poor reproducibility may be generated unnecessarily.

[0005] An embodiment of the present invention has been made in consideration of the above-mentioned problems, and prevents the unnecessary generation of a first output image and a second output image with poor reproducibility. [Means for solving the problem]

[0006] In order to solve the above problem, an image processing device according to one embodiment includes an input unit that inputs a plurality of input images corresponding to a plurality of viewpoints; an image generation unit that uses the plurality of input images to generate the first output image and the second output image, which at least partially reproduce the input image corresponding to each viewpoint when the viewpoint at which the first output image and the second output image are observed is changed by superimposing the first output image and the second output image; and a determination unit that determines, before generating the first output image and the second output image, reproducibility of a reproduction image reproduced by the generated first output image and the second output image. The determination unit compares the plurality of input images with a conversion profile of an input viewpoint image and a converted observation viewpoint image, which is prepared in advance for each conversion quality, and determines the conversion quality of the first output image and the second output image. do. [Effects of the Invention]

[0007] According to one embodiment of the present invention, it is possible to prevent the first output image and the second output image with poor reproducibility from being generated unnecessarily. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a diagram illustrating an example of a system configuration of an image processing system according to an embodiment. [Figure 2] 1A and 1B are diagrams illustrating a light field printed matter according to an embodiment. [Figure 3] FIG. 2 is a diagram illustrating an example of a hardware configuration of a computer according to an embodiment. [Figure 4] FIG. 1 is a diagram illustrating an example of a functional configuration of an image processing apparatus according to an embodiment. [Figure 5] 4 is a flowchart illustrating an example of processing performed by the image processing device according to the first embodiment. [Figure 6] FIG. 10 is a diagram illustrating an image of a plurality of input images according to an embodiment. [Figure 7] 10 is a flowchart illustrating an example of a determination process according to an embodiment. [Figure 8] FIG. 10 is a diagram illustrating an example of a method for dividing an input image according to an embodiment. [Figure 9] FIG. 10 is a diagram illustrating a comparison with a conversion profile according to an embodiment. [Figure 10]FIG. 10 is a diagram illustrating an example of a determination result according to an embodiment. [Figure 11] 10 is a flowchart showing an example of processing performed by an image processing device according to a second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

[0010] <System configuration> 1 is a diagram showing an example of the system configuration of an image processing system according to an embodiment. The image processing system 1 includes, for example, an image processing device 10 and a printing device 20. The printing device 20 is communicably connected to the image processing device 10 via, for example, a wired cable such as a USB (Universal Serial Bus) cable, short-range wireless communication, or a communication network such as a LAN (Local Area Network).

[0011] The image processing device 10 is, for example, an information processing device having a computer configuration, or a system including multiple computers. The image processing device 10 executes a predetermined program to generate a first output image 12A and a second output image 12B based on multiple input images 11-1, 11-2, ..., 11-N (N is an integer equal to or greater than 2) corresponding to multiple viewpoints.

[0012] The first output image 12A and the second output image 12B generated by the image processing device 10 at least partially reproduce the input image corresponding to each viewpoint when the first output image 12A and the second output image 12B are superimposed and observed from different viewpoints. For example, the image processing device 10 can generate a desired light field printout by using the printing device 20 to print the first output image 12A and the second output image 12B on both sides of a transparent transfer medium such as a transparent sheet.

[0013] The multiple input images 11-1, 11-2, ..., 11-N are image data corresponding to multiple viewpoints. The multiple input images 11-1, 11-2, ..., 11-N are prepared, for example, by a user so that a desired light field printout can be obtained. In the following description, the multiple input images 11-1, 11-2, ..., 11-N may be simply referred to as "input images 11."

[0014] The image processing device 10 generates a first output image 12A and a second output image 12B by solving a predetermined optimization problem based on a plurality of input images 11-1, 11-2, ..., 11-N. The first output image 12A and the second output image 12B are unique pattern images obtained by solving the optimization problem, and a meaningful image (light field image) is reproduced by superimposing the first output image 12A and the second output image 12B.

[0015] In response to a printing command from the image processing device 10, the printing device 20 forms a first output image 12A and a second output image 12B on both sides of a transparent transfer medium 13C, thereby outputting a light field print 13. The light field print 13 includes a transparent transfer medium 13C, a front image layer 13A, and a rear image layer 13B arranged substantially parallel to and spaced apart from each other. The light field print 13 is a so-called light field print, in which the image observed changes depending on the viewing point.

[0016] When the multiple input images 11 represent images of a specific subject as observed from multiple viewpoints, the light field print 13 is a print that represents the subject in three dimensions. However, the light field print 13 according to this embodiment is not limited to one that provides a stereoscopic view. For example, when the multiple input images represent different subjects, the light field print 13 may be a print in which the subject observed changes when the viewpoint is changed.

[0017] 2 is a diagram illustrating a light field printed matter according to one embodiment. In the light field printed matter 13, a front image layer 13A based on a first output image 12A is formed on the front surface of a transparent transfer medium 13C, and a back image layer 13B based on a second output image 12B is formed on the back surface. Furthermore, because the front and back surfaces of the transparent transfer medium 13C are flat and spaced apart, the front image layer 13A and the back image layer 13B are arranged substantially parallel and spaced apart.

[0018] Image formation by the printing device 20 according to this embodiment is performed by applying and fixing a color developer such as toner or ink to both sides of the transparent transfer medium 13C. By forming images on both sides, the work of laminating transparent layers, which was performed in the prior art of Patent Document 1, is no longer necessary, and it is possible to easily obtain a light field print 13. In addition, it is possible to prevent image degradation due to the infiltration of materials such as air or adhesive between the laminated layers.

[0019] The light field print 13 is based on the principle that the thickness of the transparent transfer medium 13C results in different pixel combinations passing through the front image layer 13A and the back image layer 13B depending on the viewpoint. By utilizing the different pixel combinations depending on the viewpoint, it is possible to change the light intensity for each viewpoint. This allows for the creation of a light field print 13 in which different images are seen from each viewpoint. These images seen from multiple viewpoints are reproduced images by light field printing. The reproduced images are approximations of the input image 11 determined for each angle of the viewpoint relative to the surface of the transfer medium. When the observer changes the viewpoint from which they observe the light field print 13, they can view each approximation of the input image 11 corresponding to the viewpoint. Note that the positional relationship between pixels in the combination of pixels through which the line of sight passes defines the viewpoint.

[0020] The transparent transfer medium 13C is a medium that transmits visible light and has two flat transfer surfaces, and in addition to being transparent, it may also be semi-transparent to the extent that the printed image on the back side can be seen through. This is because even if the transparent transfer medium 13C is semi-transparent rather than completely transparent, the intensity of light passing through the front image layer 13A and the back image layer 13B changes depending on the viewpoint.

[0021] With the system configuration shown in Figure 1, the image processing system 1 can generate a light field print 13 by converting multiple input images 11 into a first output image 12A and a second output image 12B and printing them on both sides using a printing device 20.

[0022] However, with conventional technology, the reproducibility of the generated images, whose appearance changes depending on the viewpoint, cannot be determined until the first output image 12A and the second output image 12B are generated, and therefore, first output images and second output images with poor reproducibility may be generated unnecessarily.

[0023] Therefore, the image processing system 1 according to this embodiment has a function of determining the reproducibility of the generated images before generating the first output image 12A and the second output image 12B. For example, the image processing device 10 compares a plurality of input images 11 with conversion profiles of input viewpoint images and converted observation viewpoint images, which are prepared in advance for each conversion quality, to determine the conversion quality of the first output image 12A and the second output image 12B.

[0024] As a result, the image processing system 1 according to this embodiment can prevent the first output image 12A and the second output image 12B, which have poor reproducibility, from being generated unnecessarily and output (printed) unnecessarily.

[0025] <Hardware configuration> (Hardware configuration of image processing device) The image processing device 10 has, for example, the hardware configuration of a computer 300 as shown in Fig. 3. Alternatively, the image processing device 10 is configured by a plurality of computers 300.

[0026] Fig. 3 is a diagram showing an example of the hardware configuration of a computer according to an embodiment. As shown in Fig. 3, the computer 300 includes, for example, a central processing unit (CPU) 301, a read-only memory (ROM) 302, a random access memory (RAM) 303, a hard disk (HD) 304, a hard disk drive (HDD) controller 305, a display 306, an external device connection interface (I / F) 307, a network I / F 308, a keyboard 309, a pointing device 310, a digital versatile disk rewritable (DVD-RW) drive 312, a media I / F 314, and a bus line 315.

[0027] Of these, the CPU 301 controls the overall operation of the computer 300. The ROM 302 stores programs used to start up the computer 300, such as an IPL (Initial Program Loader). The RAM 303 is used, for example, as a work area for the CPU 301. The HD 304 is an example of a storage device that stores programs such as an OS (Operating System), applications, and device drivers, as well as various data. The HDD controller 305 controls the reading and writing of various data from and to the HD 304, for example, under the control of the CPU 301.

[0028] The display 306 displays various types of information such as a cursor, menus, windows, characters, or images. The display 306 may be provided external to the computer 300. The external device connection I / F 307 is an interface such as a USB (Universal Serial Bus) for connecting various external devices such as the printing device 20 to the computer 300. The network I / F 308 is an interface for communicating with other devices using a communication network, for example.

[0029] The keyboard 309 is a type of input means having multiple keys for inputting characters, numbers, various instructions, etc. The pointing device 310 is a type of input means for selecting and executing various instructions, selecting a processing target, moving a cursor, etc. The keyboard 309 and pointing device 310 may be provided external to the computer 300. The DVD-RW drive 312 controls reading and writing of various data from a DVD-RW 311, which is an example of a removable storage medium. The DVD-RW 311 is not limited to a DVD-RW, and may be another storage medium.

[0030] A media I / F 314 controls reading and writing (storing) of data from and to a medium 313 such as a flash memory. A bus line 315 includes an address bus, a data bus, and various control signals for electrically connecting the above components.

[0031] 3 is an example of the hardware configuration of the computer 300. The computer 300 may have any configuration as long as it has, for example, a CPU 301, a ROM 302, a RAM 303, a network I / F 308, and a bus line 316.

[0032] <Functional configuration> (Functional configuration of image processing device) 4 is a diagram showing an example of the functional configuration of an image processing device according to an embodiment. The image processing device 10, for example, realizes an input unit 401, an image generation unit 402, a determination unit 403, a notification unit 404, a reception unit 405, an output unit 406, a storage unit 407, etc. by the CPU 301 executing a predetermined program. Note that at least a part of the above functional configurations may be realized by hardware.

[0033] The input unit 401 executes input processing for inputting a plurality of input images 11 corresponding to a plurality of viewpoints to the image processing device 10. For example, the input unit 401 receives input of the plurality of input images 11 from the network I / F 308, an external memory connected to the external device connection I / F 307, the medium 313, or the like.

[0034] The plurality of input images 11 preferably includes three or more input images 11-1 to 11-N (N≧3) corresponding to three or more viewpoints. Each of the input images 11-1 to 11-N has a predetermined viewpoint (1, 2, ... x, ... N) set thereto, and includes viewpoint information describing the correspondence between each viewpoint and the input images 11-1 to 11-N. The viewpoint information may be provided separately from the data of the plurality of input images 11, or may be embedded in file names according to a predetermined rule (such as sequential numbers) and provided together with the data of each input image 11-1 to 11-N. In the latter case, the input unit 401 can interpret the correspondence with the viewpoint from the file names.

[0035] The multiple input images 11 define the desired image when observed from each of multiple viewpoints, and are prepared by the user so that the desired light field printout can be obtained. The input unit 401 passes the multiple input images 11 input by the user to, for example, the image generation unit 402 and the determination unit 403.

[0036] The image generation unit 402 executes a generation process to generate, using a plurality of input images 11, a first output image 12A and a second output image 12B that at least partially reproduce the input images 11 corresponding to each viewpoint when the input images 11 are superimposed and observed from different viewpoints.

[0037] For example, the image generation unit 402 generates a set of a first output image 12A and a second output image 12B to be printed on both sides of a transparent transfer medium through optimization processing based on a plurality of input images 11. The optimization processing is performed so that when a front image layer and a back image layer based on the first output image 12A and the second output image 12B are formed on both sides of the transparent transfer medium, the input image 11 corresponding to each viewpoint is at least partially reproduced when the observation viewpoint is changed.

[0038] Normally, when the image generation unit 402 generates the first output image 12A and the second output image 12B based on the multiple input images 11, the input images 11 are compressed to reduce the amount of information. For this reason, the input images 11 are approximately reproduced in the optimization process.

[0039] The optimization process optimizes a set of output images based on a plurality of input images 11 so as to minimize the difference between a reproduced image reproduced at each viewpoint using a plurality of image layers spaced apart from one another, which is a set of tentative output images under trial, and the input image corresponding to each viewpoint. At this stage, the optimization calculation does not take into account the fact that the plurality of image layers spaced apart are printed on both sides of the transfer medium, and instead minimizes the error between the image reproduced using the plurality of image layers spaced apart (hereinafter referred to as the reproduced image) and the input image. The error between the reproduced image and the input image is the sum of the differences across the plurality of viewpoints, which serves as the objective function. The optimization process can be performed using known techniques such as gradient descent.

[0040] Furthermore, the image generation unit 402 performs an inversion process on one of the pair of output images generated by the optimization process, inverting the image in accordance with the opening direction of double-sided printing, thereby enabling the image generation unit 402 to generate the final first output image 12A and second output image 12B.

[0041] Before the image generation unit 402 generates the first output image 12A and the second output image 12B, the determination unit 403 executes a determination process to determine the reproducibility of the reproduced images reproduced by the generated first output image 12A and the second output image 12B. For example, the determination unit 403 compares the multiple input images 11 with conversion profiles of input viewpoint images and converted observation viewpoint images prepared in advance for each conversion quality, and determines the conversion quality of the first output image 12A and the second output image 12B. The determination process executed by the determination unit 403 will be described later with a specific example.

[0042] The notification unit 404 executes a notification process to notify the user of the determination result by the determination unit 403. For example, the notification unit 404 displays a notification indicating the conversion quality (e.g., "good," "medium," "poor," etc.) when the first output image 12A and the second output image 12B are generated using the input multiple input images 11 on a display unit such as the display 306. However, the determination result by the determination unit 403 is not limited to the conversion quality, and may be, for example, a numerical value indicating whether the reproducibility is good or bad, or the quality of the reproducibility. Furthermore, the notification unit may notify the user of the determination result by the determination unit 403 by voice, a text message, etc.

[0043] This notification allows the user to cancel the generation of the first output image 12A and the second output image 12B, for example, if the reproducibility of the reproduced images reproduced by the first output image 12A and the second output image 12B generated by the image generation unit 402 is poor. Also, if the reproducibility of the reproduced images cannot be said to be either good or bad (for example, if the conversion quality is "medium"), the user can determine whether to generate the first output image 12A and the second output image 12B depending on the presence or absence of other input images 11, the urgency, the importance, etc. As a result, the image processing system 1 according to this embodiment can prevent the wasteful generation of first output images and second output images with poor reproducibility.

[0044] The reception unit 405 executes a reception process for receiving a user's input operation in response to the determination result notified by the notification unit 404, using an input device such as the keyboard 309 or the pointing device 310. For example, the reception unit 405 receives a user's permission operation for permitting the generation of the first output image 12A and the second output image 12B, or an execution operation for requesting the execution of printing of the light field printout 13.

[0045] When the receiving unit 405 receives an authorization operation that permits the generation of the first output image 12A and the second output image 12B, or an execution operation that requests the printing of the light field print 13, the image generation unit 402 executes the image generation process. On the other hand, when the receiving unit 405 receives a cancellation operation that requests the cancellation of the generation of the first output image 12A and the second output image 12B, or a cancellation operation that requests the cancellation of the printing of the light field print 13, the image generation unit 402 cancels the image generation process. Note that if the receiving unit 405 does not receive an authorization operation or an execution operation within a predetermined time, the image generation unit 402 may, for example, cancel the image generation process.

[0046] The output unit 406 outputs a print instruction including print data to the printing device 20 based on the first output image 12A and the second output image 12B generated by the image generation unit 402. This print instruction includes print setting information such as the designation of the paper feed tray in which the transparent transfer medium is set, the designation of double-sided printing, etc.

[0047] The memory unit 407 is realized, for example, by a program executed by the CPU 301, the HD 304, the HDD controller 305, etc., and stores various data, information, programs, etc., including the conversion profiles for each conversion quality and multiple input images 11 described above.

[0048] With the above-described functional configurations, the image processing device 10 can prevent the first output image and the second output image with poor reproducibility from being generated unnecessarily.

[0049] As another example, the image processing device 10 may generate the first output image 12A and the second output image 12B and the output unit 406 may suspend output of the print instruction during the period from when the notification unit 404 notifies the determination result until when the reception unit 405 receives a user operation. In this case, the image processing device 10 can promptly output the print instruction when it receives an authorization operation permitting generation of the first output image 12A and the second output image 12B, and can prevent unnecessary printing of light field printouts with poor reproducibility.

[0050] <Processing flow> Next, the processing flow of the image processing method according to this embodiment will be described.

[0051] [First embodiment] (Image processing device processing) 5 is a flowchart showing an example of processing performed by the image processing device according to the first embodiment. This processing is an example of processing performed by the image processing device 10 having the functional configurations described in FIG.

[0052] In step S501, the input unit 401 receives input of a plurality of input images corresponding to a plurality of viewpoints.

[0053] Fig. 6 is a diagram showing an image of a plurality of input images according to one embodiment. For example, if the number of viewpoints in the horizontal direction (X direction) is X and the number of viewpoints in the vertical direction (Y direction) is Y, then the number of multiple input images 600 is X × Y, as shown in Fig. 6. Here, among the multiple input images 600, an input image at each viewpoint is represented by input image Iab (1 ≦ a ≦ Y, 1 ≦ b ≦ X), as shown in Fig. 6.

[0054] In step S502, the determination unit 403 executes a determination process to determine the reproducibility of the reproduction images reproduced from the first output image 12A and the second output image 12B generated by the image generation unit 402. The determination process executed by the determination unit 403 will be described later using another flowchart.

[0055] In step S503, the notification unit 404 notifies the user of the determination result made by the determination unit 403.

[0056] In step S504, the receiving unit 405 determines whether or not it has received a permission operation from the user to permit generation of the first output image 12A and the second output image 12B. If the permission operation has been received, the receiving unit 405 shifts the process to step S505. On the other hand, if the permission operation has not been received, the receiving unit 405 ends the process of FIG. 5.

[0057] For example, the receiving unit 405 may determine that the permission operation has not been accepted when the receiving unit 405 has accepted a refusal operation by the user to refuse generation of the first output image 12A and the second output image 12B. Furthermore, the receiving unit 405 may determine that the permission operation has not been accepted when the receiving unit 405 has not accepted the permission operation for a predetermined time or longer.

[0058] In step S505, the image generation unit 402 generates a first output image 12A and a second output image 12B based on the plurality of input images 600 received from the input unit 401.

[0059] In step S506, the output unit 406 instructs the printing device 20 to perform light field printing based on the first output image 12A and the second output image 12B generated by the image generation unit 402.

[0060] 5, the image processing device 10 determines the reproducibility of the reproduced image and notifies the user before the image generation unit 402 generates the first output image 12A and the second output image 12B. Furthermore, the image processing device 10 does not generate the first output image 12A and the second output image 12B until it receives an authorization operation from the user, thereby preventing the first output image 12A and the second output image 12B with poor reproducibility from being generated unnecessarily.

[0061] Note that the processing of the image processing device shown in Fig. 5 is an example. For example, in step S504 in Fig. 5, when the receiving unit 405 receives an execution operation requesting the execution of light field printing instead of (or in addition to) the permission operation, the processing may proceed to step S505.

[0062] Furthermore, in this case, the image processing device 10 may execute the process of step S505 while waiting for a user operation, and may execute the process of step S506 when the execution operation is accepted. This allows the image processing device 10 to prevent the first output image 12A and the second output image 12B, which have poor reproducibility, from being printed unnecessarily.

[0063] In short, the image processing device 10 is required to determine the reproducibility of the reproduced image and notify the user before generating the first output image 12A and the second output image 12B, and to suspend the execution of the image generation process or output process until approval is obtained from the user.

[0064] (Determination process) 7 is a flowchart showing an example of the determination process according to an embodiment, which is executed by the determination unit 403 in step S502 in FIG.

[0065] In step S701, the determination unit 403 divides the input image Iab at each viewpoint into a plurality of regions. For example, the horizontal resolution of each input image Iab is set to k1X and the vertical resolution is set to k2Y. In this case, the determination unit 403 divides the input image Iab at each viewpoint into a plurality of regions Iabij (1≦i≦k2, 1≦j≦k1) as shown in FIG. 8, for example, according to the number of viewpoints X in the horizontal direction and the number of viewpoints Y in the vertical direction. Since each divided region is sufficiently small, it can be considered that the pixel values ​​within the region are nearly uniform.

[0066] In step S702, the determination unit 403 calculates the average pixel value within each divided region.

[0067] In step S703, the determination unit 403 compares the calculated average pixel values ​​within the region with conversion profiles prepared in advance for each conversion quality to determine the conversion quality of the generated first output image 12A and second output image 12B. Note that the determination of the conversion quality of the first output image 12A and the second output image 12B is an example of determining the reproducibility of the reproduced image reproduced by the first output image 12A and the second output image 12B. For example, if the conversion quality of the first output image 12A and the second output image 12B is good, the reproducibility of the reproduced image reproduced by the first output image 12A and the second output image 12B will be high.

[0068] 9 is a diagram for explaining a comparison with a conversion profile according to an embodiment. The image processing device 10 stores, in the storage unit 407 or the like, conversion profiles (A-1), (A-2), (A-3), and (A-4) prepared in advance for each conversion quality, as shown in FIG.

[0069] 8, the determination unit 403 creates a graph 901 that represents the relationship between the viewpoint number and the average pixel value within a region, where Y=1 and the value of X is the viewpoint number 1, 2, . . . , X, among multiple regions Iabij obtained by dividing the input image Iab at each viewpoint. In this case, viewpoint number 1 represents region I11ij, viewpoint number 2 represents region I12ij, and viewpoint number X represents region I1Xij.

[0070] The determination unit 403 compares the created graph 901 with the prepared conversion profiles (A-1), (A-2), (A-3), and (A-4) to determine the conversion quality of the areas I11ij, I12ij, . . . I1Xij.

[0071] The conversion profile indicates the relationship between the average pixel values ​​within the region of the input viewpoint image at each viewpoint and the converted observation viewpoint image. As shown in FIG. 9, the conversion profile includes average data of pixel values ​​within the region of the input viewpoint image and the observation viewpoint image corresponding to each viewpoint number. In the example of FIG. 9, conversion profiles (A-1) and (A-2) indicate examples of conversion profiles when the conversion quality is "good." Furthermore, conversion profile (B) indicates an example of a conversion profile when the conversion quality is "medium," and conversion profile (C) indicates an example of a conversion profile when the conversion quality is "low."

[0072] For example, when determining the conversion quality of area I11ij, the determination unit 403 compares the average pixel values ​​within the area of ​​the input image at viewpoint number 1 of graph 901 with the average pixel values ​​within the area at viewpoint number 1 of multiple conversion profiles, and determines the conversion quality of the conversion profile with the closest value as the determination result. Similarly, the determination unit 403 determines the conversion quality for other areas I12ij,...I1Xij at viewpoint numbers 2,...,X.

[0073] Furthermore, the determination unit 403 determines the conversion quality of each region in the same manner for Y=2 and thereafter, thereby generating, for example, a determination result 1000 of the input image Iab as shown in FIG.

[0074] 5, the notification unit 404 notifies the user of the determination result 1000 shown in FIG. 10 for each input image Iab as the determination result by the determination unit 403. However, without being limited to this, the notification unit 404 may notify the user of, for example, information about input images Iab with poor conversion quality as the determination result by the determination unit 403. In this case, the notification unit 404 may determine that an input image Iab in which the proportion of areas with "bad" conversion quality is equal to or greater than a threshold is an input image Iab with poor conversion quality.

[0075] [Second embodiment] (Image processing device processing) Fig. 2 is a flowchart showing an example of processing by an image processing device according to the second embodiment. This processing shows another example of processing executed by, for example, the image processing device 10 having the functional configurations described in Fig. 4. Note that the basic processing content is similar to the processing by the image processing device according to the first embodiment described in Fig. 5, and therefore detailed description of the processing content similar to that of the first embodiment will be omitted here.

[0076] In step S1101, the input unit 401 receives input of a plurality of input images 600 corresponding to a plurality of viewpoints, as shown in FIG.

[0077] In step S1102, the determination unit 403 executes a determination process to determine the reproducibility of the reproduced images reproduced from the first output image 12A and the second output image 12B generated by the image generation unit 402. For example, the determination unit 403 executes the determination process shown in Fig. 7 for each of the input images Iab, and obtains a determination result 1000 shown in Fig. 10.

[0078] In step S1103, the determination unit 403 determines whether the reproducibility of the reproduced image is good. For example, the determination unit 403 may determine that the reproducibility of the input image ab is good if the proportion of areas with "bad" conversion quality in the determination result 1000 shown in Fig. 10 is equal to or less than a predetermined first threshold. Furthermore, the determination unit 403 may determine that the reproducibility of the reproduced images reproduced by the generated first output image 12A and second output image 12B is good if the proportion of the input image ab with good reproducibility is equal to or greater than a predetermined second threshold.

[0079] If the reproducibility is good, the determination unit 403 shifts the process to step S1104. On the other hand, if the reproducibility is not good, the determination unit 403 shifts the process to step S1106.

[0080] In step S1104, the image generation unit 402 generates a first output image 12A and a second output image 12B based on the plurality of input images 600 received from the input unit 401.

[0081] In step S1105, the output unit 406 instructs the printing device 20 to perform light field printing based on the first output image 12A and the second output image 12B generated by the image generation unit 402.

[0082] On the other hand, when the process proceeds from step S1103 to step S1106, the notification unit 404 notifies the user with a message or the like urging the user to change or correct the input image 600. Preferably, this notification includes, for example, information about the input image Iab with poor reproducibility (poor conversion quality). This makes it easy for the user to change or correct the input image Iab with poor reproducibility.

[0083] In the second embodiment, as in the first embodiment, it is possible to prevent the first output image 12A and the second output image 12B, which have poor reproducibility, from being generated unnecessarily and to prevent the first output image 12A and the second output image 12B from being printed unnecessarily.

[0084] As described above, according to each embodiment of the present invention, it is possible to prevent the first output image 12A and the second output image 12B with poor reproducibility from being needlessly generated.

[0085] <Supplementary information> Each function of each embodiment described above can be realized by one or more processing circuits. Here, the term "processing circuit" in this specification includes a processor programmed to perform each function by software, such as a processor implemented by an electronic circuit, as well as devices such as an ASIC (Application Specific Integrated Circuit), a DSP (Digital Signal Processor), an FPGA (Field Programmable Gate Array), and conventional circuit modules designed to perform each of the functions described above.

[0086] The functional configuration of the image processing device 10 shown in Fig. 4 is an example. For example, the functional configurations of the image processing device 10 in Fig. 4 may be distributed among multiple computers 300. The storage unit 407 may be realized by, for example, a storage server or a cloud service that can communicate via a communication network. Furthermore, the image generation unit 402 may generate the first output image 12A and the second output image 12B by using, for example, a cloud service that can communicate via a communication network. [Explanation of symbols]

[0087] 1. Image processing system 10 Image processing device 11,600 multiple input images 12A First output image 12B Second output image 300 Computers 401 Input section 402 Image Generation Unit 403 Judgment section 404 Notification Department A-1, A-2, B, C conversion profiles [Prior art documents] [Patent documents]

[0088] [Patent Document 1] Special Publication No. 2018-537046

Claims

1. an input unit that inputs a plurality of input images corresponding to a plurality of viewpoints; an image generation unit that generates, using the plurality of input images, the first output image and the second output image, which at least partially reproduce the input image corresponding to each viewpoint when the viewpoint from which the first output image and the second output image are observed is changed by superimposing the first output image and the second output image; a determination unit that determines, before generating the first output image and the second output image, reproducibility of a reproduction image reproduced by the generated first output image and the generated second output image; and The determination unit compares the multiple input images with conversion profiles of input viewpoint images and converted observation viewpoint images prepared in advance for each conversion quality, and determines the conversion quality of the first output image and the second output image.

2. The determination unit Dividing the input image into a plurality of regions; The image processing device according to claim 1 , wherein for each of the regions, an average of pixel values ​​within the region is compared with the conversion profile to determine the conversion quality of the region.

3. The image processing device according to claim 2 , wherein the conversion profile includes average data of pixel values ​​within the region of the input viewpoint image and the observation viewpoint image.

4. The image processing device according to claim 1 , further comprising a notification unit that notifies a user of a result of the determination made by the determination unit.

5. The image processing device according to claim 4 , wherein, when an enabling operation or an execution operation by the user in response to the determination result is accepted, the image generating unit generates the first output image and the second output image.

6. The image processing device according to claim 4 , wherein the image generating unit does not generate the first output image and the second output image when an authorization operation or an execution operation by the user in response to the determination result is not accepted.

7. The image processing device according to claim 1 , wherein the image generating section stops generating the first output image and the second output image when the reproducibility of the reproduced image is not good.

8. an input process for inputting a plurality of input images corresponding to a plurality of viewpoints; an image generation process for generating, using the plurality of input images, the first output image and the second output image, which at least partially reproduce the input image corresponding to each viewpoint when the viewpoint from which the first output image and the second output image are observed is changed by superimposing the first output image and the second output image; a determination process for determining reproducibility of images reproduced by the first output image and the second output image after generation, before the first output image and the second output image are generated; The computer executes The determination process is an image processing method in which the plurality of input images are compared with conversion profiles of input viewpoint images and converted observation viewpoint images prepared in advance for each conversion quality to determine the conversion quality of the first output image and the second output image.

9. an input process for inputting a plurality of input images corresponding to a plurality of viewpoints; an image generation process for generating, using the plurality of input images, the first output image and the second output image, which at least partially reproduce the input image corresponding to each viewpoint when the viewpoint from which the first output image and the second output image are observed is changed by superimposing the first output image and the second output image; a determination process for determining reproducibility of images reproduced by the first output image and the second output image after generation, before the first output image and the second output image are generated; on the computer, The determination process is a program that compares the multiple input images with conversion profiles of input viewpoint images and converted observation viewpoint images prepared in advance for each conversion quality, and determines the conversion quality of the first output image and the second output image.

Images