Image processing device, image processing method, and image processing program

Abstract

According to an embodiment, this video processing device comprises an image acquiring unit, an optimization processing unit, a pattern generating unit, and an image generating unit. The image acquiring unit acquires a left-eye viewpoint image obtained by imaging a display area from a left-eye position, a right-eye viewpoint image obtained by imaging the display area from a right-eye position, and a center viewpoint image obtained by imaging the display area from a center position between the left-eye position and the right-eye position. The optimization processing unit optimizes, on the basis of the left-eye viewpoint image and the right-eye viewpoint image, a phase shift amount and a weight both calculated on the basis of the center viewpoint image. The pattern generating unit generates, on the basis of the optimized phase shift amount and the optimized weight, a parallax induction pattern corresponding to the parallax between the left-eye position and the right-eye position. The image generating unit generates stereo-paired images on the basis of the center viewpoint image and the parallax induction pattern.

Description

[Technical field]

[0001] The present invention relates to a video processing device, a video processing method, and a video processing program. [Background technology]

[0002] The "HiddenStereo method" is known as a stereo image generation technology that applies the visual mechanism that works when humans perceive depth. In the HiddenStereo method, a parallax-inducing pattern with a phase difference of 90 degrees from a reference image is generated, and the stereo pair image created by adding or subtracting it from the reference image is displayed on a 3D display. Using this method, users wearing 3D glasses can perceive a 3D image through binocular stereoscopic vision, while users without 3D glasses can perceive a 2D image (the reference image mentioned above) without ghosts or double images.

[0003] However, in this method, the phase difference between the reference image and the parallax induction pattern is fixed at 90 degrees, so the left and right parallax induction amounts for the reference image are always equal. As a result, when the parallax is symmetrical, for example, for an object in front of the user, accurate depth can be reproduced, but when the parallax is asymmetrical, for example, for an object in a position horizontally away from the user, accurate depth reproduction is difficult. This phenomenon is particularly noticeable for objects at the edge of the screen in a large 3D display. Therefore, there is a demand for a stereo image generation technology that can properly reproduce depth even when the parallax is asymmetrical. [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] ACM Transactions on Graphics, (US), July 2017, Vol. 36, No. 4, Article 147 Summary of the Invention [Problem to be solved by the invention]

[0005] An object of the present invention is to provide an image processing device, an image processing method, and an image processing program that are capable of appropriately reproducing depth even when parallax is asymmetric. [Means for solving the problem]

[0006] According to an embodiment, a video processing device includes an image acquisition unit, an optimization processing unit, a pattern generation unit, and an image generation unit. The image acquisition unit acquires a left-eye viewpoint image obtained by capturing a display area from a left eye position, a right-eye viewpoint image obtained by capturing a display area from a right eye position, and an intermediate viewpoint image obtained by capturing a display area from an intermediate position between the left eye position and the right eye position. The optimization processing unit optimizes the phase shift amount and weight calculated based on the intermediate viewpoint image based on the left eye viewpoint image and the right eye viewpoint image. The pattern generation unit generates a parallax-inducing pattern corresponding to the parallax between the left eye position and the right eye position based on the optimized phase shift amount and the optimized weight. The image generation unit generates a stereo pair image based on the intermediate viewpoint image and the parallax-inducing pattern. Effect of the Invention

[0007] According to the embodiments, it is possible to provide a video processing device, a video processing method, and a video processing program that are capable of appropriately reproducing depth even when the parallax is asymmetric. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is an explanatory diagram illustrating parallax occurring at a user's viewpoint when a 3D object is reproduced using a video processing device according to an embodiment. [Diagram 2] FIG. 2 is a diagram illustrating an example of the configuration of the video processing device according to the embodiment. [Diagram 3] FIG. 3 is a diagram illustrating an example of a functional configuration of the video processing device according to the embodiment. [Figure 4]FIG. 4 is an explanatory diagram illustrating an example of a process in which the video processing device according to the embodiment generates a stereo pair of images. [Diagram 5] FIG. 5 is a flowchart illustrating an example of processing executed by the video processing device according to the embodiment. [Figure 6] FIG. 6 is an explanatory diagram illustrating an example of screen division when a video processing device according to a modification of the embodiment generates a stereo pair image. [Figure 7] FIG. 7 is an explanatory diagram illustrating an example of a process in which a video processing device according to a modification of the embodiment generates a stereo pair image. [Figure 8] FIG. 8 is a flowchart illustrating an example of processing executed by a video processing device according to a modified example of the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] An embodiment of the present invention will be described in detail with reference to the drawings as appropriate.

[0010] FIG. 1 is an explanatory diagram for explaining parallax occurring at a user's viewpoint when a 3D object is reproduced using a video processing device according to an embodiment. In FIG. 1, a depth direction (direction indicated by arrows Y1 and Y2) and a horizontal direction (direction indicated by arrows X1 and X2) are defined. The depth direction intersects with the vertical direction (perpendicular or substantially perpendicular). The horizontal direction intersects with both the depth direction and the vertical direction (perpendicular or substantially perpendicular). In FIG. 1, the user is located on the near side (arrow Y2 side) of the real display area RS in the depth direction. Therefore, the user perceives the 3D object by viewing the display area RS from the near side. At this time, the depth width reproduced by the 3D object is set as a distance D. The distance D is the distance in the depth direction between the real display area RS and the virtual display surface VS. In addition, in FIG. 1, a left eye position PL, a right eye position PR, and an intermediate position PC are defined. The left eye position PL is the position of the assumed viewpoint corresponding to the left eye among the assumed viewpoints of both eyes of the user perceiving the 3D object. The right eye position PR is the position of the assumed viewpoint corresponding to the right eye among the assumed viewpoints of both eyes of the user perceiving a 3D object. The intermediate position PC is a position in the horizontal direction intermediate between the left eye position PL and the right eye position PR.

[0011] For example, when a user perceives a 3D object by viewing the viewing area DL, the user perceives a virtual point VL on a virtual display surface VS at a distance D away. At this time, as shown in the enlarged view of the viewing area DL, a parallax W1 along the horizontal direction occurs for the user when the image of the display area RS is perceived at the left eye position PL and when the image of the display area RS is perceived at the intermediate position PC. Similarly, a parallax W2 along the horizontal direction occurs for the user when the image of the display area RS is perceived at the right eye position PR and when the image of the display area RS is perceived at the intermediate position PC. In this example, as shown in the enlarged view of the viewing area DL, the parallax W1 and the parallax W2 are almost equal when the user views the viewing area DL.

[0012] On the other hand, when the user views the visual recognition area DR, the visual recognition area DR is farther from the user in the horizontal direction than the visual recognition area DL. In this case, as shown in the enlarged view of the visual recognition area DR, the parallax W2 is larger than the parallax W1. Thus, the parallax W1 and the parallax W2 change as the visual recognition area of ​​the user changes. In the video processing device 20 of the present embodiment, by generating a parallax induction pattern corresponding to such a change in parallax, it is possible for the user to perceive, for example, a 3D object displayed at a position horizontally far from the user with a correct depth representation.

[0013] 2 is a diagram showing an example of the configuration of the video processing device 20. The video processing device 20 is, for example, a computer. The video processing device 20 includes, for example, a processor 201, a storage medium 202, a user interface 203, and a communication module 204. The processor 201, the storage medium 202, the user interface 203, and the communication module 204 are connected to each other via a bus 205.

[0014] The processor 201 includes any one of a central processing unit (CPU), a graphics processing unit (GPU), an application specific integrated circuit (ASIC), a microcomputer, a field programmable gate array (FPGA), and a digital signal processor (DSP), etc. The storage medium 202 may include an auxiliary storage device in addition to a main storage device such as a memory.

[0015] The primary storage device is a non-transient storage medium. The primary storage device is, for example, a non-volatile memory such as a hard disk drive (HDD) or a solid state drive (SSD) that can be written to and read from at any time, or a non-volatile memory such as a read only memory (ROM). These non-volatile memories may also be used in combination. The auxiliary storage device is a tangible storage medium. The auxiliary storage device is a combination of the aforementioned non-volatile memory and a volatile memory such as a random access memory (RAM). In the video processing device 20, only one processor 201 and one storage medium 202 may be provided, or multiple processors 201 and one storage medium 202 may be provided.

[0016] In the video processing device 20, the processor 201 performs processing by executing a program or the like stored in a storage medium 202. In addition, in the video processing device 20, the program executed by the processor 201 may be stored in a computer (server) connected via a network such as the Internet or a server in a cloud environment. In this case, the processor 201 downloads the program via the network.

[0017] In the user interface 203, various operations are input by the user of the video processing device 20, and information to be notified to the user is displayed, etc. The user interface 203 may be a display unit such as a display, or an input unit such as a touch panel or a keyboard. Note that a device connected to the video processing device 20 may be used as the input unit, or an input unit of another processing device capable of communicating via a network may be used.

[0018] Fig. 3 is a diagram showing an example of a functional configuration of the video processing device 20. As shown in Fig. 3, the video processing device 20 includes, for example, an image acquisition unit 31, an optimization processing unit 32, a pattern generation unit 33, an image generation unit 34, and a communication unit 35. The processes of the image acquisition unit 31, the optimization processing unit 32, the pattern generation unit 33, the image generation unit 34, and the communication unit 35 are realized by, for example, a processor 201 and a communication module 204.

[0019] The image acquisition unit 31 acquires a viewpoint image used in the video processing device 20. The image acquisition unit 31 is, for example, a camera. The optimization processing unit 32 executes a predetermined process based on the viewpoint image acquired by the image acquisition unit 31. The pattern generation unit 33 generates a parallax induction pattern based on the viewpoint image acquired by the image acquisition unit 31 and the processing result of the optimization processing unit 32. The parallax induction pattern realizes, for example, the parallax W1 and parallax W2 described in FIG. 1 by executing a predetermined process together with a predetermined reference image. The image generation unit 34 generates a stereo pair image based on the parallax induction pattern generated by the pattern generation unit 33 and the predetermined reference image. The communication unit 35 transmits the stereo pair image generated by the image generation unit 34 in a predetermined manner. For example, the communication unit 35 transmits the stereo pair image to an image output device connected to the video processing device 20, thereby causing the image output device to output the stereo pair image.

[0020] Next, a method for generating stereo pair images by the video processing device 20 will be described in detail. FIG. 4 is an explanatory diagram for explaining an example of a process for generating stereo pair images by the video processing device 20. As described above, the image acquisition unit 31 acquires a left eye viewpoint image PLP, an intermediate viewpoint image PCP, and a right eye viewpoint image PRP. The optimization processing unit 32 generates a parallax induction pattern ID based on these three images. In this embodiment, the intermediate viewpoint image PCP is used as a predetermined image (reference image) that serves as a reference. The optimization processing unit 32 converts each of the left eye viewpoint image PLP, the intermediate viewpoint image PCP, and the right eye viewpoint image PRP into frequency-phase components. The conversion into frequency-phase components is performed by a process similar to that described in Non-Patent Document 1. The phase components of frequency i and position j after the conversion of the intermediate viewpoint image PCP into frequency-phase components are denoted as X(i,j). The phase components of frequency i and position j after the conversion of the left eye viewpoint image PLP into frequency-phase components are denoted as L(i,j). The phase component of frequency i and position j after conversion into the frequency-phase components of the right-eye viewpoint image PRP is denoted as R(i,j).

[0021] The optimization processing unit 32 shifts the phase of the intermediate viewpoint image PCP by y degrees. The optimization processing unit 32 adds a phase shift amount y(i,j) to the phase component X(i,j) of the intermediate viewpoint image PCP to obtain the phase-shifted intermediate viewpoint image PCP. shift The optimization processing unit 32 generates the intermediate viewpoint image PCP and the phase-shifted intermediate viewpoint image PCP shift The estimated left-eye viewpoint image PLP is obtained by adding asm At this time, the optimization processing unit 32 generates the phase-shifted intermediate viewpoint image PCP shift The optimization processor 32 multiplies the estimated left-eye viewpoint image PLP by a weight A and adds the result to the intermediate viewpoint image PCP. The value of the weight A is set to, for example, a predetermined initial value in advance. asm Based on the estimated left eye view PLP asm The estimated phase shift amount zL(i,j) from the intermediate viewpoint image PCP is calculated.

[0022] Similarly, the optimization processing unit 32 obtains an intermediate viewpoint image PCP P shift The estimated right-eye viewpoint image PRP is calculated by subtracting asm At this time, the optimization processing unit 32 generates the phase-shifted intermediate viewpoint image PCP shift The optimization processor 32 multiplies the estimated right-eye viewpoint image PRP by a weight A and subtracts it from the intermediate viewpoint image PCP. asm Based on the estimated right eye view PRP asm The estimated phase shift amount zR(i,j) from the intermediate viewpoint image PCP is calculated.

[0023] The optimization processing unit 32 optimizes a set (A, y) of weight A and phase shift amount y under the condition of minimizing the error N expressed by equation (1). Since the estimated phase shift amounts zL(i, j) and zR(i, j) vary depending on both the weight A and the phase shift amount y, the set (A, y) is optimized by minimizing the error N. The set of weight A and phase shift amount y is determined, for example, by a full search. By this calculation to minimize the error N, the optimization processing unit 32 optimizes the optimal weight A. opt and the optimal phase shift amount y opt A set of opt ,y opt ) is calculated.

[0024]

number

[0025] Optimal weight A calculated by the optimization processing unit 32 opt and the optimal phase shift amount y optBased on this, the pattern generating unit 33 generates a parallax induction pattern ID. The generation of the parallax induction pattern ID is executed by a process similar to the method described in Non-Patent Document 1. The image generating unit 34 generates one of the stereo pair images at the left eye position PL by adding the parallax induction pattern ID generated by the pattern generating unit 33 and the intermediate viewpoint image PCP. The image generating unit 34 generates the other of the stereo pair images at the right eye position PR by subtracting the parallax induction pattern ID generated by the pattern generating unit from the intermediate viewpoint image PCP. In this way, the image generating unit 34 generates a stereo pair image.

[0026] The above-mentioned optimization process will be specifically described in the case where the level at the phase θ of the viewpoint position is expressed by the intensity of a sine wave. In this case, the intensity at the phase X of the intermediate position PC is expressed as sin(X). At this time, when the phase X is shifted by y degrees, the intensity after the phase shift is expressed as sin(X+y). The optimization processing unit 32 adds the intensity at the phase X, sin(X), and the intensity after the phase shift, sin(X+y), weighted by weight A, to obtain Asin(X+y), and sets the result as the estimated intensity at the phase L of the left eye position PL. The estimated intensity at this time is expressed as shown in Equation (2) using the phase X, the estimated phase shift amount zL, and the weight BL.

[0027]

number

[0028] Therefore, the estimated phase shift amount zL is expressed as in equation (3).

[0029]

number

[0030] Similarly, the optimization processor 32 subtracts Asin(X+y), which is the intensity sin(X+y) after the phase shift weighted by weight A, from the intensity sin(X) at phase X, and determines the result as the estimated intensity at phase R of the right eye position PR. The estimated intensity at this time is expressed as equation (4) using the phase X, the estimated phase shift amount zR, and the weight BR.

[0031]

number

[0032] Therefore, the estimated phase shift amount zR is expressed as in equation (5).

[0033]

number

[0034] Using these, the optimization processor 32 optimizes a set (A, y) of the weight A and the phase shift amount y under the condition that the error N expressed by the formula (1) is minimized.

[0035] Fig. 5 is a flowchart for explaining an example of processing executed by the image processing device 20 according to the embodiment. The processing in Fig. 5 is repeatedly executed at the timing when the image processing device 20 generates a stereo pair image. Therefore, the processing in Fig. 5 is an example of a flowchart for one processing of the image generation processing for generating a stereo pair image.

[0036] At the timing of generating a stereo pair image, the video processing device 20 acquires viewpoint images by the image acquisition unit 31 (S501). At this time, the image acquisition unit 31 acquires a left-eye viewpoint image PLP, an intermediate viewpoint image PCP, and a right-eye viewpoint image PRP, respectively. As described above, the video processing device 20 converts the acquired left-eye viewpoint image PLP, intermediate viewpoint image PCP, and right-eye viewpoint image PRP into frequency-phase components (S502). The video processing device 20 converts the phase-shifted intermediate viewpoint image PCP into frequency-phase components (S503). shiftAs described above, the image processing device 20 generates the intermediate viewpoint image PCP and the intermediate viewpoint image PCP obtained by shifting the phase of the intermediate viewpoint image PCP (S503). shift Based on and, the estimated left eye view image PLP asm and the estimated right-eye view image PRP asm The image processing device 20 generates an estimated left-eye viewpoint image PLP asm The estimated phase shift amount zL(i,j) is estimated based on the estimated right-eye viewpoint image PRP asm The image processing device 20 estimates an estimated phase shift amount zR(i,j) based on the above (S505). As described above, the image processing device 20 optimizes a set of the phase shift component y(i,j) and the weight A(i,j) under the condition that the error N is minimized (S506). As described above, the image processing device 20 optimizes the set of the optimal weight A(i,j). opt and the optimal phase shift amount y opt Based on the above, the image processing device 20 generates a parallax-guiding pattern ID (S507). As described above, the image processing device 20 generates a stereo pair image based on the intermediate viewpoint image PCP and the parallax-guiding pattern ID (S508). With the above, the image processing device 20 completes the process of generating a stereo pair image.

[0037] In this embodiment, the video processing device 20 includes an image acquisition unit 31, an optimization processing unit 32, a pattern generation unit 33, and an image generation unit 34. The image acquisition unit 31 acquires a left-eye viewpoint image PLP obtained by capturing the display region RS from a left eye position PL, a right-eye viewpoint image PRP obtained by capturing the display region RS from a right eye position PR, and an intermediate viewpoint image PCP obtained by capturing the display region RS from a position between the left eye position PL and the right eye position PR. The optimization processing unit 32 optimizes the phase shift amount y and weight A calculated based on the intermediate viewpoint image PCP based on the left-eye viewpoint image PLP and the right-eye viewpoint image PRP. The pattern generation unit 33 optimizes the optimized optimal phase shift amount y opt and the optimized weight A optBased on the above, a parallax induction pattern ID corresponding to the parallax between the left eye position PL and the right eye position PR is generated. The image generation unit 34 generates a stereo pair image based on the intermediate viewpoint image PCP and the parallax induction pattern ID. In this way, the video processing device 20 optimizes the phase shift amount and the weight A based on the left eye viewpoint image PLP and the right eye viewpoint image PRP, so that an appropriate parallax can be provided to the user even when the parallax is asymmetric. Therefore, the video processing device 20 can provide an appropriate depth representation to the user.

[0038] (Modification) FIG. 6 is an explanatory diagram for explaining an example of a method of dividing the display area RS when the video processing device 20 according to the modified example generates a stereo pair image. In this modified example, the real display area RS is divided into a predetermined number of areas based on the intermediate position PC, and the phase shift amount y and weight A are optimized for each divided area. That is, the phase shift amount y and weight A are the same in each divided area. The number of divisions of the real display area RS is not particularly limited. In the example of FIG. 6, the real display area RS is divided into three areas, a left area AL, a center area AC, and a right area AR. In this case, the video processing device 20 optimizes the phase shift amount y and weight A in the left area AL, optimizes the phase shift amount y and weight A in the center area AC, and optimizes the phase shift amount y and weight A in the right area AR.

[0039] 7 is an explanatory diagram illustrating an example of a process in which a video processing device 20 according to a modified example generates a stereo pair image. In this modified example, when the optimization processing unit 32 shifts the phase of the intermediate viewpoint image PCP by y degrees, a common phase shift amount y is applied to each of the divided regions into which the display region RS is divided. part The optimization processing unit 32 applies a phase shift amount y to the phase component X(i, j) of the intermediate viewpoint image PCP in each divided region. part By adding the phase-shifted intermediate viewpoint image (PCP shift ) part is generated for each divided region.

[0040] The optimization processing unit 32 calculates the intermediate viewpoint image PCP and the phase-shifted intermediate viewpoint image (PCP shift ) part The estimated left-eye viewpoint image (PLP) is obtained by adding the corresponding parts of the left-eye viewpoint image PLP to the corresponding parts of the left-eye viewpoint image PLP. asm ) part At this time, the optimization processing unit 32 generates a phase-shifted intermediate viewpoint image (PCP shift ) part Weight A part The optimization processor 32 multiplies the left-eye viewpoint image (PLP) by the intermediate viewpoint image PCP. asm ) part Based on this, the estimated phase shift amount zL from the intermediate viewpoint image PCP is part Estimate.

[0041] The optimization processor 32 optimizes the weight A for each divided region under the condition that the error N expressed by the formula (1) is minimized. part and the phase shift amount y part A set of part ,y part The optimization processing unit 32 optimizes the optimal weight (A part ) opt and the optimal phase shift amount (y part ) opt Based on this, the pattern generating unit 33 generates a parallax-inducing pattern ID for each divided region. part The image generating unit 34 generates the parallax-inducing pattern ID generated by the pattern generating unit 33. part The image generating unit 34 generates one of the stereo pair images at the left eye position PL corresponding to the divided region by adding the portion of the intermediate viewpoint image PCP corresponding to the divided region to the portion of the intermediate viewpoint image PCP corresponding to the divided region. part After the generation of the stereo pair images corresponding to the divided regions is completed for all divided regions, the image generation unit 34 generates the other stereo pair image corresponding to the display region RS by synthesizing the stereo pair images corresponding to the divided regions.

[0042] FIG. 8 is a flowchart for explaining an example of processing executed by the video processing device 20 of this modified example. At the timing for generating a stereo pair image, the video processing device 20 acquires viewpoint images by the image acquisition unit 31 (S801). At this time, the image acquisition unit 31 acquires a left-eye viewpoint image PLP, an intermediate viewpoint image PCP, and a right-eye viewpoint image PRP, respectively. As described above, the video processing device 20 converts the acquired left-eye viewpoint image PLP, intermediate viewpoint image PCP, and right-eye viewpoint image PRP into frequency-phase components (S802). The video processing device 20 converts the phase-shifted intermediate viewpoint image (PCP) into frequency-phase components (S803). shift ) part The image processing device 20 generates the intermediate viewpoint image PCP and the intermediate viewpoint image (PCP shift ) part Based on this, the estimated left eye view image (PLP asm ) part and the estimated right-eye view image (PRP asm ) part is generated for each divided region (S804).

[0043] The image processing device 20 receives the estimated left-eye viewpoint image (PLP asm ) part The estimated phase shift amount zL based on part is estimated for each divided region, and the estimated right-eye viewpoint image (PRP asm ) part The estimated phase shift amount zR based on part The image processing device 20 estimates the phase shift amount y part and weight A part The image processing device 20 optimizes the set of optimal weights (A part ) opt and the optimal phase shift amount (y part ) opt Based on the parallax induced pattern ID partThe image processing device 20 generates the intermediate viewpoint image PCP and the parallax induction pattern ID for each divided region (S807). part Based on this, a stereo pair image is generated for each divided region (S808). The video processing device 20 generates a stereo pair image corresponding to the display region RS by synthesizing the stereo pair images for each divided region (S809). With the above, the video processing device 20 completes the generation of the stereo pair image.

[0044] In this modification, as in the above embodiment, the image processing device 20 optimizes the phase shift amount y and the weight A based on the left eye viewpoint image PLP and the right eye viewpoint image PRP, so that the image processing device 20 can provide the user with an appropriate parallax even when the parallax is asymmetric. Therefore, the image processing device 20 can provide the user with an appropriate depth representation.

[0045] In another modified example, the image processing device 20 may acquire the left eye position PL, the right eye position PR, and the intermediate position PC in real time. The left eye position PL, the right eye position PR, and the intermediate position PC are acquired in real time, for example, by head tracking of the user. In this case, the image processing device 20 generates a stereo pair image in real time based on the left eye position PL, the right eye position PR, and the intermediate position PC acquired in real time. Therefore, the image processing device 20 executes the process shown in FIG. 5 or FIG. 8 every time the left eye position PL, the right eye position PR, and the intermediate position PC are updated. In this way, by acquiring the left eye position PL, the right eye position PR, and the intermediate position PC in real time, a stereo pair image reflecting the user's viewpoint in real time is generated. Therefore, the image processing device 20 can provide the user with a more appropriate depth expression.

[0046] The methods described in the above-mentioned embodiments and the like can be distributed as a program (software) that can be executed by a computer by storing it in a storage medium such as a magnetic disk, an optical disk, a semiconductor memory, etc. The storage medium is not limited to a storage medium for distribution, but includes a storage medium such as a magnetic disk, a semiconductor memory, etc. provided inside a computer or in a device connected via a network. The methods described in the above-mentioned embodiments can be distributed by transmitting it through a communication medium. The program stored on the medium side also includes a setting program that configures the software to be executed by a computer in the computer. The software includes not only an execution program but also a table and a data structure. The computer that realizes this system reads the program recorded in the storage medium and executes the above-mentioned processing by being controlled by the software. The software may be constructed by the computer using the setting program.

[0047] The present invention is not limited to the above-mentioned embodiment, and can be modified in various ways without departing from the gist of the present invention. The embodiments may be combined as appropriate, and in that case, the combined effect can be obtained. Furthermore, the above-mentioned embodiment includes various inventions, and various inventions can be extracted by combinations selected from the multiple components disclosed. For example, if the problem can be solved and the effect can be obtained even if some components are deleted from all the components shown in the embodiment, the configuration from which the components are deleted can be extracted as an invention. [Explanation of symbols]

[0048] 20...Image processing device 201…Processor 202…Storage medium 203...User Interface 204...Communication module 31…Image acquisition section 32...Optimization processing section 33...Pattern generation section 34...Image generation unit 35…Communications Department ID: Parallax induction pattern RS…display area PL…Left eye position PR…Right eye position PC…Intermediate position PLP…Left-eye image PRP…Right-eye image PCP: Mid-point image

Claims

1. an image acquisition unit that acquires a left-eye viewpoint image obtained by capturing a display area from a left eye position, a right-eye viewpoint image obtained by capturing the display area from a right eye position, and an intermediate viewpoint image obtained by capturing the display area from an intermediate position between the left eye position and the right eye position; an optimization processing unit that optimizes the phase shift amount and the weight calculated based on the intermediate viewpoint image based on the left-eye viewpoint image and the right-eye viewpoint image; a pattern generator that generates a disparity-inducing pattern corresponding to a disparity between the left eye position and the right eye position based on the optimized phase shift amount and the optimized weight; an image generating unit that generates a stereo pair image based on the intermediate viewpoint image and the parallax inducing pattern; A video processing device comprising:

2. The optimization processing unit: shifting a phase of the intermediate viewpoint image based on the phase shift amount; generating an estimated left-eye viewpoint image by estimating the left-eye viewpoint image and an estimated right-eye viewpoint image by estimating the right-eye viewpoint image based on the weights, the intermediate viewpoint image, and the phase-shifted intermediate viewpoint image; calculating an estimated phase shift amount in the estimated left-eye viewpoint image from the intermediate viewpoint image, and an estimated phase shift amount in the estimated right-eye viewpoint image from the intermediate viewpoint image; optimizing the weight and the phase shift amount based on an estimated phase shift amount of the estimated left-eye viewpoint image and an estimated phase shift amount of the right-eye viewpoint image; The video processing device according to claim 1 .

3. The image generating unit includes: generating an image corresponding to the left eye position among the stereo pair images by adding the intermediate viewpoint image and the parallax inducing pattern; generating an image corresponding to the right eye position from the stereo pair images by subtracting the parallax inducing pattern from the intermediate viewpoint image; The video processing device according to claim 1 .

4. The display area includes a plurality of the display areas, the optimization processing unit optimizes the phase shift amount and the weight in each of the plurality of display regions; The pattern generation unit generates the parallax inducing pattern in each of the plurality of display regions, the image generation unit generates the stereo pair images in each of the plurality of display areas, and generates the stereo pair image corresponding to the entirety of the display area from the stereo pair images corresponding to each of the plurality of display areas, The video processing device according to claim 1 .

5. The image acquisition unit acquires the left eye position, the right eye position, and the intermediate position in real time. The video processing device according to claim 1 .

6. Obtaining a left-eye viewpoint image obtained by photographing a display area from a left eye position, a right-eye viewpoint image obtained by photographing the display area from a right eye position, and an intermediate viewpoint image obtained by photographing the display area from an intermediate position between the left eye position and the right eye position; optimizing the phase shift amount and the weight calculated based on the intermediate viewpoint image based on the left eye viewpoint image and the right eye viewpoint image; generating a disparity-inducing pattern corresponding to a disparity between the left eye position and the right eye position based on the optimized phase shift amount and the optimized weights; generating a stereo pair image based on the intermediate viewpoint image and the parallax inducing pattern; A video processing method comprising:

7. On the computer, acquiring a left-eye viewpoint image obtained by photographing a display area from a left eye position, a right-eye viewpoint image obtained by photographing the display area from a right eye position, and an intermediate viewpoint image obtained by photographing the display area from an intermediate position between the left eye position and the right eye position; optimizing the phase shift amount and the weight calculated based on the intermediate viewpoint image based on the left eye viewpoint image and the right eye viewpoint image; generating a disparity-inducing pattern corresponding to a disparity between the left eye position and the right eye position based on the optimized phase shift amount and the optimized weights; generating a stereo pair image based on the intermediate viewpoint image and the parallax inducing pattern; Image processing program.

Images