Video presentation device, video presentation method, and program

Abstract

It is desired to present a stereoscopic video which spans a plurality of displays and appears to pop out in front of the displays by allowing a non-alignment arrangement in which displays of different types and sizes are mixed. For this purpose, the video presentation device according to the technology disclosed herein comprises a gap luminance measurement unit and a luminance adjustment unit. The gap luminance measurement unit measures the luminance (L gap) between the screens of the displays. The luminance adjustment unit adjusts the luminance p of pixels constituting a stereoscopic target to p'. On such occasion, p' satisfies Lgap ≤ p' < Lbg, or Lgap ≥ p' > Lbg, where Lbg is the luminance of pixels constituting the background of the stereoscopic target.

Description

Image presentation device, image presentation method, and program

[0001] The disclosed technology relates to a technique for presenting large 3D images across multiple displays, allowing for misaligned arrangements that mix displays of different types and sizes.

[0002] Displaying large 3D images (images that create a stereoscopic perception) can create impressive effects, but large displays are expensive and vulnerable to damage and disasters. One alternative is to display a single large image by arranging multiple general-purpose displays (Figure 1(1), Non-Patent Document 1). 101 represents the target object for which stereoscopic perception is desired, and 102 represents a group of aligned displays. This method is limited to 2D displays or, even 3D displays, to display images behind the displays. Furthermore, conventional technology (Non-Patent Document 1) assumes that identical displays are neatly aligned. This requires a lot of effort. To meet a wider range of needs, a misaligned arrangement (103 in Figure 1(2)) is also permitted, mixing displays of different types and sizes, and even allowing images to appear to pop out in front of the displays. This arrangement results in misalignment or gaps between the displays, which can make the 3D image appear to be further behind the displays.

[0003] Daniel J. Sandin et al., "The VarrierTM Autostereoscopic Virtual Reality Display", ACM Transactions on Graphics, Vol. 24, Issue 3, pp.894-903, 2005.

[0004] The part of a display that displays the image is called the screen, and the frame that surrounds the screen is called the bezel. It is generally difficult to present 3D images that span multiple displays and appear to stand out in front. This is because the image cannot be presented in the gaps between displays or the bezels of the displays themselves, resulting in missing parts of the image, and no parallax can be applied to the edges of these missing parts. In such a situation, even if parallax is applied to the 3D image so that it appears to stand out, inconsistencies in parallax information will arise with the edges of the missing parts, making it difficult to achieve stereoscopic perception, and the images will not appear connected, resulting in a lack of impact.

[0005] In order to solve the above problem, the image display device according to the disclosed technology is a device that displays parallax images for stereoscopic viewing using a plurality of displays, and includes a gap brightness measurement unit and a brightness adjustment unit. The gap brightness measurement unit measures the brightness (L gap The brightness adjustment unit adjusts the brightness p of the pixels that make up the stereoscopic target to p'. At this time, p' is calculated by adjusting the brightness of the pixels that make up the background of the stereoscopic target to L. bg As, L gap ≦p'<L bg , or L gap ≧p'>L bg Make sure to satisfy the following.

[0006] The disclosed technology can present powerful 3D images by arranging multiple small displays of different types and sizes, and can also realize a large 3D display that is cost-effective and robust against damage (only damaged displays can be replaced).

[0007] 1 is a diagram illustrating a method (conventional / disclosed technology) for presenting a single large image by arranging multiple general-purpose displays.

[0023] FIG. 1 is a diagram illustrating the Transmittance Anchoring Principle (TAP).

[0024] FIG. 1 is a diagram illustrating modal completion occurring when parallax is applied to an image that satisfies the TAP.

[0025] FIG. 2 is a diagram illustrating conditions for transparency and modal completion occurring in an image spanning multiple displays.

[0026] FIG. 2 is a diagram illustrating a specific example of brightness adjustment for a target.

[0027] FIG. 3 is a functional block diagram of a 3D image presentation system according to a first embodiment.

[0028] FIG. 4 is a flowchart illustrating the operation of a display control device according to the first embodiment.

[0029] FIG. 5 is a diagram illustrating an example of display arrangement and ChArUco presented on the displays.

[0029] FIG. 6 is a diagram illustrating examples of checkerboard pattern detection results and pose estimation results.

[0030] FIG. 7 is a diagram illustrating a method for generating parallax images.

[0031] FIG. 8 is a flowchart illustrating the operation of a display control device according to a second embodiment.

[0032] FIG. 9 is a diagram illustrating a method for synchronizing images between displays.

[0033] FIG. 10 is a functional block diagram of a 3D image presentation system according to a third embodiment.

[0034] FIG. 11 is a flowchart illustrating the operation of a display control device according to the third embodiment.

[0035] FIG. 12 is a diagram illustrating various tracking techniques.

[0036] FIG. 13 is a diagram illustrating a case where gradation is required on both the u-axis and the v-axis. 10 is a diagram illustrating application areas of u-direction gradation and v-direction gradation. 11 is a diagram illustrating an example of the functional configuration of a computer.

[0008] Hereinafter, embodiments of the disclosed technology will be described in detail. Note that components having the same functions are assigned the same numbers, and duplicated descriptions will be omitted.

[0009] [Background of the Perception of Transparency: Transmittance Anchoring Principle (Reference 1)] A boundary of brightness is called an "edge." In Figure 2, the boundaries between areas A and B, and between areas P and Q are edges. The sign of the luminance gradient representing the change from the dark side to the light side across an edge is called "contrast polarity." The arrows in Figure 2 indicate positive contrast polarity. When two adjacent edges with different contrast (the edge between A and B and the edge between P and Q) have the same contrast polarity, transparency occurs, and the surface containing the low-contrast edge is perceived as having a semitransparent filter (Transmittance Anchoring Principle, hereafter referred to as "TAP").

[0010] Let the luminance of areas A, B, P, and Q be a, b, p, and q. In Figure 2(1), the contrast |pq| (the absolute value of the difference between p and q) of the PQ edge is smaller than the contrast |ab| of the A-B edge, and a semitransparent filter is perceived to be in the foreground in the area including the PQ edge (the central circle). In Figure 2(2), the contrast |ab| of the A-B edge is smaller than the contrast |pq| of the PQ edge, and a semitransparent filter is perceived to be in the foreground in the area including the A-B edge (the area surrounding the central circle). On the other hand, in Figure 2(3), the contrast polarity of the A-B edge and the contrast polarity of the P-Q edge are opposite, and no semitransparent filter is perceived.

[0011] Reference 1: Barton L. Anderson, "The Role of Occlusion in the Perception of Depth, Lightness, and Opacity", Psychological Review, Vol 110, No. 4, pp.785-801, 2003.

[0012] When parallax is applied to an image with an edge junction (T-junction, 301 in the lower right of Figure 3) that satisfies the TAP condition, transparency results in the completion of the missing edge (modal completion). For example, in Figure 3, where parallax is applied so that the circle appears to be in front of the triangle (the two left images are perceived as cross fusion, and the two right images are perceived as uncross fusion), the junction of the triangle and the circle forms a T-junction. In this case, the circle is perceived as semitransparent and in front only when the contrast polarity of the head of the T-junction is consistent on both sides of the stem of the T-junction, as in (1) and (2). In (3), where the contrast polarity is reversed, transparency does not occur, and the consistent perception of the circle being in front is not achieved. Furthermore, when transparency is perceived in (1) and (2), the presence of a semitransparent edge is perceived in the missing area of ​​the circle, resulting in the illusion of a continuous image (modal completion).

[0013] [Key points of the disclosed technology] Based on the above findings, the disclosed technology adjusts the brightness of the presented image so that transparency and modal image complementation occur, while taking into account the brightness of the gaps between the screens, when presenting an image across multiple displays.

[0014] This will be explained using Figure 4. 401 and 402 are two screens separated by a gap G (403). 404 is an image (target) presented across the screens 401 and 402. The luminance of the target area P on the screen 401 is p, the luminance of the non-target area A is a, and the luminance of the gap G is b (= q). The same applies to (2) to (6) in Figure 4. In Figures 4(1) and (2), the contrast polarity of the edge between A and G matches the contrast polarity of the edge between P and G. Since |pq| > |ab|, the target 404 tends to appear closer. In Figures 4(3) and (4), the contrast polarity of the edge between A and G matches the contrast polarity of the edge between P and G. Since |pq| < |ab|, the non-target area A tends to appear closer. In Figures 4(5) and (6), the contrast polarity of the edge between A and G and the contrast polarity of the edge between P and G are reversed, and transparency is not achieved.

[0015] Therefore, the brightness of the area that you want to appear to stand out (target area) is adjusted so that when the brightness of the gap is dark, it becomes the state shown in Figure 4(1), and when it is bright, it becomes the state shown in Figure 4(2).The brightness of the background is also adjusted as necessary.

[0016] <Brightness range limiting process 1> Let p be the brightness of the pixels that make up the target area, a be the brightness of the pixels that make up the background of the target (outside the target area), and b be the brightness of the gap between the screens. When the target area is a texture that contains various brightnesses, one possible method of brightness adjustment is to remap the pixel values ​​of the target area to the range from a to b. In this case, let p be the minimum brightness of the target area. min , maximum brightness is p max , the minimum luminance of the target area after remapping is p' min , the maximum brightness is p' max Then, the larger of a and b is p' max , the smaller one is p' min The luminance value p'(u,v) obtained by remapping the luminance value p(u,v) of the target area is as follows:

[0017] This allows the luminance in the target area to be p' = L obj is always L gap ≦L obj <L bg , or L gap ≧L obj >L bg In this case, if the brightness b of the gap is lower than a predetermined value, the background a can be set to the maximum brightness (white), and conversely, if the brightness b of the gap is equal to or greater than a predetermined value, the background a can be set to the minimum brightness (black), so that the contrast in the target image can be kept as high as possible. An example of such adjustment is shown in Figure 5. Figure 5 (1) shows the result before adjustment, in which transparency is not achieved and the target cat image is difficult to see in front. Figure 5 (2) shows an example of the result after adjustment, in which the condition for transparency to be achieved (L gap ≦L obj <L bg ) is satisfied, and the cat image appears to jump out easily. Figure 5 (3) shows an example where the background is set to black and the brightness of the target area is adjusted when the gap is bright. gap ≧L obj >L bg This satisfies the above criteria, making the cat image appear to jump out into the foreground.

[0018] <Brightness range limiting process 2> In practice, the maximum and minimum brightness of the target after remapping (p' max , p' min ) does not need to match the brightness of the background and gap, as long as they are both between the background brightness (a) and the brightness of the gap (b). Also, since the background brightness (a) is also a controllable variable, if the background brightness after remapping is set to a', then the constraint (a'< p' min ,p' max ≦b or a'>p' min ,p' max ≧b) min , p' max ) that makes the target area appear closer to the front. min , p' maxA perceptual model is established in advance through psychophysical experiments, using the luminance of the target area as input to predict the "probability that the target area appears to be closer (the degree to which it is perceived as being closer)." Based on this model, the optimal (a', p' min , p' max However, in this case, the most trivial solution is p' min =p' max = b (i.e., the target area is filled with the same brightness as the gap). Therefore, the visibility of the image information in the target area is evaluated by, for example, the Michelson contrast C M =(p' max -p' min ) / (p' max +p' min ), and the probability that the target area appears in front is equal to or greater than a predetermined value (a', p' min , p' max ) among the sets of C M may be determined as the optimum solution such that .times. ...

[0019] <Gradient processing> Furthermore, since human vision uses local information, processing only the vicinity of the gap can make the target appear to cross the gap. By utilizing this, it is possible to achieve transparency without causing a significant change in the appearance of the entire target. Specifically, a gradation is applied to the target area so that the closer it gets to the gap, the closer it becomes to the luminance p'(u,v) calculated by the above method. An example of this type of image manipulation is shown in Figure 5 (4). Compared to the case of Figure 5 (2) without gradation processing, the contrast of the image in the target area as a whole is kept high, while the condition (L gap ≦L obj <L bg ) and the cat image appears to jump out at the front.

[0020] The above is a description of the gist of the disclosed technology. Below, embodiments of the disclosed technology will be described.

[0021] [First Embodiment] In the first embodiment, a group of stationary displays is used to present still images that create a stereoscopic perception. Furthermore, the brightness adjustment described below uses the above-described <Brightness Range Limitation Process 1> or <Brightness Range Limitation Process 2>. Figure 6 is a functional block diagram showing an example configuration of a 3D image presentation system according to the first embodiment. The 3D image presentation system 6 includes multiple displays 602, a display control device 601, a world camera 608, and a brightness measurement camera 609. The display control device 601 includes a geometric calibration unit 603, an optical calibration unit 604, a gap brightness measurement unit 605, a parallax image generation unit 606, and a brightness adjustment unit 607.

[0022] 7 is a flowchart illustrating an example of the operation of the display control device 601. The following description will be given with reference to FIGS.

[0023] [Geometric Calibration of Displays] The disclosed technology allows for misaligned arrangements of displays of different types and sizes. Therefore, the display control device 601 first performs geometric calibration of each display (step S701). Specifically, as shown in FIG. 8, the displays are appropriately aligned within the area where the target (the image desired for stereoscopic perception) is to be presented, and a ChArUco checkered pattern is presented on each display. ChArUco places unique ArUco markers in the white areas of a typical black-and-white checkered pattern, making it easy to associate each checkered pattern with the display. For details, see Reference 2.

[0024] Reference 2: S. Garrido-Jurado et al. "Automatic generation and detection of highly reliable fiducial markers under occlusion", Pattern Recognition Vol.47, Issue 6, pp.2280-2292, 2014.

[0025] The actual size of the checkered pattern can be calculated by multiplying the nominal pixel size of each display by the number of pixels in both the vertical and horizontal directions of the checkered pattern. The number of checkered patterns should be as large as possible within a range that can be recognized by the camera. This is because the finer the grid, the more feature points there are, allowing for more accurate estimation.

[0026] Next, each checkered pattern is photographed using a high-resolution stereo camera (world camera 608) capable of capturing the entire display group. The orientation (rotation matrix R and translation vector t) of each display is estimated based on the stereo camera's camera calibration values ​​and the actual dimensions of the checkered pattern. While this can be achieved with a single camera, using two cameras (stereo) improves depth estimation accuracy and enables more accurate position estimation. Figure 9 shows an example of the checkered pattern detection and orientation estimation results. Each detected ArUco marker is displayed with markers (902) at the four corners. The orientation estimation results can be displayed on the screen with the horizontal axis (x-axis), vertical axis (y-axis), and depth axis (z-axis) clearly distinguishable (e.g., as red, blue, and green axes, respectively). For ease of explanation, 901 in Figure 9 is displayed with x, y, and z added to the white lines.

[0027] [Optical Calibration of Displays] The display control device 601 then performs optical calibration of each display (step S702). <Color Space Calibration> When using multiple different displays, there is a problem that the color tones vary depending on the product. For example, in Figure 8, the second display from the right in the top row clearly has different colors. To address this issue, a commercially available device (such as the Spyder X2 Ultra) that performs color calibration to match the color spaces of multiple displays is used to match the color spaces of each display.

[0028] <Brightness calibration> There is a problem that many display brightness values ​​take different values ​​from the command value. x , the luminance value actually presented by the display is L y , a is the power parameter, and Lx and L y The relationship follows an exponential function: Therefore, the luminance of the display is measured externally, and the linearity of the display is calibrated to command the display luminance appropriately. Specifically, a function fitting of the luminance value is performed using an industrial camera (luminance measurement camera 609) with guaranteed linearity. The luminance value is increased from 0 in a range of 0 to 255 at regular intervals and presented on the entire display. The luminance value is measured by the luminance measurement camera 609, and multiple command values ​​L are set. x and the measured value L y The exponent a is calculated by performing exponential fitting using the pair of L x Instead of L' x =L x 1 / a By specifying the above, linear brightness values ​​can be presented.

[0029] [Measurement of the brightness of the gap between the screens] The display controller 601 then measures the brightness of the gap between the screens (L gap ) is measured (step S703). gap This is used in the "Brightness Adjustment" section described later. gap To measure the brightness, a luminance measurement camera 609 gap Just take a picture and use it. gap Image smoothing processing > L gap The brightness of the image (including the display bezel and the gap between the displays) is often not perfectly uniform, so gap uses the luminance of the display edge (usually the bezel) closest to each screen point, except for L gap L referenced at a point equidistant from the two edges (a point on a straight line at a 45° angle from a corner on the screen) gap It is not desirable for the target brightness to change suddenly due to the switching of gap Smoothing the spatial variation of L gap The image is blurred by convolving it with a Gaussian kernel, etc., and L' gapGenerate and use images.

[0030] [Generation of Parallax Images] The display control device 601 then generates parallax images for each of the left and right eyes (step S704). Parallax images are generated for each display. This will be explained using FIG. 10. First, as shown on the left side of FIG. 10, the positions and rotational orientations of each display and the target 3D object are grasped in the world camera coordinate system (world coordinate system). Note that the rotational and translational orientations of each display in the world coordinate system are known through the above-mentioned [Geometric Calibration of Displays]. Then, the representative observation positions assumed in the world coordinate system are determined by defining the positions of the left eye and right eye as E L , E R It should be noted that 3D viewing is possible even if the image is viewed from a different position from the above observation position.

[0031] In the 3D image presentation system according to the disclosed technology, it is assumed that each display is not necessarily in front of the viewpoint, and therefore a frustum consisting of each left and right viewpoint position and each screen surface is assumed, as shown on the right side of Figure 10, and the perspective projection matrix P of the view frustum is calculated based on this dimension. In addition, a model matrix M that determines the rotational and translational orientation of the 3D object and a view matrix V that determines the orientation of the viewpoint position (left and right viewpoint positions E L ,E R For each vertex v (3D vector) of the 3D object, the vertex position w (3D vector) in the graphics space is calculated using the following: w = P V M v Hereinafter, the matrices M, V, and P will be collectively referred to as the MVP matrix. By calculating w for all target vertices v, preparations for stereoscopic presentation are complete. This is performed for each display and for each left and right viewpoint position.

[0032] [Brightness Adjustment] The display control device 601 performs brightness adjustment processing on the target immediately before the execution of stereoscopic presentation (step S705). The brightness adjustment unit 607 receives the position and color information of each vertex of the target and the MVP matrix, and performs the same processing on all vertices. Each vertex v is transformed by the MVP matrix to obtain w=(w x,w y ,w z ) is generally expressed as a parameter w=k×(u,v,1), i.e., w with scaling variable k. x ,w y By dividing the above equation, the coordinates of each point w of the object on the projection surface (=screen surface) can be converted into coordinates (u, v) (u, vε[−1, 1]) as shown in FIG.

[0033] The minimum target luminance p' that can induce transparency by the method described in <Luminance Range Limitation Process 1> or <Luminance Range Limitation Process 2> is set. min and the maximum value p' max In this case, to find the minimum and maximum brightness of the target area for each screen coordinate (u, v), we calculate L' gap Image (above <L gap The brightness adjustment unit 607 uses the brightness (L') of the edge of the display closest to each point on the screen. gap A spatial map b(u,v) is created for each display, referencing the brightness of the image (the brightness of the image) (step S711). Note that (u,v) are screen coordinates. Next, the brightness adjustment unit 607 blurs b(u,v) by convolving it with a Gaussian kernel or the like to generate b'(u,v) (step S712). Next, the brightness adjustment unit 607 calculates p' using b'(u,v) and the brightness a of the out-of-target area. max and p' min When the <brightness range limiting process 2> is used, the determined p' is max and p' min Next, the brightness adjustment unit 607 calculates p'(u, v) using equation (1) (step S714).

[0034] [Stereoscopic Presentation] Finally, the display control device 601 outputs the brightness-adjusted left and right parallax images to the display (step S706). The method of presenting binocular parallax 3D images varies depending on the 3D method used. For example, in the anaglyph method (a method of 3D observation using 3D glasses with red, cyan, or other color filters), the 3D images from each viewpoint are presented in red and cyan, overlapping each other. In the shutter method (a method in which the left and right shutters of the glasses close alternately at a high frame rate), the image is presented in synchronization with the shutter of the open eye.

[0035] The above is the description of the first embodiment.

[0036] Second Embodiment In the second embodiment, a group of stationary displays is used to present a "moving image" that creates a stereoscopic perception. The configuration example of the 3D image presentation system according to the second embodiment is the same as that of the first embodiment ( FIG. 6 ). FIG. 12 is a flowchart illustrating an example of the operation of the display control device 601 according to the second embodiment. Steps S701 to S706 are the same as those of the first embodiment, except that after a frame constituting the moving image is output (step S706), it is determined whether the output frame is the last frame. If it is not the last frame (No in step S1201), the generation of the next parallax image (step S704) and the brightness adjustment process (step S705) are repeatedly executed.

[0037] [Video Synchronization Between Displays] Video synchronization between all displays can be easily achieved if all displays are connected to a single computer (such as a desktop PC). As shown in Figure 13, a graphics window spanning all displays is displayed, the content of the video to be displayed is set for each area of ​​the window occupied by each display, and finally, a command to update the entire screen is issued all at once, making it easy to present synchronized video on the entire screen. If the displays are connected to different computers or if a tablet or smartphone is used as a display, synchronization processing using a wired connection or a timestamp, etc., is required.

[0038] The above is the description of the second embodiment.

[0039] [Third Embodiment] In the third embodiment, a case will be described in which the display position is not stationary but moves autonomously or is freely moved by the viewer. This embodiment applies when it is desired to enlarge or reduce the range of video presentation or to change the front-to-back relationship of the display. The video to be presented is a moving image.

[0040] Fig. 14 is a functional block diagram showing an example of the configuration of a 3D image presentation system according to the third embodiment. The 3D image presentation system 14 differs from the 3D image presentation system 6 in that the display control device 1401 includes a display tracking unit 1402 instead of the geometric calibration unit 603. Fig. 15 is a flowchart explaining an example of the operation of the display control device 1401. The following description will be made with reference to Figs. 14 and 15.

[0041] [Optical Calibration of Displays] The display control device 1401 performs optical calibration of each display (step S702). Details of the process in step S702 are the same as in the first embodiment.

[0042] [Display Tracking] The display control device 1401 tracks the orientation of each display using the world camera 608 and the display tracking unit 1402, and estimates the rotation matrix R and translation vector t (step S1501). Because the displays always display stereoscopic images, it is not possible to present a checkered pattern as shown in Figure 8 for display tracking. However, there are many tracking methods that can be used instead of presenting a checkered pattern (described below).

[0043] Next, the display control device 1401 measures the brightness of the gap between the screens (step S703), generates a parallax image (step S704), adjusts the brightness of the target (step S705), and outputs the image to the display (step S706). The details of each step are the same as those in the first embodiment. Next, the display control device 1401 determines whether the output frame is the last frame, and if it is not the last frame (No in step S1201), the process returns to step S1501, where the process starts with display tracking and processes the next frame.

[0044] [Tracking methods] Tracking methods are divided into two categories: methods that place markers on the corners or edges of the display (Figs. 16(a), (b), and 17(c)), and markerless methods that use the display bezel (Fig. 17(d)). Tracking using markers has the advantage of high detection accuracy and speed, and can accurately and quickly visualize the target in response to changes in display position. Disadvantages include the time and effort required to attach markers to each display, the degradation of appearance, and the need for special hardware.

[0045] <(a) AR Marker> AR markers are easy to use, as they can use a general RGB camera as a detection device, but they are the most unsightly. Another disadvantage is that the marker needs to be fairly large to achieve high detection performance. Tracking is performed by corner detection, but this involves heavy image processing, making it slower than (b) and (c).

[0046] <(b) Retroreflective Marker> Retroreflective markers allow infrared light to be coaxially coupled with an infrared camera, enabling images with little noise, with only the marker appearing bright, and therefore offering high detection power. However, this method has the drawback of requiring a special camera optical system. Since infrared light is emitted from near the camera, reflected by the display, and then attenuated before entering the camera, it is not suitable for cases where the camera is far from the display.

[0047] <(c) Infrared LED marker> Infrared LED markers are self-luminous, allowing a relatively high amount of light to reach the infrared camera, and like (b), they offer excellent detection accuracy and speed. Infrared light is invisible, so it does not significantly impair the appearance. Self-luminous markers such as LEDs are suitable for long-distance tracking, and are often used in practice. On the other hand, they have the disadvantage of requiring a power supply.

[0048] <(d) No marker> In contrast, tracking without markers uses the black bezel of the display and the bright screen surface as clues, but there is a lot of noise at the boundary with the outside of the display, and the signal-to-noise ratio (S / N) is low, resulting in lower detection accuracy compared to (a), (b), and (c). Low detection accuracy causes the display position, which is updated with each frame, to become unstable, resulting in the object position fluctuating from frame to frame, which reduces viewing quality. In addition, corner detection, like in (a), does not increase speed, and the larger image processing area required compared to marker tracking also increases the processing load.

[0049] The above is an overview of the tracking method, but it is necessary to select an appropriate method taking into consideration the equipment available and the quality desired for viewing. This concludes the description of the third embodiment.

[0050] [Fourth Embodiment] A case will be described in which <gradation processing> is further applied to p'(u, v) obtained by the <brightness adjustment> of the first, second, and third embodiments. As described in the explanation of <gradation processing>, a gradation is applied to the target brightness so that it approaches p'(u, v) the closer it is to the edge of the screen, and the original target brightness is maintained in areas far from the edge of the screen.

[0051] The process by which the brightness adjustment unit 607 obtains p'(u, v) is as described in the first embodiment. The brightness adjustment unit 607 obtains the brightness L' of the target, which has been brightness-adjusted with gradation, using the following formula: obj is calculated (step S1805). However, L objis the original luminance of the target, α is the gradient rate between the edge and the center of the display, and clamp(p, p min , p max ) is the upper bound for the value of p max and the lower bound p min It is a function that limits the range between . When applying a gradation along the u axis, X = u, and when applying a gradation along the v axis, X = v. The bottom right of Figure 11 shows an example where u is substituted for X and α = 0.7, giving a gradation over 70% of the distance from the edge of the display to the center.

[0052] When a target is presented in area 1801 in Fig. 18, gradation is performed on three sides of display 1802 and on two sides of displays 1803 to 1806, and there are areas where the gradation in the u-axis direction and the gradation in the v-axis direction overlap on each display. In this case, for example, as shown in Fig. 19, gradation is performed in areas R1 and R3 using the following formula: In areas R2 and R4, gradation can be performed using the following formula.

[0053] The above is the description of the fourth embodiment.

[0054] [Program, Recording Medium] The functions realized by the components described in this specification may be implemented in circuitry or processing circuitry, including general-purpose processors, application-specific processors, integrated circuits, ASICs (Application Specific Integrated Circuits), CPUs (Central Processing Units), conventional circuits, and / or combinations thereof, programmed to realize the described functions. A processor includes transistors and other circuits and is considered to be circuitry or processing circuitry. A processor may be a programmed processor that executes a program stored in a memory.

[0055] In this specification, a circuitry, unit, or means is hardware that is programmed to realize or performs the described functions, which may be any hardware disclosed herein or any hardware known to be programmed to realize or perform the described functions.

[0056] If the hardware is a processor considered to be a type of circuitry, the circuitry, means, or unit is a combination of the hardware and software used to configure the hardware and / or processor.

[0057] The various processes described above can be implemented by loading a program that executes each step of the above method into the recording unit 2020 of the computer 2000 shown in Figure 20, and operating the control unit 2010, input unit 2030, output unit 2040, display unit 2050, etc.

[0058] The program describing this content can be recorded on a computer-readable recording medium, which may be, for example, a magnetic recording device, an optical disk, a magneto-optical recording medium, a semiconductor memory, or any other suitable recording medium.

[0059] The program may be distributed by, for example, selling, transferring, lending, etc. portable recording media such as DVDs and CD-ROMs on which the program is recorded. Furthermore, the program may be stored in a storage device of a server computer, and then transferred from the server computer to other computers via a network, thereby distributing the program.

[0060] A computer that executes such a program may first temporarily store the program recorded on a portable recording medium or transferred from a server computer in its own storage device. Then, when executing a process, the computer reads the program stored on its own recording medium and executes the process in accordance with the read program. Alternatively, the computer may read the program directly from a portable recording medium and execute the process in accordance with the program. Furthermore, the computer may execute the process in accordance with the program each time a program is transferred from a server computer to the computer. Alternatively, the server computer may not transfer the program to the computer, but may instead execute the process through a so-called ASP (Application Service Provider) service, which realizes the processing function by issuing an execution instruction and obtaining the results. Furthermore, the server computer may execute the process at the terminal using a so-called SaaS (Software as a Service) service, which allows users to use part of a server computer along with the program. In this embodiment, the program includes information used for processing by an electronic computer that is equivalent to a program (such as data that is not a direct instruction to a computer but has properties that dictate computer processing).

[0061] Furthermore, in this embodiment, the device is configured by executing a predetermined program on a computer, but at least a part of the processing contents may be realized by hardware.

[0062] 6 3D image presentation system 601 Display control device 602 Display 603 Geometric calibration unit 604 Optical calibration unit 605 Gap luminance measurement unit 606 Parallax image generation unit 607 Luminance adjustment unit 608 World camera 609 Luminance measurement camera 14 3D image presentation system 1401 Display control device 1402 Display tracking unit 2000 Computer 2010 Control unit 2020 Recording unit 2030 Input unit 2040 Output unit 2050 Display unit

Claims

1. A device that presents parallax images for stereoscopic vision using multiple displays, and the brightness between the display screens (L gap The brightness adjustment unit adjusts the brightness of the pixels constituting the background of the stereoscopic target from L to L'. bg As such, p' is L gap ≦p'<L bg , or L gap ≧p'>L bg An image presentation device characterized by satisfying the above.

2. A device that presents parallax images for stereoscopic vision using multiple displays, and the brightness between the display screens (L gap The brightness of the pixels that make up the background of the stereoscopic target is measured by the gap brightness measurement unit. bg Let p be the brightness of the pixels that make up the stereoscopic target, L gap ≦p'<L bg , or L gap ≧p'>L bg and a brightness adjustment unit for adjusting the brightness p' to satisfy the following: max , the smallest one is p' min , and L gap and L bg and p' max and p' min Using L gap Correction value L' gap and L bg Correction value L' bg and calculate the luminance p' by L' gap ≦p"<L' bg , or L' gap ≧p">L' bg The image display device adjusts p" to satisfy the above.

3. The image display device according to claim 1, wherein the brightness adjustment unit further adjusts brightness p from brightness p'. grad Generate p grad The brightness of a display device is p in areas far from the edge of the screen, and approaches p' as it approaches the edge of the screen.

4. The image display device according to claim 1, wherein the gap luminance measurement unit measures the image between the screens (L gap image) and L gap Smoothing the image to L' gap Generate an image and for each coordinate (u,v) on the screen, find the L' nearest to (u,v). gap b(u,v) is determined using the brightness of the image, and b'(u,v) is smoothed to obtain p', and b'(u,v)≦p'<L bg , or b'(u,v)≧p'>L bg The image display device is characterized by adjusting the image to satisfy the following.

5. An image presentation device according to claim 1, comprising: a first calibration unit that measures the orientation of each display; a second calibration unit that adjusts the color space and brightness of each display; and a parallax image generation unit.

6. An image presentation device according to claim 5, wherein the parallax image generation unit determines a viewing frustum for each of the displays and determines a perspective projection matrix.

7. A method for presenting parallax images for stereoscopic viewing using multiple displays, in which the gap luminance measurement unit measures the luminance between the screens of the displays (L gap ) and the brightness adjustment unit adjusts the brightness p of the pixels that make up the stereoscopic target to p', and adjusts the brightness of the pixels that make up the background of the stereoscopic target to L bg As such, p' is L gap ≦p'<L bg , or L gap ≧p'>L bg An image presentation method characterized by satisfying the following.

8. A program for causing a computer to function as the image presentation device according to any one of claims 1 to 6.

Images