Display control device and calibration method

Abstract

This display control device includes a display control unit, a reception unit, and a correction unit. The display control unit controls a stereoscopic display so as to display a guide object representing a parallax corresponding to a physical display surface of the stereoscopic display on the basis of a first display parameter related to stereoscopic vision. The reception unit receives confirmation information indicating a confirmation result of a user after the guide object is displayed. The correction unit corrects the first display parameter on the basis of a second display parameter related to stereoscopic vision set in the stereoscopic display after the confirmation information is received.

Description

Display control device and calibration method

[0001] The present disclosure relates to a display control device and a calibration method.

[0002] Conventionally, stereoscopic displays that display virtual objects as if they existed in real space have been known. In stereoscopic displays, the positions of the user's left and right eyes are estimated by tracking using a camera, and different images (hereinafter referred to as "parallax images") directed toward each estimated position are generated and displayed in real time, thereby realizing the stereoscopic display.

[0003] The estimation of the positions of the left and right eyes and the generation and display of a group of parallax images in real time are performed using display parameters related to stereoscopic vision, such as the angle and position of the user's face relative to the display surface and the inter-pupilary distance (IPD).

[0004] In a stereoscopic display, if the display parameters are not set correctly, a discrepancy will occur between the virtual space in which the virtual objects are displayed and the real space perceived by the user. Specifically, this can result in, for example, distortion of the display in the virtual space, or a display discrepancy between real and virtual objects that are intended to be displayed simultaneously.

[0005] In other words, the stereoscopic display needs to have its display parameters properly set by calibration. Patent Document 1 discloses a technique for calibrating, based on a user's input, the parallax used when a virtual object is overlaid on a real object in a see-through HMD (Head Mounted Display).

[0006] Specifically, this technology places a real object for calibration in front of a user wearing an HMD. Then, a distance measurement system measures the distance between the real object and the user's eyes, and the distance measurement result is used to overlay a virtual object on the real object. The user can input to move the virtual object, and the system calibrates the parallax based on the input result.

[0007] JP 2015-142383 A

[0008] However, the above-mentioned conventional technology is based on a see-through HMD and cannot be applied directly to a stereoscopic display. For example, in a stereoscopic display, it is not possible to place a real object behind the display that the user views, as is the case with an HMD.

[0009] Furthermore, stereoscopic displays generally do not include a distance measurement system like the conventional technology described above, so if you try to apply the conventional technology described above, you need to introduce this distance measurement system, which is cumbersome.

[0010] Therefore, the present disclosure proposes a display control device and a calibration method that can achieve simple and highly accurate calibration of display parameters in a stereoscopic display.

[0011] In order to solve the above problem, a display control device according to one embodiment of the present disclosure includes a display control unit, a receiving unit, and a correcting unit. The display control unit controls the stereoscopic display to display a guide object representing a parallax corresponding to a physical display surface of the stereoscopic display based on a first display parameter related to stereoscopic vision. The receiving unit receives confirmation information indicating a user's confirmation result after displaying the guide object. The correcting unit corrects the first display parameter based on a second display parameter related to stereoscopic vision that is set in the stereoscopic display after receiving the confirmation information.

[0012] 15 is a schematic explanatory diagram of a display control device according to an embodiment of the present disclosure. FIG. 16 is a schematic explanatory diagram (part 1) of a calibration method according to an embodiment of the present disclosure. FIG. 17 is a schematic explanatory diagram (part 2) of a calibration method according to an embodiment of the present disclosure. FIG. 18 is a schematic explanatory diagram (part 3) of a calibration method according to an embodiment of the present disclosure. FIG. 19 is a schematic explanatory diagram (part 4) of a calibration method according to an embodiment of the present disclosure. A block diagram showing an example configuration of a display control device according to an embodiment of the present disclosure. FIG. 19 is a block diagram showing an example configuration of a parameter setting unit shown in FIG. 6. FIG. 19 is an explanatory diagram of a basic calibration method executed by a display control device. FIG. 20 is a flowchart showing a processing procedure of basic calibration processing executed by a display control device. FIG. 21 is an explanatory diagram of definitions related to calibration of display parameters. FIG. 22 is a diagram showing an example display of an index and a guide object in calibration processing of an IPD. FIG. 23 is an explanatory diagram of a calibration method of an IPD executed by a display control device. FIG. 24 is a flowchart showing a processing procedure of calibration processing of an IPD executed by a display control device. FIG. 25 is a diagram showing an example display of an index and a guide object in calibration processing of a yaw angle. FIG. 26 is a diagram showing an example arrangement of a jig corresponding to FIG. 14. FIG. 27 is an explanatory diagram of a calibration method of a yaw angle executed by a display control device. FIG. 28 is a flowchart showing a processing procedure of calibration processing of a yaw angle executed by a display control device. FIG. 29 is an explanatory diagram of a calibration method according to a modified example. FIG. 29 is a hardware configuration diagram showing an example of a computer that realizes the functions of a display control device.

[0013] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. In the following embodiments, the same components are designated by the same reference numerals, and redundant description will be omitted.

[0014] In the following, a display control device according to an embodiment of the present disclosure (hereinafter referred to as "this embodiment") is assumed to be a display control device 10 that controls the display of the stereoscopic display 5 shown in Fig. 1 and subsequent figures. In the following, a calibration method according to this embodiment is assumed to be a method for calibrating display parameters related to stereoscopic vision of the stereoscopic display 5, executed by the display control device 10. In the following, the display parameters related to stereoscopic vision of the stereoscopic display 5 may be abbreviated to "parameters".

[0015] The present disclosure will be described in the following order: 1. Overview 2. Configuration example of a display control device 3. Processing procedure 3-1. Basic calibration processing 3-2. IPD calibration processing 3-3. Yaw angle calibration processing 4. Modified examples 4-1. When there is a distance between the display surface and the jig 4-2. How to use display parameters 4-3. GUI for calibration 4-4. Other 5. Hardware configuration 6. Conclusion

[0016] <<1. Overview>> Fig. 1 is a schematic explanatory diagram of a display control device 10 according to an embodiment of the present disclosure. Fig. 2 is a schematic explanatory diagram (part 1) of a calibration method according to an embodiment of the present disclosure. Fig. 3 is a schematic explanatory diagram (part 2) of a calibration method according to an embodiment of the present disclosure. Fig. 4 is a schematic explanatory diagram (part 3) of a calibration method according to an embodiment of the present disclosure. Fig. 5 is a schematic explanatory diagram (part 4) of a calibration method according to an embodiment of the present disclosure.

[0017] The display control device 10 according to this embodiment is a device that controls the display of a stereoscopic display 5. As shown in FIG.

[0018] The camera 3 is provided so as to be able to capture an image of the face of the user U at a predetermined angle of view FV. The position of the camera 3 is fixed relative to the position of the stereoscopic display 5. Based on the control of the display control device 10, the stereoscopic display 5 displays a virtual object to be displayed stereoscopically on the display surface 5a so that the user U can visually recognize the virtual object as if it were present in real space. Fig. 1 schematically shows a state in which a cat, which is a virtual object, is stereoscopically displayed by being drawn three-dimensionally on the display surface 5a.

[0019] 1 illustrates an example in which the display control device 10, the camera 3, and the stereoscopic display 5 are mounted together in a single housing, but the configuration of this embodiment is not limited to this example. For example, these may be provided as separate devices at positions separated from each other, and may communicate with each other via wire or wirelessly.

[0020] The display surface 5a is also provided so as to be able to impart directionality to light rays emitted from pixels arranged on the display panel.

[0021] The display control device 10 estimates the positions of the left and right eyes of the user U in real time by tracking using the camera 3. Then, the display control device 10 generates a group of parallax images directed toward each of the estimated positions, and displays the generated group of parallax images separately for each of the left and right eyes according to the above-mentioned directionality. The estimation of the positions of the left and right eyes and the generation and display of the group of parallax images are performed using the above-mentioned display parameters.

[0022] If the display parameters are not set correctly on the stereoscopic display 5, a discrepancy will occur between the virtual space in which the virtual objects are displayed and the real space perceived by the user U. Specifically, in this case, for example, the display in the virtual space may be distorted, or a display discrepancy may occur between a real object and a virtual object that are intended to be displayed simultaneously.

[0023] In other words, the display parameters of the stereoscopic display 5 need to be properly set by calibration. However, if a calibration method according to existing technology is used, it is difficult to achieve simple and highly accurate calibration of the display parameters of the stereoscopic display 5.

[0024] Therefore, the calibration method according to this embodiment utilizes the fact that virtual objects displayed on the display surface 5 a are displayed correctly without depending on display parameters. In addition, this embodiment realizes a simple and highly accurate calibration method by displaying indices on the display surface 5 a and placing real objects with guaranteed accuracy in accordance with the indices.

[0025] To provide a more specific explanation, in the stereoscopic display 5, in order to provide a proper display, an appropriate group of parallax images must be generated from the position and angle of the face of the user U and the IPD in accordance with the distance of the virtual object to be displayed from the display surface 5a-V (see FIG. 5) in the virtual space to the user U. If the generated group of parallax images is not appropriate, the virtual object cannot be displayed correctly in accordance with the coordinate system of the virtual space.

[0026] Consider a case where a virtual object is desired to be stereoscopically displayed at a position O1 away from the display surface 5a, as shown in the left diagram of Figure 2. In this case, a ray is projected from the optical center of one of the user U's eyes toward the virtual object, and the intersection point with the display surface 5a is calculated, and the color to be displayed at the intersection point on the display surface 5a is calculated.

[0027] This is performed for all points in the virtual space to calculate an image to be displayed on the display surface 5a for one eye. The same procedure is performed for the other eye to calculate an image to be displayed on the display surface 5a for the other eye.

[0028] The multiple images generated in this manner are a group of parallax images, and by presenting these groups of parallax images to the left and right eyes, respectively, the user U can be made to visually perceive the virtual object in three dimensions.

[0029] However, as shown in the center diagram of Figure 2, if the display parameters are not appropriate, for example, if the actual positions of the left and right eyes are different from the positions at the time of generating the disparity image group, when the disparity image group is generated using the above-mentioned procedure, the user U will end up viewing a virtual object that should be viewed at position O1 at, for example, position P1.

[0030] 2, when the distance between the display surface 5a and the virtual object is 0, the intersections on the display surface 5a corresponding to both eyes of the user U coincide. In other words, in this case, the intersections on the display surface 5a are uniquely determined regardless of the positions of the eyes, so that the virtual object can always be displayed appropriately regardless of the display parameters.

[0031] Therefore, by displaying an index on the display surface 5a and placing a real object having a shape with precision corresponding to this index on the display surface 5a in accordance with the index, the appropriateness of the display parameters can be determined based on whether or not there is a display misalignment in the display of a virtual object corresponding to this real object.

[0032] In the calibration according to this embodiment, a GUI (Graphical User Interface) for correcting this display misalignment is displayed, and the display parameters are corrected so as to correct the display misalignment based on input from the user U via this GUI. In the present disclosure, the display parameters before correction may be referred to as first display parameters, and the display parameters adjusted for correction may be referred to as second display parameters.

[0033] The physical object to be placed on the display surface 5a will now be described in more detail. In this embodiment, a calibration jig J1 is used as the physical object to be placed on the display surface 5a. As shown in FIGS. 3 and 4, the jig J1 is formed, for example, in the shape of a transparent cube with an open bottom. Note that FIG. 3 is a schematic perspective view of the jig J1 as seen from the bottom side. Meanwhile, FIG. 4 is a schematic perspective view of the jig J1 as seen from the top side.

[0034] 3 and 4, the jig J1 is provided with a reference frame F1 whose upper surface is a reference surface. This reference frame F1 is used during calibration of the IPD, which will be described later.

[0035] The reference frame F1 may be any frame that allows the user U to visually recognize the depth position of the upper surface of the jig J1 relative to the display surface 5a, and may simply be a line drawn on the upper surface of the jig J1. Alternatively, the reference frame F1 may be formed by the outline of tape or the like attached to the upper surface of the jig J1.

[0036] The jig J1 has, for example, a side length of 100 mm. The dimensions of this jig J1 are guaranteed to be accurate so that if the jig J1 is displayed in a virtual space as a virtual object, the dimensions will match those of the virtual object. The jig J1 is made of, for example, acrylic, but the material is not particularly limited as long as it does not interfere with the display of the virtual object on the stereoscopic display 5. The jig J1 may also be formed as an object consisting only of a framework, for example, a wire frame.

[0037] 5, the display control device 10 draws an index M1 indicating the placement position of the jig J1 on a display surface 5a-V, which is a virtual display surface corresponding to the display surface 5a in the virtual space where the virtual object is displayed. The index M1 is displayed on the display surface 5a as a guideline that coincides with the bottom surface of the jig J1 based on the shape information of the jig J1.

[0038] The display control device 10 also draws the guide object G1 together with the index M1 in the virtual space. The user U then places the jig J1 so that its bottom surface exactly matches the index M1 displayed on the display surface 5 a, and checks whether the guide object G1, which is displayed, for example, overlapping the jig J1, is misaligned with the jig J1.

[0039] If there is a display misalignment, the user U makes an input via a GUI (not shown) to eliminate the display misalignment of the guide object G1, and performs an operation to confirm the input as a confirmation result. In the example of Fig. 5, the user U makes an input so that the depth position of the top surface of the jig J1 indicated by the reference frame F1 matches the depth position of the guide object G1. Note that the example is not limited to that shown in Fig. 5, and for example, a cubic guide object corresponding to the jig J1 itself may be displayed, and the user U's input may be received so that the display misalignment of the guide object relative to the jig J1 is eliminated.

[0040] Then, the display control device 10 receives the confirmation result confirmed by the user U as confirmation information, and corrects the display parameters based on the received confirmation information.

[0041] This makes it possible to easily and highly accurately calibrate the display parameters of the stereoscopic display 5.

[0042] The confirmation information is information relating to how the guide object G1 appears when viewed from the user U. The confirmation information is information indicating at least whether the guide object G1 appears correctly to the user U. The confirmation information also includes position and orientation information relating to the position and orientation of the guide object adjusted by the user as parameters of the guide object, shape information of the guide object, information for confirming the set second display parameters, and the like.

[0043] An example of the configuration of the display control device 10 to which the calibration method according to this embodiment is applied will be described in more detail below.

[0044] <<2. Configuration Example of Display Control Device>> Fig. 6 is a block diagram showing a configuration example of a display control device 10 according to an embodiment of the present disclosure. Fig. 7 is a block diagram showing the configuration of a parameter setting unit 12e shown in Fig. 6. Note that Figs. 6 and 7 show only components necessary for explaining features of the embodiment of the present disclosure, and descriptions of general components are omitted.

[0045] 6 and 7 are functional concepts and do not necessarily have to be physically configured as shown. For example, the specific form of distribution and integration of each block is not limited to that shown, and all or part of them can be functionally or physically distributed and integrated in any unit depending on various loads, usage conditions, etc.

[0046] In addition, in the description using FIG. 6 or FIG. 7, the description of components that have already been described may be simplified or omitted.

[0047] 6, the display control device 10 includes a storage unit 11 and a control unit 12. The display control device 10 is also connected to a camera 3, a stereoscopic display 5, and an operation unit 7. The display control device 10 is connected to the camera 3, the stereoscopic display 5, and the operation unit 7 by wire or wirelessly.

[0048] The camera 3 and the stereoscopic display 5 have already been described, and therefore will not be described here. The operation unit 7 is an operation component for operating a GUI for calibrating display parameters displayed on the stereoscopic display 5. The operation unit 7 is realized by, for example, a keyboard, a mouse, a dedicated GUI controller, or the like.

[0049] The storage unit 11 is realized by a storage device such as a RAM (Random Access Memory), a ROM (Read Only Memory), a flash memory, or an HDD (Hard Disk Drive). In the example of Fig. 6, the storage unit 11 stores jig shape information 11a, guide object information 11b, and parameter information 11c. The storage unit 11 also stores a program according to this embodiment (not shown).

[0050] The jig shape information 11a is model information that indicates the shape of the jig J1. The jig shape information 11a is defined, for example, as a set of vertices of the jig J1 and a mesh in which each vertex is a node. The jig shape information 11a includes an index that specifies an index face corresponding to the above-mentioned index M1 among the faces of the jig J1.

[0051] The guide object information 11b is information about each guide object including the above-mentioned guide object G1. The guide object information 11b includes definition information about the shape, position, etc. of the guide object G1 according to the jig shape information 11a. The guide object information 11b also includes coordinates that are the reference position for displaying the guide object G1 according to the jig shape information 11a.

[0052] The parameter information 11c is information in which display parameters (first display parameters before correction) of the stereoscopic display 5 are set. The display control device 10 controls the display on the stereoscopic display 5 based on the display parameters set in this parameter information 11c.

[0053] The control unit 12 corresponds to a so-called processor. The control unit 12 is realized by, for example, a central processing unit (CPU), a micro processing unit (MPU), a graphics processing unit (GPU), etc. The control unit 12 executes a program according to this embodiment (not shown) stored in the storage unit 11, using RAM as a working area. The control unit 12 can also be realized by, for example, an integrated circuit such as an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA).

[0054] The control unit 12 has an acquisition unit 12a, a coordinate calculation unit 12b, a display control unit 12c, a receiving unit 12d, and a parameter setting unit 12e, and realizes or executes the functions and actions of information processing described below.

[0055] The acquisition unit 12a acquires, from the camera 3, a camera image captured by the camera 3. The acquisition unit 12a also acquires, from the storage unit 11, jig shape information 11a, guide object information 11b, and parameter information 11c.

[0056] The coordinate calculation unit 12b calculates the coordinates of the jig J1 in the virtual space. The display control unit 12c controls the display of the index M1 at the coordinates calculated by the coordinate calculation unit 12b. The display control unit 12c also controls the display of the guide object G1 at a reference position corresponding to the index M1.

[0057] The receiving unit 12d receives confirmation information input by the user U via the operation unit 7. The parameter setting unit 12e sets display parameters. As shown in Fig. 7, the parameter setting unit 12e includes an initial setting unit 12ea and a correction unit 12eb.

[0058] The initial setting unit 12ea sets appropriate initial values ​​for the parameter information 11c. The correcting unit 12eb corrects the display parameters of the parameter information 11c based on the confirmation information received by the receiving unit 12d.

[0059] <<3. Processing Procedure>> <3-1. Basic Calibration Processing> Next, a specific example of the calibration processing executed by the display control device 10 will be described. First, the basic calibration processing will be described. FIG. 8 is an explanatory diagram of the basic calibration processing executed by the display control device 10.

[0060] As shown in Figure 8, the coordinate calculation unit 12b receives input of jig shape information 11a, the index surface indicated by the jig shape information 11a, the index position at which index M1 is displayed, and the position and orientation of the display surface 5a-V in the virtual space.

[0061] The coordinate calculation unit 12b outputs the index M1 and the index position to the display control unit 12c, and the display control unit 12c controls the display of the index M1 so that the index M1 is displayed on the stereoscopic display 5 at the specified index position.

[0062] Furthermore, the coordinate calculation unit 12b converts the shape information of the jig J1 defined in the local coordinate system of the jig shape information 11a into shape information Pos in the coordinate system in the virtual space by the following formula (a): ｖｉｒｔｕａｌ and outputs it to the parameter setting unit 12e.

[0063]

[0064] Here, pos ｌｏｃａｌ are the coordinates in the local coordinate system of the jig J1, R ｄｉｓｐｌａｙ are transformed coordinates obtained by transforming the coordinates in this local coordinate system into the display coordinate system defined on the display surface 5a. ｄｉｓｐｌａｙ can be obtained from the correspondence between the position in the local coordinate system of each vertex arranged on the display surface 5a as the index M1 and the position on the display surface 5a. ｖｉｒｔｕａｌ are transformation coordinates from the display coordinate system to the virtual space coordinate system, which can be determined from the position and orientation of the display surface 5a-V in the virtual space.

[0065] The parameter setting unit 12e calculates the shape information Pos of the jig J1 in the virtual space input from the coordinate calculation unit 12b. ｖｉｒｔｕａｌ Based on the above, the shape, position and currently set parameters (first display parameters) of the guide object G1 are output to the display control unit 12c.

[0066] The display control unit 12c controls the display of the guide object G1 on the stereoscopic display 5 based on the shape, position and parameters of the guide object G1 input from the parameter setting unit 12e.

[0067] The user U places the jig J1 on the display surface 5a so that the displayed index M1 and a specific surface of the jig J1 (for example, the bottom surface) are exactly aligned. Then, the user U checks whether there is any display misalignment of the guide object G1, which is displayed so as to appear to overlap the jig J1, with respect to the jig J1.

[0068] If the display parameters before correction are appropriate, the guide object G1 appears to be displayed so that the portion corresponding to the jig J1 matches without any misalignment. However, if the display parameters are inappropriate, the display will appear misaligned. If the display is misaligned, the user U performs user input via the operation unit 7, and the parameter setting unit 12e changes the display parameters based on the user input.

[0069] The display control unit 12c performs display control to change the display on the stereoscopic display 5 in accordance with the changed display parameters. The user U changes the display parameters by this input until the display misalignment of the guide object G1 appears to be resolved. When the display misalignment appears to be resolved, the user U performs an operation to confirm the display parameters, and the parameter setting unit 12e outputs the confirmed display parameters as appropriate parameters.

[0070] The appropriate parameters output from the parameter setting unit 12e can be used as display parameters for performing appropriate display as they are, or can be used to determine other parameters from multiple parameters obtained by changing conditions, etc. This point will be described later in the modified example.

[0071] The processing procedure of the basic calibration processing described with reference to Fig. 8 is shown in Fig. 9. Fig. 9 is a flowchart showing the processing procedure of the basic calibration processing executed by the display control device 10.

[0072] The parameter setting unit 12e acquires coordinates of a reference position for displaying the guide object G1 from the shape information of the jig J1 converted into the virtual space coordinate system (step S101). Then, the display control unit 12c displays the guide object G1 at the same position as the reference position acquired by the parameter setting unit 12e (step S102).

[0073] Next, the parameter setting unit 12e acquires parameters set by the user U so that the reference position and the guide object G1 appear to be in the same position (step S103). Then, the parameter setting unit 12e outputs the set parameters as appropriate parameters (step S104), and ends the process.

[0074] Next, a processing procedure for calibrating specific display parameters will be specifically described in addition to the basic calibration processing described with reference to FIGS.

[0075] The positions of the eyes of the user U relative to the display surface 5a are necessary for displaying images on the stereoscopic display 5. Although there are methods for directly estimating the positions of the eyes, in this embodiment, the position and posture of the face of the user U are estimated, and the positions of the eyes are estimated from the position and posture.

[0076] Specifically, the positions of the eyes of the user U are estimated based on the roll angle, yaw angle, and IPD, with the position between the eyes of the user U as the origin. In this embodiment, the display parameters related to the stereoscopic vision of the stereoscopic display 5 are corrected based on the results of estimating the positions of the eyes. The display parameters to be corrected include at least the yaw angle and the IPD.

[0077] The calibration of these display parameters is defined as shown in FIG. 10 . FIG. 10 is an explanatory diagram of the definition of the calibration of the display parameters. As shown in FIG. 10 , in this embodiment, an X-axis is defined with the right direction of the display surface 5 a as the positive direction and the left direction as the negative direction. Also, a Y-axis is defined with the upward direction of the display surface 5 a as the positive direction and the downward direction as the negative direction. Also, a Z-axis is defined that is perpendicular to the XY plane, and with the direction toward the user U as the positive direction and the direction away from the user U as the negative direction.

[0078] The subsequent three-dimensional position and rotation are determined in this XYZ Cartesian coordinate system. The roll angle is determined by the face position Pos, which is the position between the eyes of the user U. Ｆａｃｅ The yaw angle is the rotation angle of the face of the user U around an axis that passes through the face position Pos and is parallel to the Z axis. Ｆａｃｅ The IPD is the rotation angle of the face of the user U around an axis that passes through the center and is parallel to the Y axis.

[0079] These face positions Pos Ｆａｃｅ , Roll angle, Yaw angle, and IPD are first estimated in the camera coordinate system using, for example, camera 3. Then, the estimation results in the camera coordinate system are transformed into the display coordinate system using a coordinate transformation matrix to the display coordinate system. The coordinate transformation matrix is ​​generally a 4x4 matrix, and includes information on translation and rotation for the coordinate transformation.

[0080] Rotation by Roll angle is R Ｒｏｌｌ , the rotation by the Yaw angle is R Ｙａｗ When expressed as above, the left eye position Pos of the user U ＬｅｆｔＥｙｅ can be calculated by the following formula (b).

[0081]

[0082] In addition, the right eye position Pos of the user U ＲｉｇｈｔＥｙｅ can be calculated by the following formula (c):

[0083]

[0084] <3-2. In the Case of IPD Calibration Processing> A case where the IPD calibration processing is performed among the display parameters to be corrected will be described. Fig. 11 is a diagram showing a display example of the index M1 and the guide object G1 in the IPD calibration processing.

[0085] As shown in FIG. 11, in the IPD calibration process, the display control device 10 draws an index M1 indicating the placement position of the jig J1 at the index position on the display surface 5a-V corresponding to the display surface 5a.

[0086] The display control device 10 also controls the stereoscopic display 5 to display the guide object G11 together with the index M1 at the same depth position as the reference plane, with the top surface of the jig J1 as the reference plane.

[0087] That is, the display control device 10 draws an index M1 indicating the placement position of a jig J1 having a predetermined shape on the virtual display surface (display surface 5a-V) in the virtual space of the stereoscopic display 5. In addition, the display control device 10 draws a guide object G11 at a predetermined reference position that corresponds to the shape of the jig J1 and is spaced apart from the virtual display surface.

[0088] In the IPD calibration process, the guide object G11 is displayed as a square with a cross drawn on it that fits within the reference frame F1 provided on the upper surface of the jig J1, for example, as shown in Fig. 11. Note that the reference frame F1 is not displayed, as indicated by only the dashed line in Fig. 11.

[0089] An appropriate value may be used as the initial setting value of the IPD. If the IPD is appropriate, the guide object G11 appears to be displayed at the same depth position as the depth position of the reference frame F1 with respect to the reference frame F1. On the other hand, if the IPD is inappropriate, the guide object G11 appears to be displayed with its depth position displaced along either the positive or negative direction of the Z axis in the figure with respect to the depth position of the reference frame F1 (see "Correcting geometric distortions in stereoscopic 3D imaging Zhongpai Gao, Alex Hwang, Guangtao Zhai, Eli Peli").

[0090] Therefore, in this case, the user U sets the IPD via the operation unit 7 so that the guide object G11 appears to be at the same depth position relative to the depth position of the reference frame F1. At this time, the user U performs an input to move the guide object G11 along the Z-axis direction via the operation unit 7, for example. Then, the display control device 10 adopts the IPD that the user U has determined to be at the same depth position as the appropriate IPD.

[0091] Fig. 12 is an explanatory diagram of an IPD calibration method executed by the display control device 10. Note that Fig. 12 corresponds to Fig. 8 already shown, and therefore differences from Fig. 8 will be mainly described here.

[0092] That is, in the IPD calibration process, as shown in FIG. 12, the parameter setting unit 12e outputs the shape and position of the guide object G11 and the currently set IPD to the display control unit 12c.

[0093] The display control unit 12c controls the display of the guide object G11 on the stereoscopic display 5 based on the shape, position and IPD of the guide object G11 input from the parameter setting unit 12e.

[0094] The user U places the jig J1 on the display surface 5a so that the displayed index M1 and the bottom surface of the jig J1 are exactly aligned. Then, the user U checks whether the guide object G11 appears at the same depth position as the depth position of the reference frame F1.

[0095] Then, the user U performs user input via the operation unit 7 until the guide object G11 and the reference frame F1 appear to be at the same depth position, and the parameter setting unit 12e changes the IPD based on the user input.

[0096] The display control unit 12c performs display control to change the display on the stereoscopic display 5 in accordance with the changed IPD. When the user U sees the guide object G11 and the reference frame F1 at the same depth position, the user U performs an operation to confirm the IPD, and the parameter setting unit 12e outputs the confirmed IPD as the appropriate IPD.

[0097] 13 shows the procedure of the IPD calibration process described with reference to Fig. 11 and Fig. 12. Fig. 13 is a flowchart showing the procedure of the IPD calibration process executed by the display control device 10.

[0098] The parameter setting unit 12e acquires the depth position of the reference plane for displaying the guide object G11 from the shape information of the jig J1 converted into the virtual space coordinate system (step S201). Then, the display control unit 12c displays the guide object G11 at the same depth position as the reference plane acquired by the parameter setting unit 12e (step S202).

[0099] Next, the parameter setting unit 12e acquires the IPD set by the user U so that the reference plane and the guide object G11 appear to be at the same depth position (step S203). Then, the parameter setting unit 12e outputs the set IPD as the appropriate IPD (step S204), and ends the process.

[0100] <3-3. Calibration Process of Yaw Angle> Next, a case where calibration process of the yaw angle, one of the display parameters to be corrected, is described. Fig. 14 is a diagram showing a display example of the index M1 and the guide object G12 in the calibration process of the yaw angle. Fig. 15 is a diagram showing an example of the arrangement of the jig J1 corresponding to Fig. 14.

[0101] As shown in FIG. 14, in the yaw angle calibration process, the display control device 10 draws an index M1 indicating the placement position of the jig J1 at the index position on the virtual display surface 5a-V corresponding to the display surface 5a.

[0102] As shown in FIGS. 14 and 15, in the yaw angle calibration process, the index M1 serves as a guideline for arranging the two jigs J1-1 and J1-2 side by side on the display surface 5a.

[0103] In addition, in the calibration process of the Yaw angle, the upper sides of two of the jigs J1 shown in the closed curve R1 in Fig. 15 become the reference sides for displaying the guide objects G12. As shown in Fig. 14, in the calibration process of the Yaw angle, the display control device 10 displays two cone-shaped guide objects G12 together with an index M1. The two guide objects G12 are displayed so that the straight line connecting the vertices is parallel to this reference side. Note that in Fig. 14, the dashed line indicates that this straight line is not actually displayed, but it may actually be displayed.

[0104] If the Yaw angle is appropriate, the two guide objects G12 appear to be displayed as if their arrangement is parallel to the reference side (or as if the vertices point to both ends of the reference side). On the other hand, if the Yaw angle is not appropriate, the two guide objects G12 appear to be displayed as if their arrangement is not parallel to the reference side.

[0105] Therefore, in this case, the user U sets the Yaw angle via the operation unit 7 so that the arrangement of the two guide objects G12 appears parallel to the reference side. At this time, the user U performs an input via the operation unit 7 to simultaneously rotate the two guide objects G12 around one rotation axis perpendicular to the virtual space plane. Then, the display control device 10 adopts the Yaw angle that the user U has determined to appear parallel as the appropriate Yaw angle.

[0106] Fig. 16 is an explanatory diagram of a yaw angle calibration method executed by the display control device 10. Note that Fig. 16 corresponds to Fig. 8 and Fig. 12 already shown, and therefore differences from Fig. 8 and Fig. 12 will be mainly described here.

[0107] That is, in the calibration process of the Yaw angle, as shown in FIG. 16, the parameter setting unit 12e outputs the shape, position and currently set Yaw angle of the guide object G12 to the display control unit 12c.

[0108] The display control unit 12c controls the display of the guide object G12 on the stereoscopic display 5 based on the shape, position, and Yaw angle of the guide object G12 input from the parameter setting unit 12e.

[0109] The user U places the jig J1 on the display surface 5a so that the displayed index M1 and the bottom surface of the jig J1 are exactly aligned. Then, the user U checks whether the two guide objects G12 appear to be aligned parallel to the reference edge.

[0110] Then, the user U performs user input via the operation unit 7 until the arrangement of the two guide objects G12 and the reference side appear parallel, and the parameter setting unit 12e changes the Yaw angle based on the user input.

[0111] The display control unit 12c performs display control to change the display of the stereoscopic display 5 according to the changed Yaw angle. When the arrangement of the two guide objects G12 and the reference side appear parallel to each other, the user U performs an operation to confirm the Yaw angle, and the parameter setting unit 12e outputs the confirmed Yaw angle as the appropriate Yaw angle.

[0112] The processing procedure of the yaw angle calibration processing described using Fig. 14 to Fig. 16 is shown in Fig. 17. Fig. 17 is a flowchart showing the processing procedure of the yaw angle calibration processing executed by the display control device 10.

[0113] The parameter setting unit 12e acquires the coordinates of the reference side on which the guide object G12 is to be displayed from the shape information of the jig J1 converted into the virtual space coordinate system (step S301). Then, the display control unit 12c displays the two guide objects G12 parallel to the reference side acquired by the parameter setting unit 12e (step S302).

[0114] Next, the parameter setting unit 12e acquires the Yaw angle set by the user U so that the reference side and the guide object G12 appear parallel to each other (step S303). Then, the parameter setting unit 12e outputs the set Yaw angle as the appropriate Yaw angle (step S304), and ends the process.

[0115] <<4. Modifications>> Furthermore, several modifications can be made to the above-described embodiments of the present disclosure.

[0116] <4-1. When there is a distance between the display surface and the jig> The stereoscopic display 5 may have a light distribution member such as a lenticular lens or a parallax barrier mounted in layers on a display surface 5a on which pixels are arranged, and the openings in the light distribution member may be configured to impart directionality to light rays emitted from the pixels. Figure 18 is an explanatory diagram of a calibration method according to a modified example.

[0117] In this case, the jig J1 cannot be placed directly on the display surface 5a. In that case, as shown in Fig. 18, for example, by obtaining the thickness of the light distribution member from its design value or actual measurement value and using it as the "thickness of the contact surface," it is possible to check whether the display is appropriate even if there is a distance between the display surface 5a and the jig J1.

[0118] The following formula (d) is the coordinate transformation formula in this case. ｇａｐ represents a coordinate transformation that translates the coordinate system by the distance mentioned above. Note that depending on the actual setup, this transformation may also include rotation and scale transformations.

[0119]

[0120] <4-2. Usage of Display Parameters> Furthermore, the appropriate parameters output in the above-described embodiment are normally used as display parameters of the stereoscopic display 5 to improve the display on the stereoscopic display 5. Note that, for example, since the IPD varies depending on the individual user U, it is preferable to perform the IPD calibration process according to this embodiment each time a different user U watches the stereoscopic display 5.

[0121] Furthermore, the optimum parameters may also be used in other ways, for example, to optimize or calibrate a process for estimating other display parameters.

[0122] That is, another display parameter is calculated from one or more appropriate parameters calculated by this embodiment, each display parameter estimated at that time, and a set of data used for the estimation. For example, if the estimation of the Yaw angle of the face by the camera 3 deviates due to individual differences or the lighting environment of the room, but it is known that the deviation changes linearly, the estimated Yaw angle (Yaw ｅｓｔ ) is input and the corrected Yaw angle (Yaw ｃｏｒ ) can be considered.

[0123]

[0124] In this case, α and β are display parameters related to the display, and these can be derived by a method such as the least squares method using the estimated Yaw angle and multiple sets of appropriate Yaw angles obtained in this embodiment.

[0125] <4-3. GUI for Calibration> Regarding the calibration process by the user U via the operation unit 7, the display control device 10 Ｆａｃｅ The stereoscopic display 5 may be controlled to display a GUI for directly changing the yaw angle or IPD. In this case, the user U directly adjusts the display parameters, thereby indirectly adjusting the position and orientation of the guide object.

[0126] Furthermore, the display control device 10 may control the stereoscopic display 5 to display a GUI that indicates the position and orientation of the guide object as a parameter of the guide object. In this case, the user U can adjust the face position Pos Ｆａｃｅ , and indirectly adjust the Yaw angle or IPD. Specifically, the display control device 10 acquires position and orientation information of the guide object as confirmation information, and reverse-calculates display parameters to be used for correction based on the confirmation information. Note that even if the user U adjusts the position and orientation of the displayed guide object, the position and orientation information of the guide object as internal parameters is not changed.

[0127] <4-4. Others> Furthermore, among the processes described in the above-described embodiments of the present disclosure, all or part of the processes described as being performed automatically can be performed manually, or all or part of the processes described as being performed manually can be performed automatically using known methods. In addition, the information including the processing procedures, specific names, various data, and parameters shown in the above documents and drawings can be changed as desired unless otherwise specified. For example, the various information shown in each drawing is not limited to the information shown in the drawings.

[0128] Furthermore, the components of each device shown in the figure are conceptual functional components and do not necessarily have to be physically configured as shown in the figure. In other words, the specific form of distribution and integration of each device is not limited to that shown in the figure, and all or part of them can be functionally or physically distributed and integrated in any unit depending on various loads, usage conditions, etc.

[0129] The above-described embodiments of the present disclosure can be combined as appropriate within the scope of the present disclosure without causing any inconsistency in the processing content. The order of the steps shown in the sequence diagrams or flowcharts of the present embodiments can be changed as appropriate.

[0130] <<5. Hardware Configuration>> The display control device 10 according to the embodiment of the present disclosure described above is realized by, for example, a computer 1000 configured as shown in Fig. 19. Fig. 19 is a hardware configuration diagram showing an example of the computer 1000 that realizes the functions of the display control device 10. The computer 1000 has a CPU 1100, a RAM 1200, a ROM 1300, a secondary storage device 1400, a communication interface 1500, and an input / output interface 1600. The components of the computer 1000 are connected by a bus 1050.

[0131] The CPU 1100 operates and controls each component based on programs stored in the ROM 1300 or the secondary storage device 1400. For example, the CPU 1100 loads the programs stored in the ROM 1300 or the secondary storage device 1400 into the RAM 1200 and executes processing corresponding to the various programs.

[0132] The ROM 1300 stores boot programs such as a Basic Input Output System (BIOS) that is executed by the CPU 1100 when the computer 1000 is started, and programs that depend on the hardware of the computer 1000 .

[0133] The secondary storage device 1400 is a computer-readable recording medium that non-temporarily records programs executed by the CPU 1100 and data used by such programs. Specifically, the secondary storage device 1400 is a recording medium that records the programs according to this embodiment.

[0134] The communication interface 1500 is an interface for connecting the computer 1000 to an external network 1550. For example, the CPU 1100 receives data from other devices and transmits data generated by the CPU 1100 to other devices via the communication interface 1500.

[0135] The input / output interface 1600 is an interface for connecting the input / output device 1650 and the computer 1000. For example, the CPU 1100 receives data from an input device such as a keyboard or a mouse via the input / output interface 1600. The CPU 1100 also transmits data to an output device such as a display, a speaker, or a printer via the input / output interface 1600. The input / output interface 1600 may also function as a media interface for reading programs and the like recorded on a predetermined recording medium. Examples of media include optical recording media such as a DVD (Digital Versatile Disc) or a PD (Phase Change Rewritable Disc), magneto-optical recording media such as an MO (Magneto-Optical Disk), tape media, magnetic recording media, and semiconductor memories.

[0136] For example, when the computer 1000 functions as the display control device 10, the CPU 1100 of the computer 1000 executes a program loaded onto the RAM 1200 to realize the functions of the control unit 12. The secondary storage device 1400 stores the program according to the present disclosure and data in the storage unit 11. The CPU 1100 reads and executes the program data 1450 from the secondary storage device 1400, but as another example, the CPU 1100 may obtain these programs from another device via an external network 1550.

[0137] <<6. Conclusion>> As described above, according to an embodiment of the present disclosure, the display control device 10 includes the display control unit 12c, the receiving unit 12d, and the correcting unit 12eb. The display control unit 12c controls the stereoscopic display 5 to display guide objects G1, G11, and G12 representing parallax corresponding to the display surface 5a (corresponding to an example of a "physical display surface") of the stereoscopic display 5, based on first display parameters related to stereoscopic vision. The receiving unit 12d receives confirmation information indicating a confirmation result of the user U after displaying the guide objects G1, G11, and G12. The correcting unit 12eb corrects the first display parameters based on second display parameters related to stereoscopic vision set in the stereoscopic display 5 after receiving the confirmation information. This makes it possible to realize simple and highly accurate calibration of the display parameters in the stereoscopic display 5.

[0138] Although the embodiments of the present disclosure have been described above, the technical scope of the present disclosure is not limited to the above-described embodiments, and various modifications are possible within the scope of the gist of the present disclosure. Furthermore, components of different embodiments and modifications may be combined as appropriate.

[0139] Furthermore, the effects of each embodiment described in this specification are merely examples and are not intended to be limiting, and other effects may also be obtained.

[0140] Note that the present technology can also be configured as follows. (1) A display control device comprising: a display control unit that controls a stereoscopic display to display a guide object representing a parallax corresponding to a physical display surface of the stereoscopic display based on a first display parameter related to stereoscopic vision; a receiving unit that receives confirmation information indicating a user's confirmation result after the guide object is displayed; and a correction unit that corrects the first display parameter based on a second display parameter related to stereoscopic vision set in the stereoscopic display after receiving the confirmation information. (2) The display control device according to (1), wherein the display control unit draws an index indicating an arrangement position of a real object having a predetermined shape on a virtual display surface of the stereoscopic display in a virtual space, and draws the guide object at a predetermined reference position that corresponds to the shape and is separated from the virtual display surface. (3) The display control device according to (2), wherein the receiving unit receives the confirmation information regarding how the guide object appears as seen by the user who has arranged the real object on the physical display surface so as to match the index. (4) The display control device according to (2) or (3), wherein the confirmation information includes position and orientation information of the guide object adjusted by the user as a parameter of the guide object. (5) The display control device according to (2), (3), or (4), wherein the physical object is a transparent jig formed in a cubic shape. (6) The display control device according to (5), wherein the first display parameter and the second display parameter each include at least a Yaw angle which is a rotation angle of the user's face around an axis parallel to the up-down direction of the physical display surface as a rotation axis, and an interpupillary distance of the user.(7) The display control device according to (6), wherein the display control unit controls the stereoscopic display to display the guide object at the same depth position as a depth position of a reference plane, the reference plane being an upper surface of the jig, the receiving unit receives the confirmation information indicating whether or not the depth position of at least the reference plane and the depth position of the guide object appear to match, and the correcting unit corrects the interpupillary distance based on the second display parameter when the confirmation information indicates that the depth position of the reference plane and the depth position of the guide object appear to match. (8) The display control device according to (7), wherein the guide object has a square shape parallel to the reference plane. (9) The display control device according to (6), (7), or (8), wherein the display control unit controls the stereoscopic display to display the plurality of guide objects lined up parallel to an upper side of the jig as a reference side with respect to the Yaw angle, the receiving unit receives the confirmation information indicating whether at least the plurality of guide objects appeared lined up parallel to the reference side, and the correction unit corrects the Yaw angle based on the second display parameter when the confirmation information indicates that the guide objects appeared lined up parallel. (10) The display control device according to (9), wherein the guide object has a cone shape with an apex oriented to point to the reference side. (11) A calibration method including: controlling the stereoscopic display to display a guide object representing a parallax corresponding to a physical display surface of the stereoscopic display based on first display parameters related to stereoscopic vision; receiving confirmation information indicating a confirmation result of the user after displaying the guide object; and correcting the first display parameters based on second display parameters related to stereoscopic vision set in the stereoscopic display after receiving the confirmation information.

[0141] 3 Camera 5 Stereoscopic display 7 Operation unit 10 Display control device 11 Storage unit 11a Jig shape information 11b Guide object information 11c Parameter information 12 Control unit 12a Acquisition unit 12b Coordinate calculation unit 12c Display control unit 12d Reception unit 12e Parameter setting unit 12ea Initial setting unit 12eb Correction unit U User

Claims

1. A display control device comprising: a display control unit that controls a stereoscopic display to display a guide object representing a parallax corresponding to the physical display surface of the stereoscopic display based on a first display parameter related to stereoscopic vision; a receiving unit that receives confirmation information indicating a user's confirmation result after displaying the guide object; and a correction unit that corrects the first display parameter based on a second display parameter related to stereoscopic vision set on the stereoscopic display after receiving the confirmation information.

2. The display control device according to claim 1, wherein the display control unit draws an indicator indicating the position of a real object having a predetermined shape on the virtual display surface of the stereoscopic display in virtual space, and draws the guide object at a predetermined reference position that corresponds to the shape and is spaced apart from the virtual display surface.

3. The display control device according to claim 2, wherein the receiving unit receives the confirmation information regarding how the guide object appears to the user who has placed the real object on the physical display surface so that it matches the indicator.

4. The display control device according to claim 2, wherein the confirmation information includes position and orientation information of the guide object adjusted by the user as a parameter of the guide object.

5. The display control device according to claim 2, wherein the physical object is a transparent jig formed in a cube shape.

6. A display control device as described in claim 5, wherein the first display parameter and the second display parameter each include at least a Yaw angle, which is a rotation angle of the user's face around an axis parallel to the vertical direction of the physical display surface, and the interpupillary distance of the user.

7. The display control device described in claim 6, wherein the display control unit controls the stereoscopic display so as to display the guide object at the same depth position as the depth position of the reference plane, using the top surface of the jig as a reference plane, with respect to the interpupillary distance; the receiving unit receives the confirmation information indicating whether or not the depth position of at least the reference plane and the depth position of the guide object appear to match; and the correction unit corrects the interpupillary distance based on the second display parameter when the confirmation information indicates that they appear to match.

8. The display control device according to claim 7, wherein the guide object has a square shape parallel to the reference plane.

9. The display control device described in claim 6, wherein the display control unit controls the stereoscopic display to display the plurality of guide objects aligned parallel to the reference edge of the upper edge of the jig with respect to the Yaw angle; the receiving unit receives the confirmation information indicating whether at least the plurality of guide objects appeared aligned parallel to the reference edge; and the correction unit corrects the Yaw angle based on the second display parameter when the confirmation information indicates that the guide objects appeared aligned parallel.

10. The display control device according to claim 9, wherein the guide object is conical in shape with its apex pointing toward the reference edge.

11. A calibration method comprising: controlling a stereoscopic display to display a guide object representing a parallax corresponding to a physical display surface of the stereoscopic display based on a first display parameter related to stereoscopic vision; receiving confirmation information indicating a user's confirmation result after displaying the guide object; and correcting the first display parameter based on a second display parameter related to stereoscopic vision set in the stereoscopic display after receiving the confirmation information.

Images