Display control device and calibration method

Abstract

The display control device of the present application is provided with a display control section, a reception section, and a correction section. The display control section controls a stereoscopic vision display so that a guide object representing a parallax corresponding to a physical display surface of the stereoscopic vision display is displayed based on a first display parameter related to stereoscopic vision. The reception section receives confirmation information representing a confirmation result of a user after the display of the guide object. The correction section corrects the first display parameter based on a second display parameter related to stereoscopic vision set to the stereoscopic vision display after the reception of the confirmation information.

Description

Technical Field

[0001] This disclosure relates to display control devices and calibration methods. Background Technology

[0002] Previously, stereoscopic displays were known to make virtual objects appear as if they existed in real space. In stereoscopic displays, for example, stereoscopic display is achieved by inferring the positions of the user's left and right eyes through camera tracking, and by generating and displaying different images (hereinafter appropriately referred to as "parallax images") facing each of the inferred positions in real time.

[0003] The inference of the positions of the left and right eyes, the generation and display of real-time parallax image sets, and other display parameters related to stereoscopic vision, such as the angle and position of the user's face relative to the display surface and the interpupil distance (IPD), are performed.

[0004] In stereoscopic displays, if the display parameters are not set correctly, an offset will occur between the virtual space where virtual objects are displayed and the real space perceived by the user. Specifically, this may manifest as display distortion in the virtual space, or a display offset between real and virtual objects that are intended to be displayed simultaneously.

[0005] In other words, for stereoscopic displays, the display parameters need to be properly set through calibration. Furthermore, Patent Document 1 discloses the following technique: calibrating the parallax used when overlaying virtual objects onto real objects in a perspective-type HMD (Head Mounted Display) based on user input.

[0006] Specifically, in this technology, a real-world object for calibration is placed in front of the user wearing the HMD. The distance between the real-world object and the user's eyes is then measured using a ranging system, and the virtual object is overlaid on the real-world object using this distance measurement result. The user can input to move the virtual object, and the system performs parallax calibration based on the result of this input.

[0007] Patent Document 1: Japanese Patent Application Publication No. 2015-142383

[0008] However, the aforementioned existing technologies are based on perspective-oriented HMDs and cannot be directly applied to stereoscopic displays. For example, in stereoscopic displays, it is not possible to place real objects on the inner side of the display compared to the user's visual perception, as is the case with HMDs.

[0009] Furthermore, stereoscopic displays generally lack the ranging system found in the aforementioned existing technologies. Therefore, implementing such a ranging system would be complex and would require its integration. Summary of the Invention

[0010] Therefore, this disclosure proposes a display control device and calibration method that can achieve simple and high-precision calibration of display parameters in a stereoscopic vision display.

[0011] To address the aforementioned issues, one aspect of the display control device disclosed herein includes a display control unit, a receiving unit, and a correction unit. The display control unit controls a stereoscopic display to display a guide object based on first display parameters related to stereoscopic vision, the guide object representing a parallax corresponding to the physical display surface of the stereoscopic display. After the guide object is displayed, the receiving unit receives confirmation information indicating a user's confirmation result. After receiving the confirmation information, the correction unit corrects the first display parameters based on second display parameters related to stereoscopic vision set on the stereoscopic display. Attached Figure Description

[0012] Figure 1 This is a schematic diagram illustrating the display control device according to an embodiment of the present disclosure.

[0013] Figure 2 This is a schematic diagram (1) illustrating the calibration method according to an embodiment of the present disclosure.

[0014] Figure 3 This is a schematic diagram (Figure 2) illustrating the calibration method according to an embodiment of the present disclosure.

[0015] Figure 4 This is a schematic diagram (3) illustrating the calibration method according to an embodiment of the present disclosure.

[0016] Figure 5 This is a schematic diagram (4) illustrating the calibration method according to an embodiment of the present disclosure.

[0017] Figure 6 This is a block diagram illustrating a structural example of a display control device according to an embodiment of the present disclosure.

[0018] Figure 7 It means Figure 6 The diagram shows a block diagram of an example of the structure of the parameter setting section.

[0019] Figure 8 This is an explanatory diagram of the basic calibration method performed by the display control device.

[0020] Figure 9 This is a flowchart illustrating the basic calibration process performed by the display control device.

[0021] Figure 10 This is an explanatory diagram of the definitions related to the calibration of display parameters.

[0022] Figure 11 This is a diagram showing the indicators in the IPD calibration process and examples of the guided objects.

[0023] Figure 12 This is an explanatory diagram of the IPD calibration method performed by the display control device.

[0024] Figure 13 This is a flowchart illustrating the processing steps of the IPD calibration process performed by the display control device.

[0025] Figure 14 This is a diagram showing the indicators in the calibration process of the yaw angle and an example of the guided object.

[0026] Figure 15 It means and Figure 14 A diagram showing an example of the configuration of the corresponding fixture.

[0027] Figure 16 This is an explanatory diagram of the calibration method for the yaw angle performed by the display control device.

[0028] Figure 17 This is a flowchart illustrating the processing steps of the calibration process for the yaw angle performed by the display control device.

[0029] Figure 18 This is an explanatory diagram of the calibration method for a modified example.

[0030] Figure 19 This is a hardware structure diagram illustrating an example of a computer that implements the functions of a display control device. Detailed Implementation

[0031] Hereinafter, embodiments of the present disclosure will be described in detail based on the accompanying drawings. Furthermore, in the following embodiments, the same reference numerals will be used to label the same parts to omit repeated descriptions.

[0032] Furthermore, in the following, the display control device of the embodiment of this disclosure (hereinafter appropriately referred to as "this embodiment") is a control... Figure 1 The display control device 10 for the stereoscopic display 5 shown below. Furthermore, in the following description, the calibration method of this embodiment is a calibration method for display parameters related to the stereoscopic vision of the stereoscopic display 5 performed by the display control device 10. Additionally, in the following description, the display parameters related to the stereoscopic vision of the stereoscopic display 5 will sometimes be simply referred to as "parameters".

[0033] In addition, this disclosure is described in the following order of items.

[0034] 1. summary

[0035] 2. Example of the structure of the display control device

[0036] 3. Processing Steps

[0037] 3-1. Basic calibration procedures

[0038] 3-2. IPD calibration process

[0039] 3-3. Calibration process for the Yaw angle

[0040] 4. Variations

[0041] 4-1. Regarding the situation where there is a distance between the monitor surface and the fixture.

[0042] 4-2. Methods for using display parameters

[0043] 4-3. About the GUI used for calibration

[0044] 4-4. Other

[0045] 5. Hardware Structure

[0046] 6. Summary

[0047] "1. summary"

[0048] Figure 1 This is a schematic diagram illustrating the display control device 10 according to an embodiment of this disclosure. Additionally, Figure 2 This is a schematic diagram (Figure 1) illustrating the calibration method according to an embodiment of this disclosure. Additionally, Figure 3 This is a schematic diagram (Figure 2) illustrating the calibration method according to an embodiment of this disclosure. Additionally, Figure 4 This is a schematic diagram (Figure 3) illustrating the calibration method according to an embodiment of this disclosure. Additionally, Figure 5 This is a schematic diagram (4) illustrating the calibration method according to an embodiment of the present disclosure.

[0049] The display control device 10 in this embodiment is a device for controlling the display of the stereoscopic vision display 5. For example... Figure 1 As shown, the display control device 10 of this embodiment includes a camera 3 and a stereoscopic display 5.

[0050] Camera 3 is configured to capture the face of user U from a specified viewing angle FV. The position of camera 3 is relatively fixed relative to the position of stereoscopic display 5. Based on the control of display control device 10, stereoscopic display 5 displays the virtual object as a stereoscopic object on display surface 5a, so that user U can visually recognize it as if it exists in real space. Figure 1 This schematically illustrates a case where a cat is depicted as a virtual object on the display surface 5a for stereoscopic display.

[0051] In addition, Figure 1 The illustration shows an example where the display control device 10, camera 3, and stereoscopic display 5 are integrated into a single frame; however, the structure of this embodiment is not limited to the example described above. For instance, they could be arranged as separate units in mutually isolated locations and communicate with each other via wired or wireless means.

[0052] In addition, the display surface 5a is configured to give directionality to the light emitted from the pixels arranged on the display panel.

[0053] The display control device 10 infers the positions of the user U's left and right eyes in real time by using the tracking of the camera 3. Furthermore, the display control device 10 generates a set of disparity images facing each of the inferred positions, and displays the generated disparity image sets separately relative to the left and right eyes according to the aforementioned directionality. The inference of the positions of the left and right eyes, the generation of the disparity image sets, and the display are performed using the aforementioned display parameters.

[0054] Furthermore, in the stereoscopic display 5, if the display parameters are not set correctly, an offset occurs between the virtual space where virtual objects are displayed and the real space perceived by the user. Specifically, in this case, for example, there may be display distortion in the virtual space, or a display offset may occur between real objects and virtual objects that are intended to be displayed simultaneously.

[0055] In other words, the stereoscopic display 5 needs to be calibrated to properly set its display parameters. However, using existing calibration methods, it is difficult to achieve simple and high-precision calibration of the display parameters in the stereoscopic display 5.

[0056] Therefore, in the calibration method of this embodiment, the virtual object displayed on the display surface 5a is correctly displayed regardless of the display parameters. Based on this, in this embodiment, by displaying an indicator on the display surface 5a and setting a real object with guaranteed accuracy according to the indicator, a simple and highly accurate calibration method is achieved.

[0057] To elaborate further, in the stereoscopic display 5, for proper display, it is necessary to correspond to the virtual space from display surface 5a-V (refer to...). Figure 5 The distance of the displayed virtual object relative to the user U is used to generate an appropriate set of parallax images based on the position or angle of the user U's face and IPD. If the generated set of parallax images is inappropriate, the virtual object cannot be displayed correctly according to the coordinate system of the virtual space.

[0058] like Figure 2 As shown in the left figure, consider the case where a virtual object is to be displayed stereoscopically at a position O1 separated from the display surface 5a. In this case, a ray is emitted from the optical center of one of the user U's eyes to the virtual object to calculate the intersection point with the display surface 5a, and the color of the intersection point to be displayed on the display surface 5a is calculated.

[0059] This process is performed on all points within the virtual space, calculating the image to be displayed on display surface 5a for one eye. The same steps are performed for the other eye, calculating the image to be displayed on display surface 5a for the other eye.

[0060] The multiple images generated in this way are parallax image groups, which are presented to the left and right eyes respectively, enabling the user to visually recognize virtual objects in a stereoscopic manner.

[0061] However, as Figure 2 As shown in the central image, if the display parameters are inappropriate, for example, if the actual positions of the left and right eyes are different from the positions when the parallax image group is generated, then when the parallax image group is generated through the above steps, the user U may visually recognize a virtual object that should be visually recognized at position O1 at position P1.

[0062] In contrast, such as Figure 2 As shown in the right figure, when the distance between the display surface 5a and the virtual object is 0, the intersection points on the display surface 5a corresponding to the user U's two eyes are the same. That is to say, in this case, the intersection points on the display surface 5a are uniquely determined regardless of the position of the eyes, so the virtual object can be displayed appropriately regardless of the display parameters.

[0063] Therefore, if an indicator is displayed on the display surface 5a, and a real object with a shape having a corresponding precision is arranged on the display surface 5a according to the indicator, the appropriateness of the display parameters can be determined by the presence or absence of display offset in the display of the virtual object corresponding to the real object.

[0064] In the calibration of this embodiment, a GUI (Graphical User Interface) is displayed to eliminate the display offset. Based on the input of the user U via the GUI, the display parameters are corrected to eliminate the display offset. In this disclosure, the display parameters before correction are sometimes referred to as the first display parameters, and the display parameters adjusted for correction are referred to as the second display parameters.

[0065] The physical object disposed on the display surface 5a will be described in further detail. In this embodiment, a calibration fixture J1 is used as the physical object disposed on the display surface 5a. Figure 3 as well as Figure 4 As shown, fixture J1 is, for example, formed as a transparent cube with an opening on the bottom side. Furthermore, Figure 3 This is a three-dimensional schematic diagram of fixture J1 viewed from the bottom side. On the other hand, Figure 4 This is a three-dimensional schematic diagram of fixture J1 viewed from the top surface side.

[0066] In addition, such as Figure 3 as well as Figure 4 As shown, fixture J1 is provided with a reference frame F1 indicating that the upper surface is a reference plane. This reference frame F1 is used during the calibration of the IPD, which will be described later.

[0067] The reference frame F1 only needs to be able to allow the user U to visually recognize the depth of the upper surface of the fixture J1 relative to the display surface 5a, or it can simply be a line drawn on the upper surface of the fixture J1. Alternatively, the reference frame F1 can also be formed by the outline of tape or the like pasted on the upper surface of the fixture J1.

[0068] For example, one side of jig J1 is 100mm. The dimensional accuracy of jig J1 is ensured so that when jig J1 is displayed as a virtual object in virtual space, its size matches that of the virtual object. Furthermore, jig J1 may be made of acrylic, but the material is not particularly limited as long as it does not obstruct the display of the virtual object on the stereoscopic display 5. Additionally, jig J1 may also be formed as an object consisting only of a frame such as a wireframe.

[0069] Moreover, in this embodiment, such as Figure 5 As shown, the display control device 10 depicts an index M1 on the virtual display surface corresponding to the display surface 5a in the virtual space of the virtual object, i.e., the display surface 5a-V, indicating the configuration position of the fixture J1. The index M1 is displayed on the display surface 5a as a guide line consistent with the bottom surface of the fixture J1, based on the shape information of the fixture J1.

[0070] Furthermore, the display control device 10 depicts the guide object G1 together with the indicator M1 in virtual space. Moreover, the user U places the fixture J1 onto the indicator M1 displayed on the display surface 5a in a manner that ensures accurate alignment of the bottom surface, and confirms, for example, whether there is a display offset of the guide object G1, which is displayed overlapping with the fixture J1, relative to the fixture J1.

[0071] If there is a display offset, user U inputs information via the GUI (without illustrations) to eliminate the display offset of the boot object G1, and then confirms the input. Figure 5 In the example, user U provides input so that the depth of the upper surface of fixture J1, represented by the reference frame F1, matches the depth of the guide object G1. Furthermore, it is not limited to... Figure 5 The example shown could also display a cube-shaped guide object corresponding to the fixture J1 itself, accepting input from user U to eliminate the display offset of the guide object relative to the fixture J1.

[0072] Furthermore, the display control device 10 receives the confirmation result determined by the user U as confirmation information, and corrects the display parameters based on the received confirmation information.

[0073] Therefore, it is possible to perform simple and high-precision calibration of the display parameters in the stereoscopic vision display 5.

[0074] Furthermore, the confirmation information relates to the appearance of the guide object G1 as observed by the user U. The confirmation information at least indicates whether the user U correctly sees the guide object G1. Additionally, the confirmation information includes position and posture information related to the position and posture of the guide object after adjustments by the user as parameters of the guide object, shape information of the guide object, and information determining the set second display parameters, etc.

[0075] Hereinafter, a structural example of the display control device 10 that applies the calibration method of this embodiment will be described in more detail.

[0076] 2. Example of the structure of a display control device

[0077] Figure 6 This is a block diagram illustrating a structural example of the display control device 10 according to an embodiment of the present disclosure. Additionally, Figure 7 It means Figure 6 The diagram shows a block diagram of the structure of the parameter setting unit 12e. Furthermore, in... Figure 6 as well as Figure 7 In this document, only structural elements necessary to illustrate the features of the embodiments of this disclosure are shown, and descriptions of general structural elements are omitted.

[0078] In other words, Figure 6 as well as Figure 7The structural elements shown in the diagram are functional concepts and do not necessarily need to be physically constructed as illustrated. For example, the specific ways in which the modules are distributed / integrated are not limited to those shown in the diagram. They can be distributed / integrated in any unit, either functionally or physically, according to various loads and usage conditions.

[0079] In addition, when using Figure 6 or Figure 7 In the description, structural elements that have already been explained are sometimes briefly described or omitted.

[0080] like Figure 6 As shown, the display control device 10 includes a storage unit 11 and a control unit 12. Furthermore, the display control device 10 is connected to the camera 3, the stereoscopic display 5, and the operation unit 7. The display control device 10 is connected to the camera 3, the stereoscopic display 5, and the operation unit 7 via wired or wireless means, respectively.

[0081] Since camera 3 and stereoscopic display 5 have already been described, their descriptions are omitted here. Operation unit 7 is an operation component that operates the GUI displayed on stereoscopic display 5 for calibrating display parameters. Operation unit 7 can be implemented, for example, using a keyboard, mouse, or a dedicated GUI controller.

[0082] The storage unit 11 is implemented, for example, by storage devices such as RAM (Random Access Memory), ROM (Read Only Memory), flash memory, and HDD (Hard Disk Drive). Figure 6 In this example, the storage unit 11 stores fixture shape information 11a, guide object information 11b, and parameter information 11c. Additionally, the storage unit 11 stores the program involved in this embodiment (not shown in the illustrations).

[0083] The jig shape information 11a is model information representing the shape of jig J1. For example, jig shape information 11a is defined as a set of vertices of jig J1 and a mesh with those vertices as nodes. The jig shape information 11a contains the indices of the index faces of jig J1 that correspond to the aforementioned index M1.

[0084] The guide object information 11b is information related to each guide object including the aforementioned guide object G1. The guide object information 11b includes definition information related to the shape, position, etc., of the guide object G1 corresponding to the fixture shape information 11a. Furthermore, the guide object information 11b includes coordinates that serve as the reference position for displaying the guide object G1 based on the fixture shape information 11a.

[0085] Parameter information 11c contains information about the display parameters (first display parameters before correction) of the stereoscopic display 5. The display control device 10 controls the display of the stereoscopic display 5 based on the display parameters set in parameter information 11c.

[0086] The control unit 12 is equivalent to a processor. The control unit 12 can be implemented, for example, by a CPU (Central Processing Unit), MPU (Micro Processing Unit), GPU (Graphics Processing Unit), etc. The control unit 12 uses RAM as its working area to execute the program according to this embodiment (not shown in the diagram) stored in the storage unit 11. Furthermore, the control unit 12 can be implemented, for example, by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or FPGA (Field Programmable Gate Array).

[0087] The control unit 12 has an acquisition unit 12a, a coordinate calculation unit 12b, a display control unit 12c, a receiving unit 12d, and a parameter setting unit 12e, and performs or executes the information processing functions described below.

[0088] The acquisition unit 12a acquires camera images captured by the camera 3. Additionally, the acquisition unit 12a acquires fixture shape information 11a, guide object information 11b, and parameter information 11c from the storage unit 11.

[0089] The coordinate calculation unit 12b calculates the coordinates of fixture J1 in virtual space. The display control unit 12c controls the display of index M1 towards the coordinates calculated by the coordinate calculation unit 12b. Additionally, the display control unit 12c controls the display of the guide object G1 towards the reference position corresponding to index M1.

[0090] The receiving unit 12d receives confirmation information from the user U input via the operation unit 7. The parameter setting unit 12e sets the display parameters. For example... Figure 7 As shown, the parameter setting unit 12e has an initial setting unit 12ea and a correction unit 12eb.

[0091] The initial setting unit 12ea sets appropriate initial values ​​for the parameter information 11c. The correction unit 12eb corrects the display parameters of the parameter information 11c based on the confirmation information received by the receiving unit 12d.

[0092] 3. Processing Steps

[0093] <3-1. Basic Calibration Process>

[0094] Next, a specific example of the calibration process performed by the display control device 10 will be explained. First, the basic calibration process will be explained. Figure 8 This is an explanatory diagram of the basic calibration process performed by the display control device 10.

[0095] like Figure 8 As shown, the coordinate calculation unit 12b inputs fixture shape information 11a, the index surface shown by the fixture shape information 11a, the index position of the display index M1, the position and posture of the display surface 5a-V in the virtual space.

[0096] The coordinate calculation unit 12b outputs the index M1 and the index position to the display control unit 12c, and the display control unit 12c controls the display of the index M1 of the stereoscopic vision display 5 to display the index M1 at the specified index position.

[0097] Furthermore, the coordinate calculation unit 12b converts the shape information of the fixture J1 defined in the local coordinate system by the fixture shape information 11a into the shape information Pos in the coordinate system within the virtual space using the following formula (a). virtual It is then output to the parameter setting unit 12e.

[0098] [Formula 1]

[0099]

[0100] Here, pos local The coordinates of fixture J1 in the local coordinate system are R. display The transformed coordinates are obtained by transforming the coordinates in the local coordinate system to the display coordinate system defined in display surface 5a. display R can be determined based on the correspondence between the positions of each vertex, which is configured as index M1 on display surface 5a, in the local coordinate system and in the display surface 5a. virtual It is a transformation coordinate from the display coordinate system to the virtual space coordinate system, which can be calculated based on the position and orientation of the display surface 5a-V in the virtual space.

[0101] The parameter setting unit 12e is based on the shape information Pos of the fixture J1 in the virtual space input from the coordinate calculation unit 12b. virtual The shape, position, and currently set parameters (first display parameters) of the guide object G1 are output to the display control unit 12c.

[0102] The display control unit 12c controls the display of the guide object G1 on the stereoscopic vision display 5 based on the shape, position and parameters of the guide object G1 input from the parameter setting unit 12e.

[0103] User U positions fixture J1 on display surface 5a such that the displayed indicator M1 is precisely aligned with a specific face of fixture J1 (e.g., the bottom face described above). User U then checks whether the guide object G1, which appears to overlap with fixture J1, has a display offset relative to fixture J1.

[0104] When the display parameters before correction are appropriate, the guide object G1 appears to be displayed identically to the part corresponding to the fixture J1 without any offset. However, when the display parameters are inappropriate, the display appears to be offset. When the display is offset, the user U inputs information via the operation unit 7, and the parameter setting unit 12e changes the display parameters based on this user input.

[0105] The display control unit 12c controls the display of the stereoscopic display 5 by changing the display parameters. The user U changes the display parameters based on the input until the display offset of the guide object G1 appears to be eliminated. When the display offset appears to be eliminated, the user U determines the display parameters, and the parameter setting unit 12e outputs the determined display parameters as appropriate parameters.

[0106] The appropriate parameters output from the parameter setting unit 12e can be used directly as display parameters for appropriate display, or they can be used to calculate other parameters based on multiple parameters obtained under changing conditions. This point will be discussed later in the modified examples.

[0107] Figure 9 Indicates use Figure 8 This describes the basic calibration process steps. Figure 9 This is a flowchart illustrating the processing steps of the basic calibration process performed by the display control device 10.

[0108] The parameter setting unit 12e obtains the coordinates of the reference position of the display guide object G1 from the shape information of the fixture J1 after conversion to the virtual space coordinate system (step S101). Then, the display control unit 12c displays the guide object G1 at the same position as the reference position obtained by the parameter setting unit 12e (step S102).

[0109] Next, the parameter setting unit 12e acquires the parameters set by the user U to make the reference position and the guide object G1 appear to be in the same position (step S103). Then, the parameter setting unit 12e outputs the set parameters as appropriate parameters (step S104) and ends the process.

[0110] Next, regarding the use Figure 8 as well as Figure 9The basic calibration process is explained in detail, including the steps for calibrating specific display parameters.

[0111] The display of the stereoscopic vision display 5 requires the position of the user U's eyes relative to the display surface 5a. There are also methods to directly infer the position of the eyes, but in this embodiment, the position and posture of the user U's face are inferred, and the position of the eyes is inferred based on the position and posture.

[0112] Specifically, the positions of the two eyes are inferred based on the Roll angle, Yaw angle, and IPD, with the position between the user U's two eyes as the origin. In this embodiment, the stereoscopic display parameters related to stereoscopic vision of the stereoscopic display 5 are corrected based on the inferred position of the two eyes. The display parameters to be corrected include at least the Yaw angle and IPD.

[0113] Regarding the calibration of these display parameters, such as Figure 10 Defined as shown. Figure 10 This is an explanatory diagram illustrating the definitions related to the calibration of display parameters. For example... Figure 10 As shown, in this embodiment, the X-axis is defined as having the rightward direction of the display surface 5a as the positive direction and the leftward direction as the negative direction. Similarly, the Y-axis is defined as having the upward direction of the display surface 5a as the positive direction and the downward direction as the negative direction. Furthermore, the Z-axis is defined as having the direction orthogonal to the XY plane, with the direction towards the user U as the positive direction and the direction away from the user U as the negative direction.

[0114] The subsequent three-dimensional position and rotation are determined in this XYZ orthogonal coordinate system. The Roll angle is the position between the user's eyes (U), i.e., the facial position Pos. Face The rotation angle of the user U's face is determined by an axis parallel to the Z-axis. The Yaw angle is the angle of rotation through the facial position Pos. Face The rotation angle of user U's face is measured by an axis parallel to the Y-axis. IPD is the distance between user U's eyes.

[0115] These facial positions Pos Face Roll angle, Yaw angle, and IPD are first inferred in the camera coordinate system, for example, using camera 3. Then, the inferred results in the camera coordinate system are transformed to the display coordinate system using a coordinate transformation matrix. The coordinate transformation matrix is ​​typically a 4×4 matrix containing information for translation and rotation used in the coordinate transformation.

[0116] Representing rotation based on the Roll angle as R Roll The rotation based on the Yaw angle is represented as R. Yaw In this case, user U's left eye position Pos LeftEyeIt can be calculated using the following formula (b).

[0117] [Equation 2]

[0118]

[0119] Additionally, user U's right eye position Pos RightEye It can be calculated using the following formula (c).

[0120] [Formula 3]

[0121]

[0122] <3-2. IPD Calibration Processing>

[0123] The calibration process for IPD in the display parameters that are subject to correction is explained. Figure 11 This is a diagram showing an example of index M1 and guide object G1 in the IPD calibration process.

[0124] like Figure 11 As shown, in the IPD calibration process, the display control device 10 describes the index M1 indicating the configuration position of the fixture J1 on the index positions of the display surfaces 5a-V corresponding to the display surface 5a.

[0125] In addition, the display control device 10 controls the stereoscopic vision display 5 to display the index M1, and displays the guide object G11 at the same depth position as the reference surface, using the upper surface of the fixture J1 as the reference surface.

[0126] That is, the display control device 10 draws an index M1 on the virtual display surface (display surface 5a-V) of the stereoscopic display 5 in virtual space, indicating the placement position of the fixture J1 with a predetermined shape. At the same time, the display control device 10 draws a guide object G11 at a predetermined reference position that corresponds to the shape of the fixture J1 and is separated from the virtual display surface.

[0127] In the calibration process of IPD, such as Figure 11 As shown, the guide object G11 is, for example, displayed as a square with a cross drawn within the aforementioned reference frame F1 set on the upper surface of the fixture J1. Furthermore, as... Figure 11 The reference frame F1 is not displayed, only shown by a dashed line.

[0128] The initial IPD setting can also use appropriate values. If the IPD is appropriate, the guide object G11 will appear to be displayed at the same depth position as the reference frame F1 relative to the reference frame F1. On the other hand, if the IPD is inappropriate, the depth position of the guide object G11 relative to the depth position of the reference frame F1 will appear to be offset along either the positive or negative direction of the Z-axis in the figure (see "Correcting geometric distortions in stereoscopic 3Dimaging Zhongpai Gao, Alex Hwang, Guangtao Zhai, Eli Peli").

[0129] Therefore, in this case, the user U sets the IPD via the operation unit 7 so that the depth position of the guide object G11 relative to the reference frame F1 appears to be the same. At this time, the user U, for example, makes an input via the operation unit 7 to move the guide object G11 along the Z-axis direction. Moreover, the display control device 10 uses the IPD that the user U determines to appear to be the same depth position as the appropriate IPD.

[0130] Figure 12 This is an explanatory diagram showing the IPD calibration method performed by the display control device 10. Furthermore, Figure 12 With what has already been shown Figure 8 Correspondingly, therefore, here, the main focus is on... Figure 8 Explain the different points.

[0131] That is, in the calibration process of IPD, such as Figure 12 As shown, the parameter setting unit 12e outputs the shape, position, and currently set IPD of the guide object G11 to the display control unit 12c.

[0132] The display control unit 12c controls the display of the guide object G11 on the stereoscopic vision display 5 based on the shape, position, and IPD of the guide object G11 input from the parameter setting unit 12e.

[0133] User U positions fixture J1 on display surface 5a so that the displayed index M1 is precisely aligned with the bottom surface of fixture J1. Then, user U verifies whether the depth position of the guide object G11 relative to the reference frame F1 appears to be the same.

[0134] Then, user U inputs information via operation unit 7 until the guide object G11 and the reference frame F1 appear to be at the same depth position, and parameter setting unit 12e changes IPD based on the user input.

[0135] The display control unit 12c controls the display of the stereoscopic display 5 according to the changed IPD. When the guide object G11 and the reference frame F1 appear to be at the same depth position, the user U determines the IPD, and the parameter setting unit 12e outputs the determined IPD as the appropriate IPD.

[0136] Figure 13 Indicates use Figure 11 as well as Figure 12 The procedure for IPD calibration is described. Figure 13 This is a flowchart illustrating the processing steps of the IPD calibration process performed by the display control device 10.

[0137] The parameter setting unit 12e obtains the depth position of the reference plane of the display guide object G11 from the shape information of the fixture J1 after conversion to the virtual space coordinate system (step S201). Then, the display control unit 12c displays the guide object G11 at the same depth position as the reference plane obtained by the parameter setting unit 12e (step S202).

[0138] Next, the parameter setting unit 12e acquires the IPD set by the user U to make the reference plane and the guide object G11 appear to be at the same depth position (step S203). Then, the parameter setting unit 12e outputs the set IPD as the appropriate IPD (step S204) and ends the process.

[0139] <3-3. Calibration of the Yaw Angle>

[0140] Next, we will explain the calibration process for the Yaw angle in the display parameters that are being corrected. Figure 14 This is a diagram showing an example of the indicator M1 and the guide object G12 in the calibration process of the yaw angle. Additionally, Figure 15 It means and Figure 14 A diagram showing the configuration example of the corresponding fixture J1.

[0141] like Figure 14 As shown, in the calibration process of the Yaw angle, the display control device 10 depicts the index M1 representing the configuration position of the fixture J1 for the index position of the virtual display surface 5a-V corresponding to the display surface 5a.

[0142] In addition, such as Figure 14 as well as Figure 15 As shown, in the calibration process of the Yaw angle, the index M1 becomes the guide line with the two fixtures J1-1 and J1-2 arranged in a left-right configuration relative to the display surface 5a.

[0143] In addition, during the calibration process of the yaw angle, Figure 15The two upper edges of fixture J1 shown within the closed curve R1 become the reference edges for displaying the guide object G12. For example... Figure 14 As shown, during the calibration process of the yaw angle, the display control device 10 displays the index M1 and two conical guide objects G12. The two guide objects G12 are displayed with the straight line connecting their vertices parallel to the reference edge. Furthermore, in Figure 14 In the text, a dashed line is used to indicate that the line is not actually displayed, but it can also be displayed.

[0144] If the yaw angle is appropriate, the two guide objects G12 will appear to be aligned parallel to the reference edge (or their vertices will point to the ends of the reference edge). On the other hand, if the yaw angle is inappropriate, the two guide objects G12 will appear to be aligned non-parallel to the reference edge.

[0145] Therefore, in this case, the user U sets the yaw angle via the operation unit 7 so that the arrangement of the two guide objects G12 appears parallel to the reference edge. At this time, the user U, for example, inputs via the operation unit 7 to simultaneously rotate the two guide objects G12 about a rotation axis perpendicular to the virtual space plane. Then, the display control device 10 uses the yaw angle determined by the user U as the appropriate yaw angle to make them appear parallel.

[0146] Figure 16 This is an explanatory diagram showing the calibration method for the yaw angle performed by the display control device 10. Furthermore, Figure 16 With what has already been shown Figure 8 as well as Figure 12 Correspondingly, therefore, here, the main focus is on... Figure 8 as well as Figure 12 Explain the different points.

[0147] That is, in the calibration process of the yaw angle, such as Figure 16 As shown, the parameter setting unit 12e outputs the shape, position, and currently set yaw angle of the guide object G12 to the display control unit 12c.

[0148] The display control unit 12c controls the display of the guide object G12 on the stereoscopic vision display 5 based on the shape, position, and yaw angle of the guide object G12 input from the parameter setting unit 12e.

[0149] User U positions fixture J1 on display surface 5a so that the displayed index M1 is precisely aligned with the bottom surface of fixture J1. User U then confirms that the two guide objects G12 appear to be aligned parallel to the reference edge.

[0150] Then, user U inputs information via operation unit 7 until the arrangement of the two guide objects G12 appears parallel to the reference edge, and parameter setting unit 12e changes the Yaw angle based on the user input.

[0151] The display control unit 12c controls the display of the stereoscopic display 5 according to the changed yaw angle. When the arrangement of the two guide objects G12 appears parallel to the reference edge, the user U determines the yaw angle, and the parameter setting unit 12e outputs the determined yaw angle as the appropriate yaw angle.

[0152] Figure 17 Indicates use Figures 14-16 This describes the calibration process for the yaw angle. Figure 17 This is a flowchart illustrating the processing steps of the calibration process for the yaw angle performed by the display control device 10.

[0153] The parameter setting unit 12e obtains the coordinates of the reference edge of the display guide object G12 from the shape information of the fixture J1 after conversion to the virtual space coordinate system (step S301). Then, the display control unit 12c displays the two guide objects G12 parallel to the reference edge obtained by the parameter setting unit 12e (step S302).

[0154] Next, the parameter setting unit 12e acquires the yaw angle set by the user U to make the arrangement of the reference edge and the guide object G12 appear parallel (step S303). Then, the parameter setting unit 12e outputs the set yaw angle as an appropriate yaw angle (step S304) and ends the process.

[0155] 4. Variations

[0156] In addition, several variations can be cited in the embodiments of this disclosure described above.

[0157] <4-1. Regarding the situation where there is a distance between the monitor surface and the fixture>

[0158] Sometimes, a stereoscopic display 5 has light distribution components such as cylindrical lenses and parallax barriers mounted in layers on a display surface 5a with arranged pixels, and is configured to give directionality to the light emitted from the pixels through the openings of the light distribution components. Figure 18 This is an explanatory diagram of the calibration method involved in the modified example.

[0159] In this situation, fixture J1 cannot be directly configured on display surface 5a. In this situation, as... Figure 18As shown, for example, the thickness of the light distribution component is obtained from the design value and measured value of the light distribution component and used as the "thickness of the ground plane", so that even if there is a distance between the display surface 5a and the fixture J1, it can be confirmed whether the display is appropriate.

[0160] Equation (d) below is the coordinate transformation formula for this case. Here, R gap This represents a coordinate transformation that moves the coordinate system by the distance described above. Furthermore, depending on the specific configuration, this transformation may also include rotation and scaling.

[0161] [Formula 4]

[0162]

[0163] <4-2. Methods for using display parameters>

[0164] Furthermore, the appropriate parameters output in the above embodiments are typically used as display parameters of the stereoscopic display 5 to improve the display performance of the stereoscopic display 5. Additionally, for example, since IPD varies for each user U, it is preferable to perform IPD calibration processing in this embodiment whenever the user U of the audiovisual stereoscopic display 5 changes.

[0165] In addition, other methods of utilizing appropriate parameters are also considered. For example, an appropriate parameter may be used to optimize and calibrate the processing of other display parameters.

[0166] In other words, other display parameters are obtained based on one or more appropriate parameters obtained through this embodiment, the display parameters inferred at this time, and the set of data used for inference. For example, the inference of the yaw angle of the face based on camera 3 may be offset due to individual differences, room lighting environment, etc., but since it is known that this offset changes linearly, the inferred yaw angle (Yaw) can be obtained by using the following formula (e). est The corrected yaw angle is calculated using the input yaw. cor ).

[0167] [Formula 5]

[0168]

[0169] In this case, α and β are display parameters related to the display, but they can be derived by methods such as least squares of multiple groups of appropriate yaw angles obtained in this embodiment, using the inferred yaw angle.

[0170] <4-3. About the GUI used for calibration>

[0171] Regarding the calibration process based on user U via operation unit 7, display control device 10 can also control stereoscopic display 5 so that the display can be used to directly change face position Pos. Face A GUI for yaw angle or IPD. In this case, the user (U) indirectly adjusts the position and pose of the guided object by directly adjusting the display parameters.

[0172] Additionally, the display control device 10 can also control the stereoscopic display 5 to display a GUI representing the position and pose of the guided object as parameters of the guided object. In this case, the user U indirectly adjusts the facial position Pos by adjusting the position and pose of the displayed guided object. Face The display control device 10 acquires the position and posture information of the guided object as confirmation information, and calculates and corrects the display parameters used based on the confirmation information. Furthermore, it should be noted that even if the user U adjusts the position and posture of the displayed guided object, the position and posture information of the guided object, as an internal parameter, remains unchanged.

[0173] <4-4. Others>

[0174] Furthermore, all or part of the processes described in the embodiments of this disclosure as automated processes can also be performed manually, or all or part of the processes described as manually performed can also be performed automatically using known methods. In addition, the processing steps, specific names, and information containing various data or parameters shown in the above documents and figures can be arbitrarily changed, except in cases of special recording. For example, the various information shown in the figures is not limited to the information illustrated.

[0175] Furthermore, the structural elements of each device shown in the diagram are functional concepts and do not necessarily need to be physically constructed as illustrated. That is, the specific ways in which each device is distributed / integrated are not limited to those shown in the diagram. Depending on various loads, usage conditions, etc., all or part of them can be distributed / integrated functionally or physically in any unit.

[0176] Furthermore, the embodiments of this disclosure described above can be appropriately combined in areas where the processing content is not contradictory. Additionally, the order of the steps shown in the timing diagrams or flowcharts of this embodiment can be appropriately changed.

[0177] 5. Hardware Structure

[0178] Furthermore, the display control device 10 of the embodiments of the present disclosure described above is, for example, composed of... Figure 19 The computer 1000 is implemented with the structure shown. Figure 19This is a hardware structure diagram illustrating an example of a computer 1000 that implements the functions of the display control device 10. The computer 1000 includes a CPU 1100, RAM 1200, ROM 1300, secondary storage device 1400, communication interface 1500, and input / output interface 1600. The various parts of the computer 1000 are connected by a bus 1050.

[0179] The CPU 1100 operates based on the program stored in the ROM 1300 or the secondary storage device 1400, and controls various components. For example, the CPU 1100 expands the program stored in the ROM 1300 or the secondary storage device 1400 in the RAM 1200 and executes the processing corresponding to each program.

[0180] ROM1300 stores startup programs such as the BIOS (Basic Input Output System) executed by CPU1100 when the computer 1000 starts up, as well as programs that depend on the hardware of the computer 1000.

[0181] The secondary storage device 1400 is a computer-readable recording medium that non-temporarily records the program executed by the CPU 1100 and the data used by the program. Specifically, the secondary storage device 1400 is a recording medium that records the program of this embodiment.

[0182] Communication interface 1500 is an interface used for connecting computer 1000 to external network 1550. For example, CPU 1100 receives data from other devices or sends data generated by CPU 1100 to other devices via communication interface 1500.

[0183] Input / output interface 1600 is used to connect input / output device 1650 to computer 1000. For example, CPU 1100 receives data from input devices such as keyboard and mouse via input / output interface 1600. Additionally, CPU 1100 sends data to output devices such as monitor, speakers, and printer via input / output interface 1600. Furthermore, input / output interface 1600 can also function as a media interface for reading programs recorded on a specified recording medium. Such media include optical recording media such as DVD (Digital Versatile Disc) and PD (Phase Change Rewritable Disk), magnetic recording media such as MO (Magneto-Optical Disk), magnetic magnetic recording media, or semiconductor memory.

[0184] For example, when the computer 1000 functions as a display control device 10, the CPU 1100 of the computer 1000 executes the program loaded into the RAM 1200, thereby realizing the function of the control unit 12. Additionally, the program disclosed herein and the data in the storage unit 11 are stored in the secondary storage device 1400. Furthermore, the CPU 1100 reads program data 1450 from the secondary storage device 1400 and executes it; however, as another example, these programs may also be obtained from other devices via an external network 1550.

[0185] 6. Summary

[0186] As described above, according to one embodiment of this disclosure, the display control device 10 includes a display control unit 12c, a receiving unit 12d, and a correction unit 12eb. The display control unit 12c controls the stereoscopic display 5 to display guide objects G1, G11, and G12 representing the parallax corresponding to the display surface 5a (an example of a "physical display surface") of the stereoscopic display 5, based on first display parameters related to stereoscopic vision. After the display of the guide objects G1, G11, and G12, the receiving unit 12d receives confirmation information representing the confirmation result of the user U. After receiving the confirmation information, the correction unit 12eb corrects the first display parameters based on second display parameters related to stereoscopic vision set in the stereoscopic display 5. This enables simple and high-precision calibration of the display parameters in the stereoscopic display 5.

[0187] The various embodiments of this disclosure have been described above, but the technical scope of this disclosure is not directly limited to the above-described embodiments, and various modifications can be made without departing from the spirit of this disclosure. Furthermore, structural elements of different embodiments and variations can be appropriately combined.

[0188] Furthermore, the effects described in the various embodiments in this specification are merely illustrative and not limited, and other effects may also occur.

[0189] In addition, this technology can also take the following structure.

[0190] (1) A display control device, comprising:

[0191] The display control unit controls the stereoscopic vision display to display a guide object based on a first display parameter related to stereoscopic vision, wherein the guide object represents the parallax corresponding to the physical display surface of the stereoscopic vision display.

[0192] The receiving unit receives confirmation information indicating the user's confirmation result after the aforementioned guidance object is displayed; and

[0193] After receiving the confirmation information, the correction unit corrects the first display parameter based on the second display parameter related to stereoscopic vision set on the stereoscopic vision display.

[0194] (2) The display control device described in (1) above, wherein,

[0195] The aforementioned display control unit depicts an index representing the placement position of a real object with a predetermined shape on the virtual display surface of the aforementioned stereoscopic display in virtual space, and depicts the aforementioned guide object at a predetermined reference position that corresponds to the aforementioned shape and is separated from the aforementioned virtual display surface.

[0196] (3) The display control device described in (2) above, wherein,

[0197] The receiving unit receives confirmation information related to the appearance of the guiding object observed by the user, and the user places the real object on the physical display surface to match the indicators.

[0198] (4) The display control device described in (2) or (3) above, wherein,

[0199] The aforementioned confirmation information includes the position and posture information of the aforementioned guide object, adjusted by the aforementioned user, as a parameter of the aforementioned guide object.

[0200] (5) The display control device described in (2), (3) or (4) above, wherein,

[0201] The aforementioned real-world object is a transparent jig shaped like a cube.

[0202] (6) The display control device described in (5) above, wherein,

[0203] The first display parameter and the second display parameter mentioned above each include at least the yaw angle and the interpupillary distance of the user. The yaw angle is the rotation angle of the user's face with an axis parallel to the vertical direction of the physical display surface as the rotation axis.

[0204] (7) The display control device described in (6) above, wherein,

[0205] Regarding the interpupillary distance, the aforementioned display control unit uses the upper surface of the fixture as a reference plane and controls the stereoscopic display to display the guide object at a depth position equal to the depth position of the reference plane.

[0206] The receiving unit receives confirmation information indicating at least whether the depth position of the reference plane and the depth position of the guide object appear to be consistent.

[0207] If the above confirmation information indicates that the above appears to be consistent, the above correction unit corrects the interpupillary distance based on the above second display parameter.

[0208] (8) The display control device described in (7) above, wherein,

[0209] The aforementioned guiding object is a square shape parallel to the aforementioned reference plane.

[0210] (9) The display control device according to (6), (7) or (8) above, wherein,

[0211] Regarding the aforementioned yaw angle, the display control unit uses the upper edge of the aforementioned fixture as a reference edge and controls the aforementioned stereoscopic vision display to display a plurality of the aforementioned guide objects arranged parallel to the reference edge.

[0212] The receiving unit receives confirmation information indicating at least whether the plurality of the aforementioned guiding objects appear to be arranged parallel to the aforementioned reference edge.

[0213] When the aforementioned confirmation information appears to be arranged in parallel, the aforementioned correction unit corrects the aforementioned yaw angle based on the aforementioned second display parameter.

[0214] (10) The display control device described in (9) above, wherein,

[0215] The aforementioned guiding object is a cone-shaped object whose vertex points towards the aforementioned reference edge.

[0216] (11) A calibration method, comprising:

[0217] Control the stereoscopic vision display such that a guide object is displayed based on a first display parameter related to stereoscopic vision, the guide object representing the parallax corresponding to the physical display surface of the stereoscopic vision display;

[0218] After the aforementioned guidance object is displayed, a confirmation message indicating the user's confirmation result is received; and

[0219] After receiving the above confirmation information, the first display parameter is corrected based on the second display parameter related to stereoscopic vision set on the stereoscopic vision display.

[0220] Explanation of reference numerals in the attached drawings: 3…camera; 5…stereoscopic display; 7…operation unit; 10…display control device; 11…storage unit; 11a…jig shape information; 11b…guided object information; 11c…parameter information; 12…control unit; 12a…acquisition unit; 12b…coordinate calculation unit; 12c…display control unit; 12d…receiving unit; 12e…parameter setting unit; 12ea…initial setting unit; 12eb…correction unit; U…user.

Claims

1. A display control device, wherein, have: The display control unit controls the stereoscopic vision display to display a guide object based on a first display parameter related to stereoscopic vision, the guide object representing the parallax corresponding to the physical display surface of the stereoscopic vision display; The receiving unit receives confirmation information indicating the user's confirmation result after the guidance object is displayed; as well as The correction unit, upon receiving the confirmation information, corrects the first display parameter based on the second display parameter related to stereoscopic vision set on the stereoscopic vision display.

2. The display control device according to claim 1, wherein, The display control unit depicts an index representing the placement position of a real object with a prescribed shape on the virtual display surface of the stereoscopic display in the virtual space, and depicts the guide object at a prescribed reference position corresponding to the shape and separated from the virtual display surface.

3. The display control device according to claim 2, wherein, The receiving unit receives confirmation information related to the appearance of the guided object as observed by the user, who then positions the real object on the physical display surface to match the indicator.

4. The display control device according to claim 2, wherein, The confirmation information includes the position and posture information of the guide object adjusted by the user, which serves as a parameter of the guide object.

5. The display control device according to claim 2, wherein, The real object is a transparent jig shaped like a cube.

6. The display control device according to claim 5, wherein, The first display parameter and the second display parameter each include at least the yaw angle and the interpupillary distance of the user. The yaw angle is the rotation angle of the user's face with an axis parallel to the vertical direction of the physical display surface as the rotation axis.

7. The display control device according to claim 6, wherein, The display control unit uses the upper surface of the fixture as a reference plane with respect to the interpupillary distance, and controls the stereoscopic vision display to display the guide object at a depth position equal to the depth position of the reference plane. The receiving unit receives confirmation information indicating at least whether the depth position of the reference plane and the depth position of the guiding object appear to be consistent. If the confirmation information indicates that the two pupils appear to be consistent, the correction unit corrects the interpupillary distance based on the second display parameter.

8. The display control device according to claim 7, wherein, The guiding object is a square shape parallel to the reference plane.

9. The display control device according to claim 6, wherein, The display control unit, with respect to the yaw angle, uses the upper edge of the fixture as a reference edge and controls the stereoscopic display to display a plurality of guide objects arranged parallel to the reference edge. The receiving unit receives confirmation information indicating at least whether the plurality of the guiding objects appear to be arranged parallel to the reference edge. When the confirmation information appears to be arranged in parallel, the correction unit corrects the yaw angle based on the second display parameter.

10. The display control device according to claim 9, wherein, The guiding object is a cone-shaped object whose vertex points toward the reference edge.

11. A calibration method, wherein, include: Control the stereoscopic vision display such that a guide object is displayed based on a first display parameter related to stereoscopic vision, the guide object representing the parallax corresponding to the physical display surface of the stereoscopic vision display; After the guidance object is displayed, a confirmation message indicating the user's confirmation result is received; as well as After receiving the confirmation information, the first display parameter is corrected based on the second display parameter related to stereo vision set on the stereo vision display.

Images