Information processing system, information processing device, information processing method, and program

Abstract

Calibrating multiple terminal devices capable of displaying virtual objects. [Solution] The information processing system comprises a plurality of terminal devices capable of displaying a composite reality image in which virtual objects are superimposed on real space or an image of real space, and includes: an identification unit that identifies the positions of one or more first adjustment virtual objects in a first terminal device as reference position coordinates; a display processing unit that causes one or more second adjustment virtual objects to be displayed in a second terminal device; a difference calculation unit that calculates the difference between the position coordinates of the one or more second adjustment virtual objects and the reference position coordinates based on the fact that the one or more second adjustment virtual objects are positioned in the second terminal device at positions corresponding to the positions of the one or more first adjustment virtual objects; and a position correction unit that corrects the display position of the virtual objects to be displayed in the second terminal device based on the difference.

Description

Technical Field

[0001] The present invention relates to an information processing system, an information processing apparatus, an information processing method, and a program.

Background Art

[0002] With the Mixed Reality technology that superimposes the real space or an image of the real space and a virtual object, it has become possible to display a video in which the images of the real world and the virtual object are seamlessly fused. These technologies are being put into practical use in the entertainment field represented by, for example, game devices, and industrial fields such as the design, development, and construction of new structures.

[0003] For example, Patent Document 1 discloses a virtual object operation device that operates a virtual object in synchronization with an HMD that displays an MR video in which a virtual object is superimposed on the real world or a video of the real world. The virtual object operation device includes an MR video generation unit that generates an MR video by superimposing a video in a specified line-of-sight direction of a sky sphere video input from a sky sphere camera that photographs the real world and the virtual object, an operation instruction input unit that inputs an operation instruction for the virtual object on the MR video generated by the MR video generation unit, an operation unit that performs an operation corresponding to the operation instruction on the virtual object, a control information transmission unit that transmits the position information of the virtual object operated by the operation unit to the HMD as control information, and a control information reception unit that receives the position information of the virtual object operated by the HMD as control information.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] In this context, for multiple head-mounted displays (HMDs) to share a common virtual object in the same space, it is necessary to synchronize (calibrate) the digital coordinate systems of each HMD.

[0006] The present invention has been made in view of these circumstances, and its object is to provide an information processing system, information processing device, information processing method, and program that can calibrate a plurality of terminal devices capable of displaying virtual objects. [Means for solving the problem]

[0007] To solve the above-mentioned problems, one aspect of the present invention is an information processing system comprising a plurality of terminal devices capable of displaying a composite reality image in which virtual objects are superimposed on real space or an image of real space, the information processing system comprising: an identification unit that identifies the positions of one or more first adjustment virtual objects in a first terminal device as reference position coordinates; a display processing unit that causes one or more second adjustment virtual objects to be displayed in a second terminal device; a difference calculation unit that calculates the difference between the position coordinates of the one or more second adjustment virtual objects and the reference position coordinates based on the fact that the one or more second adjustment virtual objects are positioned in the second terminal device at positions corresponding to the positions of the one or more first adjustment virtual objects; and a position correction unit that corrects the display position of the virtual objects to be displayed in the second terminal device based on the difference.

[0008] Furthermore, one aspect of the present invention is an information processing device comprising: a display processing unit for displaying one or more virtual adjustment objects; a difference calculation unit for calculating the difference between the position coordinates of the one or more second virtual adjustment objects and the reference position coordinates, based on the fact that one or more second virtual adjustment objects are positioned at positions corresponding to the positions of one or more first virtual adjustment objects set as reference position coordinates by another terminal device; and a position correction unit for correcting the display position of the virtual objects based on the difference.

[0009] Furthermore, one aspect of the present invention is an information processing method in an information processing apparatus, comprising: a display processing step of displaying one or more virtual adjustment objects; a difference calculation step of calculating the difference between the position coordinates of the one or more second virtual adjustment objects and the reference position coordinates, based on the fact that one or more second virtual adjustment objects are placed at positions corresponding to the positions of one or more first virtual adjustment objects set as reference position coordinates by another terminal device; and a position correction step of correcting the display position of the virtual objects based on the difference.

[0010] Furthermore, one aspect of the present invention is a program that causes a computer of an information processing device to perform a display processing step of displaying one or more virtual adjustment objects; a difference calculation step of calculating the difference between the position coordinates of the one or more second virtual adjustment objects and the reference position coordinates, based on the fact that one or more second virtual adjustment objects are placed at positions corresponding to the positions of one or more first virtual adjustment objects set as reference position coordinates by another terminal device; and a position correction step of correcting the display position of the virtual objects based on the difference. [Effects of the Invention]

[0011] As described above, according to one aspect of the present invention, multiple terminal devices capable of displaying virtual objects can be calibrated. [Brief explanation of the drawing]

[0012] [Figure 1] This is a system configuration diagram showing an example of the configuration of the information processing system SYS according to the first embodiment of the present invention. [Figure 2] This figure shows an example of a display image before calibration according to this embodiment. [Figure 3] This is a block diagram showing an example of the configuration of the head-mounted display 100 according to this embodiment. [Figure 4] This figure shows an example of the arrangement of virtual objects for anchor objects according to this embodiment. [Figure 5] This figure shows an example of a display image after calibration according to this embodiment. [Figure 6] This flowchart shows an example of information processing according to this embodiment. [Figure 7] This flowchart shows an example of information processing according to this embodiment. [Modes for carrying out the invention]

[0013] (First embodiment) Hereinafter, a first embodiment of the present invention will be described with reference to the drawings. Figure 1 is a system configuration diagram showing an example of the configuration of the information processing system SYS according to the first embodiment of the present invention. The information processing system SYS comprises multiple head-mounted displays, for example, head-mounted display 100-1 and bed-mounted display 100-2. Head-mounted display 100-1 and bed-mounted display 100-2 are connected in a communicative manner.

[0014] The head-mounted display 100-1 and the bed-mounted display 100-2 may be connected to an external network such as the Internet in a manner that enables communication.

[0015] Head-mounted displays 100-1 and 100-2 are systems that present mixed reality (MR) images, which are created by overlaying virtual objects onto real space or images of real space, to user H wearing head-mounted display 100-1 or user wearing head-mounted display 100-2. Here, when head-mounted displays 100-1 and 100-2 are not distinguished, they will be referred to as head-mounted display 100. Furthermore, in the following description, head-mounted display 100 may also be referred to as an information processing device, terminal device, etc.

[0016] The head-mounted display 100 is worn on the head of a user H or the like, and displays an MR video in which virtual objects are superimposed and drawn on the real world according to the position and orientation of the head of the user H or the like. The head-mounted display 100 stores position information of virtual objects such as position information when the user H or the like moves a virtual object in the MR video. The position information of the virtual object is position coordinates (three variables), angles (two variables), scale (one variable), etc. in a three-dimensional virtual space.

[0017] Note that the head-mounted display 100 has a hand-tracking function for the user H to move a virtual object in the MR video.

[0018] The head-mounted display 100 may be an optical see-through type device that directly views the real world through a translucent display, or may be a video see-through type device that captures the real world and displays the video on the display.

[0019] In the real space, there is a real object (here, a desk OR). The virtual object is, for example, an object object OV. The object object OV is, for example, a three-dimensional structure generated by 3D (three-dimensional) CAD or the like, and is a virtual object to be observed and manipulated.

[0020] For example, in the head-mounted display 100, since the video of the real space and the video of the virtual object are mixed, the object object OV is presented semi-transparently, and the real object, the desk OR, can be seen through.

[0021] The user H wearing the head-mounted display 100 can perform various operations such as moving, rotating, and changing the size of the object object OV.

[0022] FIG. 2 is a diagram showing an example of a display image before calibration according to the present embodiment. For example, Figure 2(A) shows an example of a display image on head-mounted display 100-1, and Figure 2(B) shows an example of a display image on head-mounted display 100-2. In the illustrated example, because the head-mounted display 100 has not been calibrated, even when attempting to display an object OV at the same position, a shift in the coordinate system causes a discrepancy in the display position of the object OV. This embodiment describes a case where calibration is performed when using multiple head-mounted displays to reduce the resulting discrepancy.

[0023] Figure 3 is a block diagram showing an example of the configuration of the head-mounted display 100 according to this embodiment. The head-mounted display 100 is comprised of a communication unit 110, a storage unit 120, an input unit 130, an output unit 140, and a control unit 150.

[0024] The communication unit 110 has the function of communicating with other terminal devices, such as other head-mounted displays, via a network.

[0025] The storage unit 120 is composed of a storage medium, such as an HDD (Hard Disk Drive), flash memory, EEPROM (Electrically Erasable Programmable Read Only Memory), RAM (Random Access read / write Memory), ROM (Read Only Memory), or any combination of these storage media. This storage unit 120 can, for example, use non-volatile memory.

[0026] The memory unit 120 stores various types of data. For example, the memory unit 120 stores structural information 121, virtual object position information 122, anchor object information 123, and anchor object position information 124.

[0027] Structural information 121 is information that identifies the structure of a virtual object, such as information about a three-dimensional structure created with 3D CAD. Furthermore, if the virtual object is in motion, the structural information also includes animation information that shows that motion.

[0028] The virtual object position information 122 includes information such as the position coordinates (3 variables), angle (2 variables), and scale (1 variable) of a virtual object in a three-dimensional virtual space. The virtual object position information 122 also includes correction value information.

[0029] The anchor object information 123 is information that identifies the structure of a virtual object (anchor object) used for position adjustment (calibration) of a virtual object, and is, for example, information about a three-dimensional structure created with 3D CAD.

[0030] The anchor object position information 124 contains information such as the position coordinates (3 variables), angle (2 variables), and scale (1 variable) of the anchor object in a 3D virtual space. The anchor object position information also includes the reference position coordinates.

[0031] The input unit 130 is an input device such as a camera that is part of the head-mounted display 100. The input unit 130 receives operation input from an external source. The input unit 130 outputs an operation signal corresponding to the operation input to the control unit 150.

[0032] The output unit 140 is an output device such as a display device of the head-mounted display 100. The output unit 140 outputs information output from the control unit 150 to its own output device.

[0033] The control unit 150 has the function of controlling each part of the head-mounted display 100. The control unit 150 is comprised of a processing unit 151, a virtual object control unit 152, and a coordinate transformation unit 153.

[0034] The processing unit 151 overlays virtual objects onto the real space and displays them on the output unit 140, presenting the MR image to the user wearing the head-mounted display 100. The processing unit 151 has a hand tracking function. Here, the processing unit 151 detects various actions such as the user wearing the head-mounted display 100 grasping, releasing, or moving a virtual object using the hand tracking function, and outputs the details of these actions as operation instructions to the virtual object control unit 152. The processing unit 151 displays the virtual object at the corresponding position on the output unit 140 based on the virtual object position information stored in the memory unit 120.

[0035] If the virtual object position information stored in the storage unit 120 includes correction value information, the processing unit 151 corrects the virtual object position information stored in the storage unit 120 using the correction value information and displays the virtual object at the corresponding position in the output unit 140.

[0036] The processing unit 151 overlays the anchor object onto the real space and displays it on the output unit 140, presenting the MR image to the user wearing the head-mounted display 100. When the processing unit 151 detects various actions such as the user wearing the head-mounted display 100 grasping, releasing, or moving the anchor object using the hand tracking function, it outputs the details of these actions as operation instructions to the virtual object control unit 152. The processing unit 151 displays the anchor object at the corresponding position on the output unit 140 based on the anchor object position information stored in the memory unit 120.

[0037] The virtual object control unit 152 performs operations on virtual objects and anchor objects based on instructions input from the processing unit 151. The virtual object control unit 152 is comprised of a selection unit 1521 and an operation unit 1522.

[0038] The selection unit 1521 selects the virtual object as the target of operation when it receives input from the processing unit 151 indicating that a virtual object has been selected (grabbed). The selection unit 1521 stores identification information for the selected virtual object in memory or the like (not shown in the diagram). The selection unit 1521 outputs the identification information of the selected virtual object to the operation unit 1522. The selection unit 1521 also removes the virtual object from the target of operation when it receives input from the processing unit 151 indicating that the virtual object has been deselected (released). When the selection unit 1521 is notified of a release, it deletes the identification information held in memory or the like (not shown in the diagram).

[0039] The selection unit 1521 selects the anchor object as the target of operation when it receives input from the processing unit 151 indicating that an anchor object has been selected (grabbed). The selection unit 1521 stores identification information for the selected anchor object in memory or the like (not shown in the diagram). The selection unit 1521 outputs the identification information for the selected anchor object to the operation unit 1522. The selection unit 1521 also removes the virtual anchor object from the target of operation when it receives input from the processing unit 151 indicating that the anchor object has been deselected (released). When the selection unit 1521 is notified of a release, it deletes the identification information held in memory or the like (not shown in the diagram).

[0040] The operation unit 1522 performs operations on the virtual object in response to operation instructions input from the processing unit 151. When the operation unit 1522 receives an operation instruction from the processing unit 151 to move the virtual object (including rotation, scaling, etc.), it updates the virtual object position information stored in the storage unit 120 and transmits the virtual object position information to another head-mounted display 100 via the communication unit 110. This allows the head-mounted display 100 to refer to the updated virtual object position information stored in the memory unit 120 and display the updated virtual object on the output unit 140.

[0041] The operation unit 1522 performs operations on the anchor object in response to operation instructions input from the processing unit 151. When the operation unit 1522 receives an operation instruction from the processing unit 151 to move the anchor object (including rotation, scaling, etc.), it updates the anchor object position information stored in the storage unit 120. This allows the head-mounted display 100 to refer to the updated anchor object position information stored in the memory unit 120 and display the updated virtual object on the output unit 140.

[0042] The coordinate transformation unit 153 performs processing based on when the head-mounted display 100 is started up or when calibration mode is activated.

[0043] The coordinate transformation unit 153 includes a reference position identification unit 1531, a difference calculation unit 1532, and a position correction unit 1533.

[0044] <Assuming the standard head-mounted display is 100> When the reference position identification unit 1531 receives a user operation from the operation unit 1522 to update the anchor object position information stored in the storage unit 120 and select it as the reference position coordinates, it identifies the anchor object position information as the reference position coordinates. The reference position identification unit 1531 then stores the identified reference position coordinates in the storage unit 120 along with the anchor object position information. Here, if the reference position coordinates are not stored as the anchor object position information, the storage unit 120 stores the initial display position coordinates of the anchor object as the anchor object position information. The reference position identification unit 1531 then transmits the identified reference position coordinates to another head-mounted display 100 via the communication unit 110.

[0045] <If it is a non-standard head-mounted display 100> When the reference position identification unit 1531 receives reference position coordinates from another head-mounted display 100, it includes these reference position coordinates in the anchor object position information and stores them in the storage unit 120.

[0046] <Assuming the standard head-mounted display is 100> The difference calculation unit 1532 and the position correction unit 1533 do not perform any processing.

[0047] <If it is a non-standard head-mounted display 100> When the operation unit 1522 updates the anchor object position information stored in the memory unit 120 and the user selects to perform calibration, the difference calculation unit 1532 calculates the difference between the anchor object position coordinates set in the head-mounted display 100 and the reference position coordinates. Specifically, the difference calculation unit 1532 calculates an approximate value of the difference in the digital coordinate system from the difference between the reference position coordinates and the coordinate values ​​of the anchor object position coordinates. The difference in the digital coordinate system is the difference in the coordinate origin of the non-reference head-mounted display relative to the reference head-mounted display, and the differences in the X, Y, and Z axes.

[0048] <If it is a non-standard head-mounted display 100> The position correction unit 1533 stores an approximate value of the difference in the digital coordinate system in the virtual object position information of the storage unit 120.

[0049] Figure 4 shows an example of the arrangement of virtual objects for anchor objects according to this embodiment. Figure 4(A) shows an example of a calibration anchor object positioned as a reference position on the reference head-mounted display 100. The anchor object for calibration is displayed on the output unit 140 of the reference head-mounted display 100 based on when the head-mounted display 100 is started up or when calibration mode is activated. In the illustrated example, multiple calibration anchor objects, such as calibration anchor object OJ1, calibration anchor object OJ2, and calibration anchor object OJ3, are displayed on the output unit 140. Calibration anchor objects are placed at any location in real space according to user operation. The location is preferably an object that exists in real space, is immovable, fixed, or a physical place.

[0050] In Figure 4(A), calibration anchor objects OJ1, OJ2, and OJ3 are positioned at the three corners of the desk OR, representing real objects in real space, as shown in the illustration. After positioning the calibration anchor objects as shown, when the user selects them as reference position coordinates, the position coordinates of each calibration anchor object are set as the reference position coordinates.

[0051] Figure 4(B) shows an example of when calibration anchor objects are displayed on a non-reference head-mounted display 100. In Figure 4(A), there is a discrepancy in the positions of the calibration anchor objects, which are placed as real objects in real space at the three corners of the desk OR. This discrepancy is due to a discrepancy in the digital coordinate systems of each head-mounted display 100. The user places the displayed calibration anchor objects OJ1a, OJ2a, and OJ3a on the non-reference head-mounted display 100 as real objects in real space at the three corners of the desk OR, similar to the calibration anchor objects placed in Figure 4(A). Then, upon receiving a user input to select the execution of calibration, the non-reference head-mounted display 100 calculates an approximate difference in the digital coordinate system from the difference between the position coordinates of the calibration anchor objects OJ1a, OJ2a, and OJ3a and the position coordinates of the reference position objects OJ1, OJ2, and OJ3, and corrects the display position of the virtual object displayed after calibration.

[0052] Figure 5 shows an example of a display image after calibration according to this embodiment. Figure 5(A) shows an example of a display image after calibration on the reference head-mounted display 100. Figure 5(B) shows an example of a display image after calibration on a non-reference head-mounted display 100. In the example shown in Figure 5, the calibration performed as shown in Figure 4 reduces the discrepancy in the display position of the virtual object (object OV) compared to Figure 2. In this way, multiple head-mounted displays can share a common virtual object in the same space.

[0053] Figure 6 is a flowchart showing an example of information processing according to this embodiment. Here, the information processing described is an example of information processing performed in the reference head-mounted display 100. Furthermore, this information processing is performed when the head-mounted display 100 is started up or when calibration mode is activated.

[0054] In step S101, the head-mounted display 100 displays one or more calibration anchor objects in their initial positions.

[0055] In step S102, the head-mounted display 100 accepts user input for one or more calibration anchor objects. When the head-mounted display 100 accepts user input to select as reference position coordinates, it determines that placement is complete and executes the process in step S103. On the other hand, if the head-mounted display 100 has not accepted user input to select as reference position coordinates, it repeatedly checks the process in step S102.

[0056] In step S103, the head-mounted display 100 stores the position coordinates of one or more calibration anchor objects as reference position coordinates in the storage unit 120. The head-mounted display 100 also transmits the reference position coordinates to other head-mounted displays 100 via the communication unit 110.

[0057] Figure 7 is a flowchart showing an example of information processing according to this embodiment. Here, the information processing described is an example of information processing performed on a non-standard head-mounted display 100. Furthermore, the information processing is performed when the head-mounted display 100 is started up or when calibration mode is activated.

[0058] In step S201, the head-mounted display 100 displays one or more anchor objects for calibration based on reference position coordinates.

[0059] In step S202, the head-mounted display 100 accepts user input for one or more calibration anchor objects. When the head-mounted display 100 accepts user input to select to perform calibration, it determines that placement is complete and executes the process in step S203. On the other hand, if the head-mounted display 100 has not accepted user input to select to perform calibration, it repeatedly checks the process in step S202.

[0060] In step S203, the head-mounted display 100 calculates the difference between the position coordinates of one or more calibration anchor objects and the reference position coordinates, and calculates an approximate value of the difference in the digital coordinate system from this difference.

[0061] In step S204, the head-mounted display 100 corrects the display position of the virtual object using an approximation of the difference in the digital coordinate system.

[0062] As described above, the information processing system SYS according to this embodiment is an information processing system comprising a plurality of terminal devices (head-mounted displays 100-1, head-mounted displays 100-2) capable of displaying augmented reality images in real space or images of real space, and includes a specification unit (reference position specification unit 1531) that specifies the position of one or more first adjustment virtual objects (one or more calibration anchor objects in a reference head-mounted display 100) in the first terminal device as reference position coordinates, and one or more second adjustment virtual objects in the second terminal device (non-reference head-mounted display The system includes a display processing unit (processing unit 151) for displaying on 100), a difference calculation unit 1532 that calculates the difference between the position coordinates of one or more second adjustment virtual objects and the reference position coordinates based on the fact that one or more second adjustment virtual objects (one or more calibration anchor objects in the non-reference head-mounted display 100) are positioned in a second terminal device (non-reference head-mounted display 100) at positions corresponding to the positions of one or more first adjustment virtual objects, and a position correction unit (position correction unit 1533, processing unit 151) that corrects the display position of the virtual objects displayed on the second terminal device based on the difference.

[0063] This method allows for the calibration of multiple terminal devices capable of displaying virtual objects. Furthermore, since it eliminates the need for dedicated cloud services or external networks, or the need to read AR markers and perform image analysis to align coordinate systems, multiple terminal devices can be easily calibrated on a local network.

[0064] The number of anchor objects used for calibration may be one or multiple. For example, if the startup position, orientation, angle, and height of each head-mounted display 100 can be roughly aligned when it is started up, then the coordinate origin can be roughly aligned, and therefore only one anchor object is needed for calibration. In this case, the computational load can be reduced.

[0065] The number of anchor objects used for calibration may be determined by the calculation method used in the calibration.

[0066] Calibration accuracy can be improved as the distance between anchor objects increases. Conversely, the user's convenience can be improved as the distance between anchor objects decreases when they are placed, because the user's movement distance for placing the anchor objects is reduced.

[0067] Furthermore, although this embodiment describes the case in which a head-mounted display 100 is used, it may also be a terminal device such as a personal computer or a server device capable of displaying augmented reality images in which virtual objects are superimposed on real space or images of real space.

[0068] Furthermore, although this embodiment describes a case where the head-mounted display 100 is equipped with a coordinate transformation unit 153, an information processing device such as a terminal device or server device connected to the head-mounted display 100 may also be equipped with a coordinate transformation unit 153.

[0069] Although embodiments of this invention have been described in detail above with reference to the drawings, the specific configuration is not limited to those described above, and various design changes can be made without departing from the spirit of this invention.

[0070] Furthermore, the program that operates on the head-mounted display 100 in one aspect of the present invention may be a program that controls one or more processors such as a CPU (Central Processing Unit) (a program that makes a computer function) in order to realize the functions shown in the above embodiments and modifications relating to one aspect of the present invention. The term "computer" here includes quantum computers. The information handled by each of these devices may be temporarily stored in RAM (Random Access Memory) during processing, and then stored in various storage devices such as flash memory or HDD (Hard Disk Drive), and may be read, modified, and written by the CPU or the like as needed.

[0071] Furthermore, some or all of the head-mounted display 100 in each of the embodiments and modifications described above may be implemented using a computer equipped with one or more processors. In that case, the program for implementing this control function may be recorded on a computer-readable recording medium, and the program recorded on this recording medium may be read by the computer system and executed.

[0072] In this context, "computer system" refers to the computer system built into the head-mounted display 100, including hardware such as the operating system and peripheral devices. Furthermore, "computer-readable recording medium" refers to portable media such as flexible disks, magneto-optical disks, ROMs, and CD-ROMs, as well as storage devices such as hard disks built into the computer system.

[0073] Furthermore, "computer-readable recording media" may include those that dynamically hold programs for a short period of time, such as communication lines used when transmitting programs via networks such as the Internet or communication lines such as telephone lines, as well as those that hold programs for a certain period of time, such as volatile memory within the computer system acting as a server or client in such cases. In addition, the above-mentioned program may be for the purpose of realizing some of the functions described above, and may also be a program that can realize the above-mentioned functions in combination with a program already recorded in the computer system.

[0074] Furthermore, in each of the embodiments and modifications described above, some or all of the head-mounted display 100 may be implemented as an LSI, which is typically an integrated circuit, or as a chipset. Also, each functional block of the head-mounted display 100 in each of the embodiments and modifications described above may be individually chipped, or some or all of them may be integrated into a chip. In addition, the method of integrated circuit implementation is not limited to LSIs, but may also be implemented using dedicated circuits and / or general-purpose processors. Furthermore, if an integrated circuit implementation technology that can replace LSIs emerges due to advances in semiconductor technology, it is also possible to use an integrated circuit based on that technology.

[0075] Although various embodiments and modifications have been described in detail above with reference to the drawings as one aspect of this invention, the specific configuration is not limited to these embodiments and modifications, and includes design changes and the like that do not depart from the gist of this invention. Furthermore, various modifications are possible within the scope of the claims for one aspect of this invention, and embodiments obtained by appropriately combining the technical means disclosed in different embodiments are also included in the technical scope of this invention. In addition, configurations in which elements described in the above embodiments and modifications that produce similar effects are substituted for each other are also included. [Explanation of Symbols]

[0076] 100 Head-Mounted Displays 100-1 Head-mounted display 100-2 Head-mounted display 110 Communications Department 120 Storage section 121 Structural information 122 Virtual object location information 123 Anchor object information 124 Anchor object position information 130 Input section 140 Output section 150 Control Unit 151 Processing Unit 152 Virtual Object Control Unit 153 Coordinate Transformation Unit 1521 Selection Section 1522 Operation section 1531 Reference position identification part 1532 Difference calculation part 1533 Position correction section SYS Information Processing System

Claims

1. An information processing system comprising multiple terminal devices capable of displaying augmented reality images in which virtual objects are superimposed on real space or images of real space, A specification unit that specifies the position of one or more first adjustment virtual objects in the first terminal device as reference position coordinates, A display processing unit that causes one or more second virtual adjustment objects to be displayed on a second terminal device, The second terminal device includes a difference calculation unit that calculates the difference between the position coordinates of the one or more second adjustment virtual objects and the reference position coordinates, based on the fact that the one or more second adjustment virtual objects are positioned at locations corresponding to the positions of the one or more first adjustment virtual objects. Based on the aforementioned difference, a position correction unit corrects the display position of the virtual object to be displayed on the second terminal device, An information processing system equipped with the following features.

2. The one or more first adjustment virtual objects are positioned at the location of the object in real space. The difference calculation unit calculates the difference based on the fact that the one or more second adjustment virtual objects are positioned at locations corresponding to the positions of the one or more first adjustment virtual objects in the object. The information processing system according to claim 1.

3. A display processing unit that displays one or more virtual objects for adjustment, A difference calculation unit calculates the difference between the position coordinates of the one or more second adjustment virtual objects and the reference position coordinates, based on the fact that one or more second adjustment virtual objects are placed at positions corresponding to the positions of one or more first adjustment virtual objects set as reference position coordinates by another terminal device. Based on the aforementioned difference, a position correction unit corrects the display position of the virtual object, An information processing device equipped with the following features.

4. An information processing method in an information processing device, A display processing step that displays one or more virtual objects for adjustment, A difference calculation step that calculates the difference between the position coordinates of the one or more second adjustment virtual objects and the reference position coordinates, based on the fact that one or more second adjustment virtual objects are placed at positions corresponding to the positions of one or more first adjustment virtual objects set as reference position coordinates by another terminal device, A position correction step to correct the display position of the virtual object based on the aforementioned difference, An information processing method having

5. In the computer of the information processing device, A display processing step that displays one or more virtual objects for adjustment, A difference calculation step that calculates the difference between the position coordinates of the one or more second adjustment virtual objects and the reference position coordinates, based on the fact that one or more second adjustment virtual objects are placed at positions corresponding to the positions of one or more first adjustment virtual objects set as reference position coordinates by another terminal device, A position correction step to correct the display position of the virtual object based on the aforementioned difference, A program to execute.

Images