Information processing method and information processing system

Abstract

An information processing system includes circuitry configured to control display of first information to guide a first user to perform a first user operation to designate a first coordinate system inside a space, recognize second information related to the designated first coordinate system, and determine a coordinate system of the information processing system to be the designated first coordinate system.

Description

INFORMATION PROCESSING METHOD AND INFORMATION PROCESSING SYSTEM

[0001] The present disclosure relates to an information processing method and an information processing system.

[0002] In order for a plurality of users to recognize a virtual object at the same position inside augmented reality (AR) or other space, it is necessary to match the coordinate systems of respective users' terminal devices. For example, PTL 1 proposes a technique for matching the coordinate systems of two terminal devices, based on position information or data of the partner terminal device as viewed through the coordinate systems of the mutual terminal devices.

[0003] Japanese Laid-open Patent Publication No. 2023-98905International Publication No. WO 2024 / 090304

[0004] The accuracy of matching coordinate systems is very important, but there is still some room for improvement.

[0005] One aspect of the present disclosure is to accurately match coordinate systems.

[0006] An information processing system according to an embodiment of the present disclosure includes circuitry configured to control display of first information to guide a first user to perform a first user operation to designate a first coordinate system inside a space, recognize second information related to the designated first coordinate system, and determine a coordinate system of the information processing system to be the designated first coordinate system.

[0007] An information processing method according to an embodiment of the present disclosure includes displaying, by a first terminal device used by a first user present inside a space, information to guide the first user to perform a first user operation of designating a first coordinate system inside the space, and displaying, by a second terminal device used by a second user present inside the space, information to guide the second user to perform a second user operation of designating a second coordinate system inside the space, the second coordinate system corresponding to the first coordinate system designated based on the first user operation.

[0008] Fig. 1 is a view of an example of a schematic configuration of an information processing system 1 according to an embodiment.Fig. 2 is a diagram of an example of schematic configurations of each terminal device 5 and the server device 6.Fig. 3 is a view of an example of unmatched coordinate systems.Fig. 4 is a view of an example of matched coordinate systems.Fig. 5 is a view of an example of a first technique.Fig. 6 is a view of an example of an information display.Fig. 7 is a view of an example of the information display.Fig. 8 is a view of an example of the information display.Fig. 9 is a view of an example of a position in a coordinate system.Fig. 10 is a flowchart of an example of a process (information processing method) to be performed by the information processing system 1.Fig. 11 is a view of an example of a second technique.Fig. 12 is a view of an example of an information display.Fig. 13 is a view of an example of the information display.Fig. 14 is a view of an example of a coordinate system after matching.Fig. 15 is a flowchart of an example of a process (information processing method) to be performed by the information processing system 1.Fig. 16 is a view of an example of a third technique.Fig. 17 is a view of an example of the third technique.Fig. 18 is a view of an example of the third technique.Fig. 19 is a view of an example of an information display.Fig. 20 is a view of an example of the information display.Fig. 21 is a view of an example of the information display.Fig. 22 is a view of an example of the information display.Fig. 23 is a flowchart of an example of a process (information processing method) to be performed by the information processing system 1.Fig. 24 is a view of an example of a fourth technique.Fig. 25 is a view of an example of an information display.Fig. 26 is a view of an example of the information display.Fig. 27 is a view of an example of the information display.Fig. 28 is a view of an example of a schematic configuration of the controller 56.Fig. 29 is a view of an example of the schematic configuration of the controller 56.Fig. 30 is a view of an example of the schematic configuration of the controller 56.Fig. 31 is a flowchart of an example of a process (information processing method) to be performed by the information processing system 1.Fig. 32 is a view of an example of remote participation.Fig. 33 is a diagram of an example of a hardware configuration of an apparatus.

[0009] Some embodiments of the present disclosure will be described below in detail with reference to the accompanying drawings. It should be noted that, in each embodiment to be described below, the identical elements are denoted by the same reference numeral, and redundant descriptions will be omitted accordingly.

[0010] The present disclosure will be described in accordance with the following order of items. 1. Embodiments 1.1 First technique 1.2 Second technique 1.3 Third technique 1.4 Fourth technique 2. Modifications 3. Example of Hardware Configuration 4. Conclusion

[0011] 1. Embodiments Fig. 1 is a view of an example of a schematic configuration of an information processing system 1 according to an embodiment. Users of the information processing system 1 are referred to as users 2. There are a plurality of users 2; Fig. 1 illustrates two users 2. The first user 2 is referred to as a user 2-1 in the drawing. The second user 2 is referred to as a user 2-2 in the drawing. When not particularly distinguished from each other, the user 2-1 and the user 2-2 are referred to simply as the users 2, for example.

[0012] The user 2-1 and the user 2-2 use the information processing system 1 inside the same space. This specific space is referred to as space 3 in the drawing. The space 3 may have an enough size for the user 2-1 and the user 2-2 to be able to conduct face to face (F-to-F) communication with each other. An example of the space 3 is a room (meeting room, laboratory, etc.). In the space 3, the user 2-1 and the user 2-2 can conduct communication by uttering or gesturing, for example, while seeing the same place or seeing the appearance of the other party inside the space 3.

[0013] The space 3 has various objects (objects) present therein. Some of such objects are referred to as objects 4 in the drawing. Examples of the objects 4 include a floor, a wall, a desk, and a door.

[0014] The information processing system 1 embodies a fused space as if a virtual object 4v (a car in this example) realized by CG were present at a specific place inside the space 3. This fused space is also referred to as the XR space, whereas the information processing system 1 can also be referred to as the XR system.

[0015] The information processing system 1 is usable for AR, virtual reality (VR), mixed reality (MR), substitutional Reality (SR), and some other reality. However, specific applications of the information processing system 1 are not particularly limited; examples thereof include computer aided design (CAD), communication between the users 2, and viewing of a VR character.

[0016] The information processing system 1 includes terminal devices 5 and a server device 6.

[0017] The terminal devices 5 are to be used by the users 2 present inside the space 3. The terminal devices 5 in this example are head mounted display (HMD) devices to be used by being worn on the head of the user 2. However, the terminal devices 5 are not limited to HMD devices. Other examples of such devices include AR glasses, smartphones, tablet terminals, and other glass-type devices.

[0018] A plurality of terminal devices 5 are present in relation to a plurality of users 2. The terminal device 5 of the user 2-1 is referred to as a terminal device 5-1 in the drawing. The terminal device 5 of the user 2-2 is referred to as a terminal device 5-2 in the drawing. When not particularly distinguished from each other, the terminal device 5-1 and the terminal device 5-2 are referred to simply as the terminal devices 5, for example.

[0019] The server device 6 communicates with each of the terminal devices 5, more specifically, each of the terminal device 5-1 and the terminal device 5-2. An example of the function of the server device 6 is a position information sharing function between the terminal device 5-1 and the terminal device 5-2. Details thereof will be described later. The server device 6 may be installed outside the space 3.

[0020] Fig. 2 is a diagram of an example of schematic configurations of each terminal device 5 and the server device 6. Each terminal device 5 includes a memory unit 50, a sensor unit 51, a display unit 52, and a processing unit 53.

[0021] The memory unit 50 stores information to be used by the terminal devices 5. This information may be interpreted as containing data. Thus, the information may be appropriately read as the data unless any contradiction arises. An example of the information stored in the memory unit 50 is an information processing program 500 as illustrated in Fig. 2. The information processing program 500 is a program (software) for use in causing computers to function as the terminal devices 5. The memory unit 50 is implemented, for example, by a semiconductor memory element, such as a random access memory (RAM) or a flash memory, or a storage device, such as a hard disk, a solid state drive (SSD), or an optical disc.

[0022] Examples of the sensor unit 51 include various sensors. Specific examples of the sensor include a camera (image sensor), a distance measuring sensor, an acceleration sensor, and a microphone (sound sensor). A description thereof will be given below in order.

[0023] The camera (image sensor) captures an image of an area containing a corresponding terminal device 5. The camera thus provides the data of the images inside the space 3. The camera thereby can help perceive the situation inside the space 3 that can contain the object 4. Depending on the application, a plurality of cameras may be used. Examples of such an application include simultaneous localization and mapping (SLAM) and video see through (VST). Alternatively, a single multi-purpose camera may be used. It should be noted that the capturing may be interpreted as implying photographing, whereas the images may be interpreted as implying videos. The capturing and images may be appropriately read, respectively, as photographing and video unless any contradiction arises.

[0024] The distance measuring sensor measures a distance from a terminal device 5 to an object 4. The distance measuring sensor, which is a LiDAR sensor, for example, can measure the shape and some other physical properties of the object 4 in addition to the distance to the object 4.

[0025] The acceleration sensor detects the acceleration of a corresponding terminal device 5. The terminal device 5 thus provides acceleration data. The acceleration sensor, which may be an inertial measurement unit (IMU) sensor, may also detect the rotation, position, and some other physical properties. The acceleration sensor can be used to recognize the position, attitude, and some other properties of the terminal device 5 inside the space 3. The acceleration sensor may be a gravity sensor. In this case, the acceleration sensor can detect and automatically perceive the gravity direction (vertical direction).

[0026] The microphone (sound sensor) detects sound in the area containing the terminal device 5. The microphone thus provides sound data containing the voice of a corresponding user 2, for example. The microphone thereby can help operate the terminal device 5 with a voice, for example.

[0027] It should be noted that the sensor unit 51 is not limited to any of the above sensors. Alternatively, the sensor unit 51 may also be any other sensor that can be mounted in the terminal device 5. For example, one of the sensors may be a device (e.g., a button) for use in operating the terminal device 5.

[0028] The display unit 52 displays information. For example, the display unit 52 displays the virtual object 4v. Each terminal device 5 illustrated in Fig. 1 is an HMD. The display unit 52, which may be a transmissive display unit (transmissive display), allows the user 2 to see the virtual object 4v displayed so as to overlap the space 3.

[0029] The processing unit 53 performs various processes necessary for the operation of the terminal device 5. The processing unit 53 is implemented by, for example, a central processing unit (CPU), a micro processing unit (MPU), a graphics processing unit (GPU), or other processing unit executing the information processing program 500 memorized in the memory unit 50 by using a RAM, for example, as a working area. Furthermore, the processing unit 53, which is a controller, may be implemented by, for example, an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a micro controller unit (MCU), or other integrated circuit.

[0030] The processing unit 53 controls the entire terminal device 5 by controlling each of the elements in the terminal device 5. For example, the processing unit 53 controls the memory unit 50, thereby writing desired information into the memory unit 50 or reading desired information from the memory unit 50. The processing unit 53 controls the sensor unit 51, thereby acquiring various data, such as image, acceleration, or sound data and subjecting the acquired data to some processes. Examples of such data processes include an image recognition process, a SLAM process, and a sound recognition process. The processing unit 53 controls the display unit 52 in such a way that various information is displayed in the display unit 52.

[0031] In the following description in which a terminal device 5 performs some operation, the subject may be appropriately read as the memory unit 50, the sensor unit 51, the display unit 52, or the processing unit 53.

[0032] The server device 6 includes a memory unit 60 and a processing unit 61. The memory unit 60 stores information to be used by the server device 6. The processing unit 61 performs various processes necessary for the operation of the server device 6.

[0033] In this example, the server device 6 functions as a position sharing server. The terminal device 5-1 transmits position information (information regarding the position and orientation) on the terminal device 5-1 to the server device 6. The position information on the terminal device 5-1 is generated in the terminal device 5-1, for example, through the above SLAM process. The terminal device 5-2 transmits position information on the terminal device 5-2 to the server device 6. The position information on the terminal device 5-2 is generated in the terminal device 5-2, for example, through the above SLAM process. These pieces of position information are stored in the memory unit 60 of the server device 6 as the position information 600. The position information 600 is transmitted from the server device 6 to each of the terminal device 5-1 and the terminal device 5-2. As a result, the position information is shared between the terminal device 5-1 and the terminal device 5-2.

[0034] Returning to Fig. 1, each of the terminal device 5-1 and the terminal device 5-2 has a coordinate system. An example of the coordinate system is a SLAM coordinate system. The coordinate system that each of the terminal device 5-1 and the terminal device 5-2 has is assumed to be the SLAM coordinate system unless otherwise specified. The coordinate system of the terminal device 5-1 is referred to as a coordinate system C1. The coordinate system of the terminal device 5-2 is referred to as a coordinate system C2. The terminal device 5-1 displays the virtual object 4v in accordance with the coordinate system C1. The terminal device 5-2 displays the virtual object 4v in accordance with the coordinate system C2.

[0035] Each of the coordinate system C1 and the coordinate system C2 has three mutually orthogonal axes, namely, three axial directions. The three axial directions include: two axial directions (horizontal biaxial directions) extending in different horizontal directions orthogonal to each other; and an axial direction (vertical uniaxial direction) extending in a vertical direction orthogonal to each axial direction.

[0036] An object of the information processing system 1 is to allow the user 2-1 and the user 2-2 to recognize the same virtual object 4v at the same place inside the space 3. To accomplish this object, it is important to match the coordinate system C1 of the terminal device 5-1 with the coordinate system C2 of the terminal device 5-2. A description thereof will be given with reference to Figs. 3 and 4.

[0037] Fig. 3 is a view of an example of unmatched coordinate systems. In the example illustrated in Fig. 3, the coordinate system C1 does not match the coordinate system C2. In this case, the position of a virtual object 4v inside the space 3 displayed by the terminal device 5-1 does not match the position of a virtual object 4v inside the space 3 displayed by the terminal device 5-2. As a result, the user 2-1 and the user 2-2 may recognize the same virtual object 4v at different places inside the space 3.

[0038] Fig. 4 is a view of an example of matched coordinate systems. The coordinate system C1 matches the coordinate system C2. This coordinate system is referred to as the coordinate system C in the drawing. In this case, the position of the virtual object 4v inside the space 3 displayed by the terminal device 5-1 matches the position of the virtual object 4v inside the space 3 displayed by the terminal device 5-2. In this case, the user 2-1 and the user 2-2 can recognize the same virtual object 4v at the same place inside the space 3.

[0039] To accurately match the coordinate system C1 with the coordinate system C2, the information processing system 1 encourages each of the user 2-1 and the user 2-2 to perform an operation of designating the coordinate system inside the space 3. The operation performed by the user 2-1 is referred to as a first user operation. The operation performed by the user 2-2 is referred to as a second user operation.

[0040] In the first user operation, the user 2-1 designates an origin and axial directions in the coordinate system inside the space 3. In the second user operation, the user 2-2 designates an origin and axial directions inside the space 3 which are the same as the origin and axial directions that have been designated by the user 2-1. Some examples of specific techniques will be described.

[0041] 1.1 First technique Fig. 5 is a view of an example of a first technique. In this case, an image-capturing range of a terminal device 5 is schematically illustrated as an image-capturing range 5v.

[0042] An origin and the axial directions in the coordinate system are designated by designating specific positions (points) inside the space 3. In this example, a point P1, a point P2, and a point P3 are designated. The point P1 corresponds to the origin. The direction from the point P1 to the point P2 and the direction from the point P1 to the point P3 correspond to the axial directions, more specifically, the horizontal biaxial directions.

[0043] In the first technique, a corner portion 4a present inside the space 3 is used. The corner portion 4a contains two mutually orthogonal sides extending in the horizontal directions. In this example, the corner portion 4a is located at one corner of a room (an example of the space 3) and is defined by a structure, such as a wall and a floor.

[0044] <First user operation> The user 2-1 designates the origin and the axial directions in the coordinate system along the corner portion 4a. More specifically, the user 2-1 designates the vertex of the corner portion 4a as the point P1, namely, the origin in the coordinate system. In addition, the user 2-1 designates positions on two sides leading to the vertex of the corner portion 4a as the point P2 and the point P3. As a result, the direction from the point P1 to the point P2 and the direction from the point P1 to the point P3 are designated as the horizontal biaxial directions in the coordinate system.

[0045] In this case, the user 2-1 may designate only one axial direction out of the horizontal biaxial directions. For example, the user 2-1 may designate only the point P2 out of the point P2 and the point P3. The reason is that, if the axial direction from the point P1, which corresponds to the vertex of the corner portion 4a, to the point P2 on one side is determined, the axial direction from the point P1 to the point P3 on the other side is uniquely determined and thus can be automatically designated. Hereinafter, it is assumed that the user 2-1 designates only two points: the point P1 and the point P2.

[0046] The vertical uniaxial direction is detected by a gravity sensor (an example of the sensor unit 51) of the terminal device 5 and is automatically designated. However, this vertical uniaxial direction may be manually designated by the user 2-1. Hereinafter, it is assumed that the vertical uniaxial direction is automatically designated.

[0047] The user 2-1 designates the point P1 and the point P2 by directly touching the positions corresponding to the points P1 and P2 (e.g., with a finger) or touching these positions with a controller. In this example, the user 2-1 uses a controller 54. The controller 54 may be a controller of the terminal device 5. For example, the controller 54 may have a tapered shape, like the shape of a pen. By using the controller 54 with this shape, the terminal device 5-1 can easily come into contact with any point inside the space 3, particularly a narrow place, such as the vertex of the corner portion 4a located at a corner of the room.

[0048] With the controller 54 or the finger touching the positions corresponding to the point P1 and the point P2, the user 2-1 performs an operation (determination operation) of designating the origin and the axial direction. Examples of the determination operation include a button operation of the controller 54 and a gesture operation. Examples of the gesture operation include moving a finger (e.g. a finger that is not the same as the finger touching the point P1) of the user 2-1 or moving his / her eyes.

[0049] The terminal device 5-1 displays information for use in encouraging the user 2-1 to perform the above first user operation. A description thereof will be given with reference to Figs. 6 and 8.

[0050] Figs. 6 to 8 are views of examples of an information display. The user 2-1 can see the information displayed by the display unit 52 of the terminal device 5-1 as if the information overlapped the space 3. It should be noted that the user 2-1 (e.g., his / her hand) holding the controller 54 is not illustrated.

[0051] As illustrated in Fig. 6, a message M1 is displayed. In accordance with the message M1 saying "designate origin", the user 2-1 designates the origin in the coordinate system. More specifically, the user 2-1 touches the vertex of the corner portion 4a by using the end of the controller 54. This point is designated as the point P1, namely, the origin in the coordinate system.

[0052] As illustrated in Fig. 7, a message M2 is displayed. In accordance with the message M2 saying "designate direction", the user 2-1 designates an axial direction in the coordinate system. More specifically, the user 2-1 touches a point on one side of the corner portion 4a by using the end of the controller 54. This point is designated as the point P2. The direction from the point P1 to the point P2 is designated as one axial direction out of the horizontal biaxial directions.

[0053] As described above, the point P3 is automatically designated. The direction from the point P1 to the point P3 is designated as the other axial direction out of the horizontal biaxial directions. The vertical uniaxial direction is also automatically designated.

[0054] As illustrated in Fig. 8, a message M3 is displayed. In accordance with the message M3 saying "share designated information with another user", the user 2-1 notifies the user 2-2 of the information on the point P1, the point P2, and others that have been designated by himself / herself. The user 2-1 and the user 2-2 can conduct F-to-F communication as described above and can easily share information accordingly.

[0055] <Second user operation> Next, the user 2-2 performs the second user operation. Referring to Fig. 5 again, the user 2-2, in turn, designates an origin and an axial direction in the coordinate system along the corner portion 4a. In this case, an origin and an axial direction to be designated by the user 2-2 are the same as the origin and the axial direction that have been designated by the user 2-1.

[0056] The display unit 52 of the terminal device 5-2 displays information for use in encouraging the user 2-2 to perform the above second user operation. Referring to Figs. 6 and 7 again, the user 2-2 designates the origin in the coordinate system in accordance with the message M1. More specifically, the user 2-2 touches the vertex of the corner portion 4a by using the end of the controller 54. This point is designated as the point P1, namely, the origin in the coordinate system. The user 2-2 then designates an axial direction in the coordinate system in accordance with the message M2. More specifically, the user 2-2 touches a point on one side of the corner portion 4a by using the end of the controller 54. This point is designated as the point P2. The direction from the point P1 to the point P2 is designated as one axial direction out of the horizontal biaxial directions. The other axial direction out of the horizontal biaxial directions and the vertical uniaxial direction are automatically designated.

[0057] <Matching of coordinate systems> Fig. 9 is a view of an example of the matched coordinate systems. Based on the result of the first user operation and the result of the second user operation described above, both the terminal device 5-1 and the terminal device 5-2 perform a process of matching the coordinate system C1 with the coordinate system C2 (coordinate matching process).

[0058] A specific example of the process is coordinate conversion. The terminal device 5-1 and the terminal device 5-2 convert the coordinate system C1 and the coordinate system C2 into the same coordinate system. For example, the terminal device 5-1 converts the coordinate system C1 into coordinates designated in the first user operation (coordinates having the origin and the axial direction designated by the user 2 -1). The terminal device 5-2 converts the coordinate system C2 into coordinates designated in the second user operation (coordinates having the origin and the axial direction designated by the user 2 -2). In this case, various known operations (e.g., matrix operations) may be used, in which case necessary parameters and some other factors may be set. After the conversion, the coordinate system C1 and the coordinate system C2 match each other to turn out to be the coordinate system C.

[0059] After the above coordinate conversion, the coordinate system C1 may be used as a new coordinate system C1 of the terminal device 5-1. It can also be said that the coordinate system C1 is reset or updated. Alternatively, it can be said that the coordinate system C1 moves to the coordinate system having the origin and the axial direction that have been designated by the user 2-1. The same applies to the coordinate system C2.

[0060] The user 2-1 and the user 2-2 present inside the same space 3 designate coordinate systems inside the space 3, thereby, based on this result, accurately matching the coordinate system C1 of the terminal device 5-1 with the coordinate system C2 of the terminal device 5-2. As a comparative example, the following technique is conceivable: the user 2-1 and the user 2-2 sequentially move to specific places inside the space 3 and then perform determination operations (e.g., by pressing buttons for a long time). In this technique, the positions and orientations of the user 2-1 and the user 2-2 at that time are designated as the origins and the axial directions in the coordinate system. The technique of this comparative example, however, may make it difficult for the user 2-1 and the user 2-2 to perform the determination operation in exactly the same position inside the space 3 while being oriented in exactly the same direction. As a result, it may also be difficult to accurately match the coordinate systems. The information processing system 1 according to an embodiment enables the user 2-1 and the user 2-2 to easily and accurately designate an origin and an axial direction of the same coordinate system. It is thereby possible to match the coordinate systems more accurately.

[0061] Since the coordinate system C1 matches the coordinate system C2, the position of a virtual object 4v inside the space 3 which is displayed by the terminal device 5-1 matches the position of a virtual object 4v inside the space 3 which is displayed by the terminal device 5-2. In this case, the user 2-1 and the user 2-2 can recognize the same virtual object 4v at the same place inside the space 3.

[0062] Fig. 10 is a flowchart of an example of a process (information processing method) to be performed by the information processing system 1. Some technical matters similar to those described above will be omitted as appropriate.

[0063] In Step S1, the terminal device 5-1 displays information for use in encouraging the user 2-1 to perform the first user operation. As in Step S1a, the user 2-1 designates an origin and an axial direction in the coordinate system along the corner portion 4a. For example, the user 2-1 touches some specific positions (positions corresponding to the point P1 and the point P2) inside the space 3 directly or with the controller 54.

[0064] In Step S2, the terminal device 5-2 displays information for use in encouraging the user 2-2 to perform the second user operation. As in Step S2a, the user 2-2 designates an origin and an axial direction along the corner portion 4a which are the same as the origin and axial direction that have been designated by the user 2-1. For example, the user 2-2 touches the same positions as the positions that have been touched by the user 2-1, directly or with the controller 54.

[0065] In Step S3, both the terminal device 5-1 and the terminal device 5-2 match the coordinates in the coordinate system C1 and the coordinate system C2, based on the result of the first user operation and the result of the second user operation. For example, the terminal device 5-1 converts the coordinate system C1 into a coordinate system having the origin and the axial direction that have been designated by the user 2-1. The terminal device 5-2 converts the coordinate system C2 into a coordinate system having the origin and the axial direction that have been designated by the user 2-2.

[0066] Based on the coordinate system C1 and the coordinate system C2 that have been matched with each other, namely, based on the coordinate system C, each of the terminal device 5-1 and the terminal device 5-2 displays the virtual object 4v. The user 2-1 and the user 2-2 can recognize the same object 4 at the same place inside the space 3.

[0067] It should be noted that, in Steps S1, S2, and S3 described above, the terminal device 5-1 and the terminal device 5-2 may exchange no data therebetween. If the terminal device 5-1 and the terminal device 5-2 exchange any data therebetween to match the coordinate system C1 with the coordinate system C2, it may be necessary to exchange important information, such as accounts during the data exchange. This might hinder the protection of privacy information. With the above method, it is possible to match the coordinate system C1 with the coordinate system C2 without exchanging data between the terminal device 5-1 and the terminal device 5-2. Thus, this method is also advantageous in protecting privacy information. In addition, the method leads to a reduction in communication loads, for example.

[0068] 1.2 Second technique Fig. 11 is a view of an example of a second technique. In the second technique, corner portions 4a present inside the space 3 are also used. In the example illustrated in Fig. 11, the corner portions 4a are corner portions of a desk (an example of the object 4). Unlike the first technique, the user 2-1 and the user 2-2 do not have to touch specific positions (positions corresponding to the point P1 and another) inside space 3.

[0069] <First user operation and second user operation> Each of the terminal device 5-1 and the terminal device 5-2 extracts the plurality of corner portions 4a present inside the space 3 and then displays these corner portions in a selectable mode. Various known image recognition processes and other similar techniques may be used. Each of the terminal device 5-1 and the terminal device 5-2 also extracts some origins and axial directions designable along the corner portions 4a (through point extraction or line extraction) and then displays these origin and axial directions in a selectable mode. For the extraction, for example, a shape recognition process based on data of a LiDAR sensor (an example of the sensor unit 51) may be used.

[0070] The user 2-1 selects one from among the corner portions 4a and then designates an origin and an axial direction along the selected corner portion 4a in the coordinate system. The user 2-2 designates an origin and an axial direction along the corner portion 4a selected by the user 2-1 which are the same as the origin and axial direction designated by the user 2-1.

[0071] The terminal device 5-1 displays information for use in encouraging the user 2-1 to perform the above first user operation. In addition, the terminal device 5-2 displays information for use in encouraging the user 2-2 to perform the above second user operation. A description thereof will be given with reference to Figs. 12 and 13.

[0072] Figs. 12 and 13 are views of examples of an information display. In this example, the display unit 52 displays the four corner portions 4a of the desk in a selectable mode (in a highlighted fashion). In addition, the display unit 52 also displays some origins and axial directions designable along the corner portions 4a in a selectable mode. Optionally, the display unit 52 may also display some corners present in places other than the desk in a selectable mode.

[0073] As illustrated in Fig. 12, a message M4 is displayed. In accordance with the message M4 saying "designate origin and direction", each of the user 2-1 and the user 2-2 designates an origin and an axial direction. This designation is performed by the user 2-1 and the user 2-2 operating the terminal device 5-1 and the terminal device 5-2. A specific operation method is not particularly limited; various operations, such as a button operation, a controller operation, a voice operation, and a gesture operation, may be performed.

[0074] More specifically, each of the user 2-1 and the user 2-2 selects a corner portion 4a from among the plurality of corner portions 4a and then designates the vertex of the selected corner portion 4a as the point P1, namely, the origin in the coordinate system. Furthermore, each of the user 2-1 and the user 2-2 designates one of the two sides of the corner portion 4a as one axial direction out of the horizontal biaxial directions. In this case, the user 2-1 and the user 2-2 conduct F-to-F communication to select the same corner portion 4a and designate the same origin and designate axial direction. The other out of the horizontal biaxial directions and the vertical axial direction are automatically designated.

[0075] As illustrated in Fig. 13, a message M5 is displayed. In response to the message M5 saying "same as another user?", the user 2-1 and the user 2-2 select OK if the origins and the axial directions that have been designated thereby are the same, thereby completing the designation. Otherwise (if they are different), the user 2-1 and the user 2-2 select Cancel to perform re-designation. If the horizontal biaxial directions are reversely designated, the user 2-1 and the user 2-2 select "reverse axial direction". In response, the re-designation is performed such that the relationship therebetween is reversed. It is thereby possible to easily correct the designation of the axial direction.

[0076] <Matching of coordinate systems> Both the terminal device 5-1 and the terminal device 5-2 match the coordinate system C1 with the coordinate system C2, based on the result of the above first user operation and the result of the above second user operation. An example of this process is the coordinate conversion described above.

[0077] Fig. 14 is a view of an example in the coordinate system after the matching. In this case, the position of the virtual object 4v inside the space 3 which is displayed by the terminal device 5-1 matches the position of the virtual object 4v inside the space 3 which is displayed by the terminal device 5-2. In this case, the user 2-1 and the user 2-2 can recognize the same virtual object 4v at the same place inside the space 3.

[0078] Fig. 15 is a flowchart of an example of a process (information processing method) to be performed by the information processing system 1. Some technical matters similar to those described above will be omitted as appropriate.

[0079] In Step S11, the terminal device 5-1 displays information for use in encouraging the user 2-1 to perform the first user operation. In addition, the terminal device 5-2 displays information for use in encouraging the user 2-2 to perform the second user operation. For example, each of the terminal device 5-1 and the terminal device 5-2 displays the plurality of corner portions 4a present inside the space 3 in a selectable mode and further displays origins and axial directions designable along the corner portions 4a in a selectable mode. As in Step S11a, the user 2-1 and the user 2-2 designate origins and axial directions along a corner portion 4a in the same coordinate system. For example, the user 2-1 selects a corner portion 4a from among the plurality of corner portions 4a and then designates an origin and an axial direction along the selected corner portion 4a in the coordinate system. The user 2-2 designates an origin and an axial direction along the same corner portion 4a as the corner portion 4a selected by the user 2-1 which are the same as the origin and the axial direction that have been designated by the user 2-1.

[0080] In Step S12, each of the terminal device 5-1 and the terminal device 5-2 matches the coordinates in the coordinate system C1 and the coordinate system C2.

[0081] Based on the coordinate system C1 and the coordinate system C2 that have been matched with each other, namely, based on the coordinate system C, each of the terminal device 5-1 and the terminal device 5-2 displays the virtual object 4v. The user 2-1 and the user 2-2 can recognize the same object 4 at the same place inside the space 3.

[0082] It should be noted that, in Steps S11 and S12 described above, the terminal device 5-1 and the terminal device 5-2 may exchange no data therebetween.

[0083] 1.3 Third technique Figs. 16 to 18 are views of examples of a third technique. In the third technique, a virtual image (CG) shaped so as to fit the space 3 is used. This image is referred to as a virtual space image 3v. In the virtual space image 3v, occlusion of the space 3 is reproduced. The occlusion may be generated, for example, due to the objects 4 present in the space 3, particularly, the objects 4 (a wall, a floor, a ceiling, etc.) that define the shape of the space 3. Portions of the virtual space image 3v which are related to these defining objects 4 are referred to as occlusion portions 4o in the drawing.

[0084] An origin is preset in the virtual space image 3v. This origin is referred to as an origin 3vp in the drawing. In this example, the vertex of a corner portion of the virtual space image 3v is set to the origin 3vp. Two directions extending from the origin 3vp along two sides of the corner portion are automatically determined to be the horizontal biaxial directions.

[0085] Each of the terminal device 5-1 and the terminal device 5-2 displays the virtual space image 3v. The terminal device 5-1 displays the virtual space image 3v in such a way that the user 2-1 can move the virtual space image 3v. The terminal device 5-2 displays the virtual space image 3v in such a way that the user 2-2 can move the virtual space image 3v. Examples of a method of moving the virtual space image 3v may include: moving the virtual space image 3v upward, downward, rightward, and / or leftward on the screen; and rotating the virtual space image 3v.

[0086] <First user operation and second user operation> As illustrated in Fig. 16, the virtual space image 3v displayed by the terminal device 5-1 does not initially match the space 3. Each of the user 2-1 and the user 2-2 matches the virtual space image 3v with the space 3. For example, each of the user 2-1 and the user 2-2 moves inside the space 3 or moves the displayed virtual space image 3v in such a way that the occlusion portions 4o in the virtual space image 3v match the corresponding objects 4 inside the space 3. As illustrated in Fig. 17, the virtual space image 3v matches with the space 3.

[0087] As illustrated in Fig. 18, with the virtual space image 3v matching the space 3, the origin 3vp of the virtual space image 3v is designated as the origin in the coordinate system. In addition, two directions extending from the origin 3vp are designated as the horizontal biaxial directions. The vertical axial direction is also automatically designated. In this way, the origin and the axial directions in the coordinate system are designated.

[0088] The terminal device 5-1 displays information for use in encouraging the user 2-1 to perform the above first user operation. In addition, the terminal device 5-2 displays information for use in encouraging the user 2-2 to perform the above second user operation. A description thereof will be given with reference to Figs. 19 and 22. In these descriptions, the user 2-1 and the user 2-2 are not distinguished from each other and simply referred to as the users 2 accordingly. The terminal device 5-1 and the terminal device 5-2 are not distinguished from each other and simply referred to as the terminal devices 5.

[0089] Figs. 19 to 22 are diagrams of examples of an information display. A user 2 can see a scene in which the information displayed by the display unit 52 of the terminal device 5, namely, the virtual space image 3v (corresponding to the hatched portion) overlaps the space 3. An occlusion portion 4o corresponding to an object 4 that generates occlusion is also displayed. In this example, the user 2 can move the virtual space image 3v by touching the virtual space image 3v with his / her virtualized hand.

[0090] As illustrated in Fig. 19, initially, the virtual space image 3v does not match the space 3, whereas the object 4 does not match the corresponding occlusion portion 4o. As illustrated in Figs. 21 to 22, the user 2 moves inside the space 3 or moves the virtual space image 3v in such a way that the occlusion portion 4o matches the object 4. As a result, the virtual space image 3v coincides with the space 3.

[0091] In the above case, to help match the occlusion portions 4o with the objects 4, for example, the matched portion of the occlusion portion 4o may be highlighted by changing the color of this portion when the occlusion portion 4o partly matches the object 4. To determine whether the occlusion portion 4o matches the object 4, for example, data of a LiDAR sensor (an example of the sensor unit 51) may be used.

[0092] Since the user 2-1 and the user 2-2 match the same virtual space image 3v with the same space 3, the user 2-1 and the user 2-2 can designate origins and axial directions of the same coordinate system. As the accuracy (of the occlusion portion 4o) of the virtual space image 3v increases, the accuracy of the coincidence also increases. Consequently, it is possible to strictly match the origins and the axial directions in the coordinate system.

[0093] <Matching of coordinate systems> Both the terminal device 5-1 and the terminal device 5-2 match the coordinate system C1 with the coordinate system C2, based on the result of the above first user operation and the result of the above second user operation. An example of this process is the coordinate conversion described above.

[0094] Fig. 23 is a flowchart of an example of a process (information processing method) to be performed by the information processing system 1. Some technical matters similar to those described above will be omitted as appropriate.

[0095] In Step S21, the terminal device 5-1 displays information for use in encouraging the user 2-1 to perform the first user operation. The terminal device 5-2 displays information for use in encouraging the user 2-2 to perform the second user operation. For example, each of the terminal device 5-1 and the terminal device 5-2 displays the virtual space image 3v in which occlusion of the space 3 is reproduced. As in Step S21a, the user 2-1 and the user 2-2 match the virtual space image 3v with the space 3, thereby designating origins and axial directions in the same coordinate system. For example, each of the user 2-1 and the user 2-2 moves inside the space 3 or moves the virtual space image 3v.

[0096] In Step S22, each of the terminal device 5-1 and the terminal device 5-2 matches the coordinates in the coordinate system C1 and the coordinate system C2.

[0097] Based on the coordinate system C1 and the coordinate system C2 that have been matched with each other, namely, based on the coordinate system C, each of the terminal device 5-1 and the terminal device 5-2 displays the virtual object 4v. The user 2-1 and the user 2-2 can recognize the same object 4 at the same place inside the space 3.

[0098] It should be noted that, in Steps S21 and S22 described above, the terminal device 5-1 and the terminal device 5-2 may exchange no data therebetween.

[0099] 1.4 Fourth technique Fig. 24 is a view of an example of a fourth technique. In the fourth technique, each of the user 2-1 and the user 2-2 places (mounts, fixes, etc.) a controller 56 on a positioning device 55 disposed inside the space 3 and designates origins and axial directions in the same coordinate system. The controller 56 and the positioning device 55 may be a controller of a terminal device 5 and a charging device (also referred to as a charging dock) therefor.

[0100] Each of the user 2-1 and the user 2-2 has a controller 56. More specifically, the controller 56 includes a pair of controllers in this example. One of these controllers is referred to as a controller 56L in the drawing. The other controller is referred to as a controller 56R in the drawing. Each of the user 2-1 and the user 2-2 holds and uses the controller 56 with his / her left hand and also holds and uses the controller 56R with his / her right hand.

[0101] <First user operation> The user 2-1 places the controller 56 to be used by user 2-1 on the positioning device 55. As an example, the user 2-1 places the controller 56R out of the controller 56R and the controller 56L.

[0102] The position and orientation of the controller 56R to be placed on the positioning device 55 with respect to the positioning device 55 are predetermined. The position and orientation are defined, for example, by magnets attached to the positioning device 55 and the controller 56R.

[0103] A predetermined position (positions of the end, center, or other arbitrary portion) of the controller 56R placed on the positioning device 55 is designated as the point P1, namely, the origin in the coordinate system. In addition, two directions along the controller 56R are designated as the horizontal biaxial directions in the coordinate system. The origin and the axial directions to be designated along the controller 56R are extracted and determined, for example, by subjecting an image containing the controller 56R to an image recognition process. The vertical axial direction is also automatically designated.

[0104] The directions along the controller 56R may be two mutually opposite directions; however, these directions may be displayed in a selectable mode, for example.

[0105] The terminal device 5-1 displays information for use in encouraging the user 2-1 to perform the above first user operation. A description thereof will be given with reference to Figs. 25 to 27.

[0106] Figs. 25 to 27 are diagrams of examples of an information display. In this example, the positioning device 55 is disposed on a desk.

[0107] As illustrated in Fig. 25, a message M6 is displayed. In accordance with the message M6 saying "place controller 56R", the user 2-1 places the controller 56R on the positioning device 55.

[0108] As illustrated in Fig. 26, a message M7 is displayed. In accordance with the message M7 saying "operate controller 56L", the user 2-1 operates the controller 56L. In this case, the operation of the controller 56L is an operation of designating an origin and horizontal biaxial directions in the coordinate system along the controller 56R. By performing such an operation using the controller 56L rather than the controller 56R, the user 2-1 can designate an origin and axial directions in the coordinate system while preventing the controller 56R placed in the positioning device 55 from moving.

[0109] As illustrated in Fig. 27, the user 2-1 designates an origin (corresponding to the point P1) and axial directions (directions from the point P1 to the point P2 and the point P3) in the coordinate system along the controller 56R. Then, a message M8 is displayed. In accordance with the message M8 saying "detach the controller 56R", the user 2-1 detaches the controller 56R from the positioning device 55. It should be noted that, if the user 2-2 performs the same operation next time, for example, an alarm message for use in preventing the movement of the positioning device 55 from the current position and orientation thereof may be displayed.

[0110] <Second user operation> The user 2-2 also performs the same operation. The user 2-2 places the controller 56R to be used by the user 2-2 on the positioning device 55 and operates the controller 56L. The user 2-2 designates an origin and horizontal biaxial directions in the coordinate system along the controller 56R placed on the positioning device 55. The vertical axial direction is also automatically designated.

[0111] <Matching of coordinate systems> Both the terminal device 5-1 and the terminal device 5-2 match the coordinate system C1 with the coordinate system C2, based on the result of the above first user operation and the result of the above second user operation. An example of this process is the coordinate conversion described above.

[0112] <Another configuration example of controller 56> In a case where a user 2 grips and uses the controller 56 with his / her hand, the user 2 may have difficulty performing another operation with the same hand or may accidentally drop the controller 56 when releasing his / her hand. To address such problems, a controller 56 having a configuration (e.g., shape, etc.) different from the above configuration may be used. A description thereof will be given with reference to Figs. 28 to 30.

[0113] Figs. 28 to 30 are views of examples of a schematic configuration of a controller 56. The controller 56 includes: a ring unit through which a finger of the user 2 is to pass; an operation unit (e.g., a slide pad, etc.) to be touched by a fingertip of the user 2; and a holding unit disposed opposite the holding unit with the ring unit therebetween. The user 2 uses the controller 56 with his / her finger passing through the ring unit and his / her hand holding the holding unit. Since a user 2 does not have to hold and use the controller 56 with his / her hand, it is possible to perform another work, such as an operation of a device other than the terminal device 5 (e.g., an operation of a keyboard of a personal computer, etc.) with the same hand. In addition, the risk of a user 2 accidentally dropping the controller 56 is significantly reduced. For more specific contents of such a controller 56, refer to PTL 2 (International Publication No. WO 2024 / 090304).

[0114] The above controller 56 may also be a pair of controllers related to the right hand and the left hand of a user 2. Since the directions along the controller 56 may be two mutually opposite directions, these directions may be displayed in a selectable mode when an axial direction in the coordinate system is designated.

[0115] Figs. 29 and 30 illustrate a state where the controller 56 is placed on the positioning device 55. Similarly to the above, the position and directions of the controller 56 are designated as the origin (corresponding to the point P1) and the horizontal biaxial directions (directions from the point P1 to the point P2 and the point P3) in the coordinate system.

[0116] Fig. 31 is a flowchart of an example of a process (information processing method) to be performed by the information processing system 1. Some technical matters similar to those described above will be omitted as appropriate.

[0117] In Step S31, the terminal device 5-1 displays information for use in encouraging the user 2-1 to perform the first user operation. As in Step S31a, the user 2-1 places the controller 56 to be used by the user 2-1 on the positioning device 55 and designates an origin and axial directions in the coordinate system. The origin and the axial directions of the coordinate system are designated along the controller 56 placed on the positioning device 55.

[0118] In Step S32, the terminal device 5-2 displays information for use in encouraging the user 2-2 to perform the second user operation. As in Step S32a, the user 2-2 places the controller 56 to be used by the user 2-2 on the positioning device 55 and designates an origin and axial directions that are the same as the origin and axial directions that have been designated by the user 2-1. The origin and the axial directions of the coordinate system are designated along the controller 56 placed on the positioning device 55.

[0119] In Step S33, each of the terminal device 5-1 and the terminal device 5-2 matches the coordinates in the coordinate system C1 and the coordinate system C2.

[0120] Based on the coordinate system C1 and the coordinate system C2 that have been matched with each other, namely, based on the coordinate system C, each of the terminal device 5-1 and the terminal device 5-2 displays the virtual object 4v. The user 2-1 and the user 2-2 can recognize the same object 4 at the same place inside the space 3.

[0121] It should be noted that, in Steps S31, S32, and S33 described above, the terminal device 5-1 and the terminal device 5-2 may exchange no data therebetween.

[0122] 2. Modifications The disclosed techniques are not limited to the above embodiments. Some modifications will be described.

[0123] In the above embodiment, two users 2, or a user 2-1 and a user 2-2, have been described as an example of users 2 present inside the space 3. However, three or more users 2 may be present inside the space 3. The same applies to terminal devices 5.

[0124] In one embodiment, another user may participate (remotely participate), from the outside of the space 3, for example, in the recognition of a virtual object 4v by the users 2 inside the space 3. A description thereof will be given with reference to Fig. 32.

[0125] Fig. 32 is a view of an example of remote participation. Another user outside space 3 is referred to as a user 7 in the drawing. An information processing system 1 further includes a VR device 8 (HMD device in this example). The user 7 can use the VR device 8 to see a virtual object 4v inside the space 3 together with a user 2-1 and a user 2-2. The VR device 8 displays both an image (may contain the user 2-1 the user 2-2) inside the space 3 and the virtual object 4v. Information necessary for generating the image inside the space 3 is prepared in advance or acquired in real time.

[0126] Each of a terminal device 5-1 and a terminal device 5-2 displays not only the virtual object 4v but also the user 7 who is virtualized. The virtualized user 7 is referred to as an avatar 7v in the drawing. The user 7 can move inside the space 3 as the avatar 7v and view the virtual object 4v from a viewpoint from a desired position. The position of the avatar 7v inside the space 3 is also shared by the server device 6. The user 2-1, the user 2-2, and the user 7 can recognize the virtual object 4v at the same place inside the space 3 in a realistic feeling as if the user 2-1, the user 2-2, and the user 7 were present at the same place beyond the space.

[0127] 3. Example of Hardware Configuration Fig. 33 is a diagram of an example of a hardware configuration of an apparatus. The terminal devices 5 and some other devices according to an embodiment of the present disclosure described above are each implemented, for example, by a computer 1000 having a configuration as illustrated. The computer 1000 includes processing circuitry 1100, a RAM 1200, a ROM 1300, a secondary storage device 1400, a communication interface 1500, an input / output interface 1600, a display unit 1700, a camera unit 1800, a microphone 1900, and a speaker 2000. The individual units of the computer 1000 are interconnected via a bus 1050.

[0128] The processing circuitry 1100 operates based on a program stored in the ROM 1300 or the secondary storage device 1400, thereby controlling each unit. For example, the processing circuitry 1100 expands, in the RAM 1200, a program stored in the ROM 1300 or the secondary storage device 1400, thereby performing processes related to various programs.

[0129] The ROM 1300 stores a boot program, such as a basic input output system (BIOS) to be executed by the processing circuitry 1100 during startup of the computer 1000 and a program depending on the hardware of the computer 1000, and some other programs.

[0130] The secondary storage device 1400 is a computer-readable recording medium, which non-transiently records a program to be executed by the processing circuitry 1100, data to be used by the program, and other information. More specifically, the secondary storage device 1400 is a recording medium that records a program (e.g., information processing program 500) for each process to be performed by the terminal devices 5 according to an embodiment of the present disclosure: this program is an example of program data 1450.

[0131] The communication interface 1500 is an interface via which the computer 1000 is connected to an external network 1550. For example, the processing circuitry 1100 receives data from another apparatus or transmits data generated by the processing circuitry 1100 to another apparatus, via the communication interface 1500.

[0132] The input / output interface 1600 is an interface via which the input / output device 1650 is connected to the computer 1000. For example, the processing circuitry 1100 receives data from an input device, such as a microphone 1900 or a touch panel, via the input / output interface 1600. In addition, the processing circuitry 1100 transmits data to output devices, such as the display unit 1700 and the speaker 2000, via the input / output interface 1600. Furthermore, the input / output interface 1600 may function as a medium interface that reads a program and data recorded in a predetermined recording medium (medium). Examples of the medium include: an optical recording medium such as a digital versatile disc (DVD) or a phase change rewritable disk (PD); a magneto-optical recording medium such as a magneto-optical disk (MO), a tape medium, a magnetic recording medium, and a semiconductor memory.

[0133] The display unit 1700 is an interface that displays information that has been processed by the computer 1000. Examples of the display unit 1700 include a liquid crystal display and an organic electro luminescence (EL) display. Furthermore, the display unit 1700 may be a touch panel display device or a video projection device. The display unit 1700 may correspond to the display unit 52 (Fig. 2) of each terminal device 5.

[0134] The camera unit 1800 is an interface via which the computer 1000 captures an image. The microphone 1900 is an interface via which the computer 1000 captures a sound. The camera unit 1800 and the microphone 1900 correspond to examples of the sensor unit 51 (Fig. 2) of each terminal device 5. The speaker 2000 is an interface that outputs a sound that has been processed by the computer 1000. The individual units of the computer 1000 are interconnected via a bus 1050. Each interface does not necessarily have to be mounted inside the computer 1000; alternatively, each interface may be disposed outside the computer 1000 via a network, for example. Furthermore, each of the units constituting the computer 1000 may be controlled by a circuit other than the processing circuitry 1100. For example, the display unit 1700 may be controlled by a display-process dedicated circuit mounted in the display unit 1700 rather than the processing circuitry 1100.

[0135] For example, if the computer 1000 functions as a terminal device 5 according to an embodiment of the present disclosure, the processing circuitry 1100 of the computer 1000 functions as the processing unit 53 (Fig. 2) by executing a program that has been loaded onto the RAM 1200. In addition, the secondary storage device 1400 stores the information processing program 500 according to the present disclosure and various types of data memorized in the memory unit 50. In this case, the processing circuitry 1100 reads the program data 1450 from the secondary storage device 1400 and executes the program data 1450; however, as another example, the processing circuitry 1100 acquires such programs from another apparatus via the external network 1550. In short, the secondary storage device 1400 does not have to be mounted inside the computer 1000; alternatively, the secondary storage device 1400 may be disposed outside the computer 1000. It should be noted that the processing circuitry 1100 is an example of an integrated circuit. Likewise, any of the CPU, the MPU, the GPU, the APU, the ASIC, and the FPGA can also be regarded as an integrated circuit.

[0136] 4. Conclusion The foregoing techniques are specified as follows, for example. One of the disclosed techniques is an information processing method. As described with reference to Figs. 1, 2, 5 to 32, and some other drawings, the information processing method includes: displaying, by a terminal device 5-1 (first terminal device) used by a user 2-1 (first user) present inside space 3 (specific space), information for encouraging the user 2-1 to perform a first user operation of designating a coordinate system inside the space 3 (Step S1, Step S11, Step S21, and Step S31); displaying, by a terminal device 5-2 (second terminal device) used by a user 2-2 (second user) present inside the space 3, information for encouraging the user 2-2 to perform a second user operation of designating the coordinate system inside the space 3 (Step S2, Step S11, Step S21, and Step S32); and matching a coordinate system C1 of the terminal device 5-1 with a coordinate system C2 of the terminal device 5-2, based on results of the first user operation and the second user operation (Step S3, Step S12, Step S22, and Step S33).

[0137] According to the above information processing system 1, the user 2-1 and the user 2-2 present inside the same space 3 designate coordinate systems inside the space 3. Therefore, the user 2-1 and the user 2-2 can designate appropriate coordinate systems with a high possibility. Based on this result, it is possible to accurately match the coordinate system C1 of the terminal device 5-1 with the coordinate system C2 of the terminal device 5-2.

[0138] As described with reference to Figs. 5 to 7, 10 to 13, 15 to 31, and some other drawings, the first user operation may include designating, by the user 2-1, an origin and an axial direction in the coordinate system inside the space 3 (Step S1a, Step S11a, Step S21a, and Step S31a). The second user operation may include designating, by the user 2-2, an origin and an axial direction inside the space 3 which are identical to the origin and the axial direction that have been designated by the user 2-1 (Step S2a, Step S11a, Step S21a, and Step S32a). Based on the same coordinate system designated by the first user operation and the second user operation, it is possible to accurately match the coordinate system C1 with the coordinate system C2.

[0139] As described with reference to Figs. 9, 10, 14, 15, 23, 31, and some other drawings, matching of the coordinate systems (Step S3, Step S12, Step S22, and Step S33) may include converting the coordinate system C1 of in the terminal device 5-1 into a coordinate system having the origin and the axial direction that have been designated by the user 2-1 and converting the coordinate system C2 of the terminal device 5-2 into a coordinate system having the origin and the axial direction that have been designated by the user 2-2. Using the coordinate conversion in this manner, for example, can match the coordinate system C1 with the coordinate system C2.

[0140] As described with reference to Figs. 5 to 8, 10 to 13, 15, and some other drawings, the first user operation may include designating, by the user 2-1, an origin and an axial direction in the coordinate system along a corner portion 4a present inside the space 3 (Step S1a and Step S11a). The second user operation may include designating, by the user 2-2, an origin and an axial direction along the corner portion 4a which are identical to the origin and axial direction that have been designated by the user 2-1 (Step S2a and Step S11a). As described with reference to Figs. 5 to 8, 10, and some drawings, the designation in the first user operation may include touching, by the user 2-1 directly or with a controller 54, a specific position inside the space 3 (Step S1a). The designation in the second user operation may include touching, by the user 2-2 directly or with the controller 54, a position identical to the position which has been touched by the user 2-1 (Step S2a). Alternatively, as described with reference to Figs. 11 to 13, 15, and some other drawings, the information processing method may further include displaying, in a selectable mode, a plurality of corner portions 4a present inside the space 3 by each of the terminal device 5-1 and the terminal device 5-2 (Step S11). The first user operation may include designating, by the user 2-1, an origin and an axial direction in the coordinate system along the corner portion 4a selected by the user 2-1 from among the plurality of corner portions 4a (Step S11a). The second user operation may include designating, by the user 2-2, an origin and an axial direction along the corner portion 4a selected by the user 2-1 which are identical to the origin and axial direction that have been designated by the user 2-1. In this case, each of the terminal device 5-1 and the terminal device 5-2 may also display, in a selectable mode, some other origins and axial directions designable along the corner portion 4a. Using the corner portion 4a in this manner, for example, can designate the origin and axial direction in the coordinate system.

[0141] As described with reference to Figs. 16 to 23 and some other drawings, the information processing method may further include displaying, by each of the terminal device 5-1 and the terminal device 5-2, a virtual space image 3v in which occlusion of the space 3 is reproduced (Step S21). The first user operation may include matching, by using the user 2-1, the virtual space image 3v with the space 3 (Step S21a). The second user operation may include matching, by the user 2-2, the virtual space image 3v with the space 3 (Step S21a). In this case, for example, the first user operation may include moving the user 2-1 inside the space 3 and moving the virtual space image 3v (Step S21a). The second user operation may include moving the user 2-2 inside the space 3 and moving the virtual space image 3v (Step S21a). Using the virtual space image 3v for occlusion in this manner, for example, can also designate the coordinate systems.

[0142] As described with reference to Figs. 24 to 31 and some other drawings, the first user operation may include placing, by the user 2-1, a controller 56 on a positioning device 55 disposed inside the space 3 (Step S31a). The second user operation may include placing, by the user 2-2, a controller 56 on the positioning device 55 (Step S32a). In this case, for example, a position and orientation of each controller 56 placed on the positioning device 55 with respect to the positioning device 55 may be predetermined. Using the positioning device 55 and the controllers 56 in this manner, for example, can also designate the coordinate systems.

[0143] As described with reference to Fig 1 and some other drawings, the space 3 may have an enough size for the user 2-1 and the user 2-2 to be able to conduct F-to-F communication with each other. The user 2-1 and the user 2-2 can designate appropriate coordinate systems with an even higher possibility.

[0144] As described with reference to Figs. 10, 15, 23, 31, and some other drawings, the information processing method may include exchanging no data between the terminal device 5-1 and the terminal device 5-2. Thus, this method is also advantageous in protecting privacy information.

[0145] An information processing system 1 described with reference to Figs. 1, 2, 5 to 32, and some other drawings is also another disclosed technique. The information processing system 1 includes: a display unit 52 that displays information for use in encouraging a user 2-1 (first user) present inside space 3 (specific space) to perform a first user operation of designating a coordinate system inside space 3 and information for use in encouraging a user 2-2 (second user) present inside the space 3 to perform a second user operation of designating the coordinate system inside space 3; and a processing unit 53 that matches a coordinate system C1 of a terminal device 5-1 (first terminal device) used by the user 2-1 with a coordinate system C2 of a terminal device 5-2 (second terminal device) used by the user 2-2, based on results of the first user operation and the second user operation. The information processing system 1, as described above, can also accurately match the coordinate system C1 of the terminal device 5-1 with the coordinate system C2 of the terminal device 5-2.

[0146] It should be noted that the effects described in the present disclosure are simply examples and are not limited to the disclosed ones. Some other effects can also be produced.

[0147] Although embodiments of the present disclosure have been described above, the technical scope of the present disclosure is not limited to the foregoing embodiments as it is. Various modifications can be made without departing from the spirit of the present disclosure. Moreover, some components of different embodiments and modifications may be combined as appropriate.

[0148] The present techniques can also employ the following configurations. (1) 　　An information processing method including: 　　displaying, by a first terminal device used by a first user present inside a particular space, information for use in encouraging the first user to perform a first user operation of designating a coordinate system inside the space; 　　displaying, by a second terminal device used by a second user present inside the space, information for use in encouraging the second user to perform a second user operation of designating the coordinate system inside the space; and 　　matching the coordinate system of the first terminal device with the coordinate system of the second terminal device, based on results of the first user operation and the second user operation. (2) 　　The information processing method according to (1), wherein 　　the first user operation includes designating, by the first user, an origin and an axial direction in the coordinate system inside the space, and 　　the second user operation includes designating, by the second user, an origin and an axial direction inside the space which are identical to the origin and the axial direction that have been designated by the first user. (3) 　　The information processing method according to (2), wherein 　　matching of the coordinate systems includes: 　　converting the coordinate system of the first terminal device into a coordinate system having the origin and the axial direction that have been designated by the first user; and 　　converting the coordinate system of the second terminal device into a coordinate system having the origin and the axial direction that have been designated by the second user. (4) 　　The information processing method according to any one of (1) to (3), wherein 　　the first user operation includes designating, by the first user, an origin and an axial direction in the coordinate system along a corner portion present inside the space, and 　　the second user operation includes designating, by the second user, an origin and an axial direction along the corner portion which are identical to the origin and the axial direction that have been designated by the first user. (5) 　　The information processing method according to (4), wherein 　　the designation in the first user operation includes touching, by the first user directly or with a controller, a specific position inside the space, and 　　the designation in the second user operation includes touching, by using the second user directly or with the controller, a position identical to the position that has been touched by the first user. (6) 　　The information processing method according to any one of (1) to (4), further including 　　displaying, in a selectable mode, a plurality of corner portions present inside the space by each of the first terminal device and the second terminal device, wherein 　　the first user operation includes designating, by the first user, an origin and an axial direction in the coordinate system along a corner portion selected from among the plurality of corner portions, and 　　the second user operation includes designating, by the second user, an origin and an axial direction along the corner portion selected by the first user which are identical to the origin and the axial direction that have been designated by the first user. (7) 　　The information processing method according to (6), wherein 　　each of the first terminal device and the second terminal device also displays, in a selectable mode, the origin and an axial direction designable along the corner portion. (8) 　　The information processing method according to any one of (1) to (3), further including 　　displaying, by each of the first terminal device and the second terminal device, a virtual space image in which occlusion of the space is reproduced, wherein 　　the first user operation includes matching, by the first user, the virtual space image with the space, and 　　the second user operation includes matching, by the second user, the virtual space image with the space. (9) 　　The information processing method according to (8), wherein 　　the first user operation includes moving the first user inside the space and moving the virtual space image, and 　　the second user operation includes moving the second user inside the space and moving the virtual space image. (10) 　　The information processing method according to any one of (1) to (3), wherein 　　the first user operation includes placing, by the first user, a controller on a positioning device disposed inside the space, and 　　the second user operation includes placing, by the second user, the controller on the positioning device. (11) 　　The information processing method according to (10), wherein 　　a position and orientation of each controller placed on the positioning device are predetermined with respect to the positioning device. (12) 　　The information processing method according to any one of (1) to (11), wherein 　　the space has an enough size for the first user and the second user to be able to conduct face-to-face communication with each other. (13) 　　The information processing method according to any one of (1) to (12), including 　　exchanging no data between the first terminal device and the second terminal device. (14) 　　An information processing system including: 　　a display unit that displays information for use in encouraging a first user present inside specific space to perform a first user operation of designating a coordinate system inside the space and information for use in encouraging a second user present inside the space to perform a second user operation of designating the coordinate system inside the space; and 　　a processing unit that matches the coordinate system of a first terminal device used by the first user with the coordinate system of a second terminal device used by the second user, based on results of the first user operation and the second user operation. (15) 　　The information processing system according to (14), wherein 　　the first user operation includes designating, by the first user, an origin and an axial direction in the coordinate system inside the space, and 　　the second user operation includes designating, by the second user, an origin and an axial direction inside the space which are identical to the origin and the axial direction that have been designated by the first user. (16) The information processing system according to (15), wherein the processing unit converts the coordinate system of the first terminal device into a coordinate system having the origin and the axial direction that have been designated by the first user, and converts the coordinate system of the second terminal device into a coordinate system having the origin and the axial direction that have been designated by the second user. (17) 　　The information processing system according to any one of (14) to (16), wherein 　　the first user operation includes designating, by the first user, an origin and an axial direction in the coordinate system along a corner portion present inside the space, and 　　the second user operation includes designating, by the second user, an origin and an axial direction along the corner portion which are identical to the origin and the axial direction that have been designated by the first user. (18) 　　The information processing system according (17), wherein 　　the designation in the first user operation includes touching, by the first user directly or with a controller, a specific position inside the space, and 　　the designation in the second user operation includes touching, by using the second user directly or with the controller, a position identical to the position that has been touched by the first user. (19) 　　The information processing system according to any one of (14) to (17), wherein 　　the display unit displays, in a selectable mode, a plurality of corner portions present inside the space, 　　the first user operation includes designating, by the first user, an origin and an axial direction in the coordinate system along a corner portion selected from among the plurality of corner portions, and 　　the second user operation includes designating, by the second user, an origin and an axial direction along the corner portion selected by the first user which are identical to the origin and the axial direction that have been designated by the first user. (20) The information processing system according to (19), wherein the display unit also displays, in a selectable mode, an origin and an axial direction designable along the corner portion. (21) 　　The information processing system according to any one of (14) to (16), wherein 　　the display unit displays a virtual space image in which occlusion of the space is reproduced, 　　the first user operation includes matching, by the first user, the virtual space image with the space, and 　the second user operation includes matching, by the second user, the virtual space image with the space. (22) The information processing system according to (21), wherein the first user operation includes moving the first user inside the space and moving the virtual space image, and the second user operation includes moving the second user inside the space and moving the virtual space image. (23) 　　The information processing system according to any one of (14) to (16), wherein 　　the first user operation includes placing, by the first user, a controller on a positioning device disposed inside the space, and 　　the second user operation includes placing, by the second user, the controller on the positioning device. (24) The information processing system according to (23), wherein a position and orientation of the controller placed on the positioning device with respect to the positioning device are predetermined. (25) The information processing system according to one of (14) to (24), wherein the space has an enough size for the first user and the second user to be able to conduct face-to-face communication with each other. (26) The information processing system according to one of (14) to (25), wherein the first terminal device and the second terminal device exchange no data therebetween. (27) An information processing system including circuitry configured to 　　control display of first information to guide a first user to perform a first user operation to designate a first coordinate system inside a space, 　　recognize second information related to the designated first coordinate system, and 　　determine a coordinate system of the information processing system to be the designated first coordinate system. (28) The information processing system according to (27), wherein the circuity is further configured to control display of third information to guide the first user to share information related to the designated first coordinate system with a second user. (29) The information processing system according to (27) or (28), wherein the first information relates to a corner portion present inside the space. (30) The information processing system according to any one of (27) to (29), wherein the first information is related to an origin of the designated first coordinate system at a vertex of the corner portion. (31) The information processing system according to any one of (27) to (30), wherein the second information includes at least one axial direction of the designated first coordinate system, the at least one axial direction being designated with respect to the origin at the vertex of the corner portion. (32) The information processing system according to any one of (27) to (31), wherein the circuitry controls a first display to display the third information to guide the first user to share information related to the origin of the designated first coordinate system with the second user. (33) The information processing system according to the any one of (27) to (32), wherein the circuitry controls a first display to display the third information to guide the first user to share information related to the at least one axial direction of the designated first coordinate system with the second user. (34) An information processing method including 　　displaying, by a first terminal device used by a first user present inside a space, information to guide the first user to perform a first user operation of designating a first coordinate system inside the space, and 　　displaying, by a second terminal device used by a second user present inside the space, information to guide the second user to perform a second user operation of designating a second coordinate system inside the space, the second coordinate system corresponding to the first coordinate system designated based on the first user operation. (35) The information processing method according to (34), wherein the first user operation includes designating, by the first user, an origin and at least one axial direction inside the space, and wherein the second user operation includes designating, by the second user, an origin and at least one axial direction inside the space to be identical to the origin and the at least one axial direction that have been designated by the first user. (36) The information processing method according to (34) or (35), wherein a coordinate system of the first terminal device is converted into the designated first coordinate system based on the first user designating the origin and the at least one axial direction with respect to a vertex of a corner portion inside the space, and wherein a coordinate system of the second terminal device is converted into the designated second coordinate system based on the second user designating the origin and the at least one axial direction with respect to the vertex of the corner portion inside the space. (37) The information processing method according to any of (34) to (36), wherein the first user operation includes designating, by the first user, an origin and an axial direction in the coordinate system along a corner portion present inside the space, and wherein the second user operation includes designating, by the second user, an origin and an axial direction along the corner portion which are identical to the origin and the axial direction that have been designated by the first user. (38) The information processing method according to any of (34) to (37), wherein the designation in the first user operation includes indicating, by the first user directly touching or by using a first controller, at least one specific position inside the space, and wherein the designation in the second user operation includes indicating, by the second user directly touching or by using a second controller, at least one position corresponding to each position that has been indicated by the first user. (39) The information processing method according to any of (34) to (38), further including displaying, in a selectable mode, a plurality of corner portions present inside the space by each of the first terminal device and the second terminal device, wherein the first user operation includes designating, by the first user, an origin and an axial direction along a corner portion selected from among the plurality of corner portions, and wherein the second user operation includes designating, by the second user, an origin and an axial direction along the corner portion which are identical to the origin and the axial direction that have been designated by the first user. (40) The information processing method according to any of (34) to (39), wherein each of the first terminal device and the second terminal device also displays, in the selectable mode, the origin at a vertex of the selected corner portion and at least one axial direction designable along the selected corner portion. (41) The information processing method according to any of (34) to (40), further including displaying, by each of the first terminal device and the second terminal device, a virtual space image in which occlusion of the space is reproduced, wherein the first user operation includes matching, by the first user, the virtual space image with the space, and wherein the second user operation includes matching, by the second user, the virtual space image with the space. (42) The information processing method according to any of (34) to (41), wherein the first user operation includes moving the first user inside the space and moving the virtual space image, and wherein the second user operation includes moving the second user inside the space and moving the virtual space image. (43) The information processing method according to any of (34) to (42), wherein the first user operation includes placing, by the first user, a first controller on a positioning device disposed inside the space, and wherein the second user operation includes placing, by the second user, a second controller on the positioning device. (44) The information processing method according to any of (34) to (43), wherein a position and an orientation of each controller placed on the positioning device are predetermined with respect to the positioning device. (45) The information processing method according to any of (34) to (44), wherein the space is configured for the first user and the second user to conduct face-to-face communication with each other. (46) The information processing method according to any of (34) to (45), further including displaying the information to guide each user to perform each user operation without exchanging data between the first terminal device and the second terminal device.

[0149] 1 Information processing system 2 User 2-1 User 2-2 User 3 Space 3v Virtual space image 3vp Origin 4 Object 4a Corner portion 4o Occlusion portion 5 Terminal device 5-1 Terminal device 5-2 Terminal device 50 Memory unit 500 Information processing program 51 Sensor unit 52 Display unit 53 Processing unit 54 Controller 55 Positioning device 56 Controller 56L Controller 56R Controller 5v image-capturing range 6 Server device 60 Memory unit 600 Position information 61 Processing unit 7 User 7v Avatar 8 VR device C Coordinate system C1 Coordinate system C2 Coordinate system M1 Message M2 Message M3 Message M4 Message M5 Message M6 Message M7 Message M8 Message P1 Point P2 Point P3 Point

Claims

1. An information processing system comprising: 　　circuitry configured to 　　　　control display of first information to guide a first user to perform a first user operation to designate a first coordinate system inside a space, 　　　　recognize second information related to the designated first coordinate system, and 　　　　determine a coordinate system of the information processing system to be the designated first coordinate system.

2. The information processing system according to claim 1, 　　wherein the circuity is further configured to control display of third information to guide the first user to share information related to the designated first coordinate system with a second user.

3. The information processing system according to claim 2, 　　wherein the first information relates to a corner portion present inside the space.

4. The information processing system according to claim 3, 　　wherein the first information is related to an origin of the designated first coordinate system at a vertex of the corner portion.

5. The information processing system according to claim 4, 　　wherein the second information includes at least one axial direction of the designated first coordinate system, the at least one axial direction being designated with respect to the origin at the vertex of the corner portion.

6. The information processing system according to the claim 4, 　　wherein the circuitry controls a first display to display the third information to guide the first user to share information related to the origin of the designated first coordinate system with the second user.

7. The information processing system according to the claim 5, 　　wherein the circuitry controls a first display to display the third information to guide the first user to share information related to the at least one axial direction of the designated first coordinate system with the second user.

8. An information processing method comprising: 　　displaying, by a first terminal device used by a first user present inside a space, information to guide the first user to perform a first user operation of designating a first coordinate system inside the space; and 　　displaying, by a second terminal device used by a second user present inside the space, information to guide the second user to perform a second user operation of designating a second coordinate system inside the space, the second coordinate system corresponding to the first coordinate system designated based on the first user operation.

9. The information processing method according to claim 8, 　　wherein the first user operation includes designating, by the first user, an origin and at least one axial direction inside the space, and 　　wherein the second user operation includes designating, by the second user, an origin and at least one axial direction inside the space to be identical to the origin and the at least one axial direction that have been designated by the first user.

10. The information processing method according to claim 9, 　　wherein a coordinate system of the first terminal device is converted into the designated first coordinate system based on the first user designating the origin and the at least one axial direction with respect to a vertex of a corner portion inside the space, and 　　wherein a coordinate system of the second terminal device is converted into the designated second coordinate system based on the second user designating the origin and the at least one axial direction with respect to the vertex of the corner portion inside the space.

11. The information processing method according to claim 8, 　　wherein the first user operation includes designating, by the first user, an origin and an axial direction in the coordinate system along a corner portion present inside the space, and 　　wherein the second user operation includes designating, by the second user, an origin and an axial direction along the corner portion which are identical to the origin and the axial direction that have been designated by the first user.

12. The information processing method according to claim 11, 　　wherein the designation in the first user operation includes indicating, by the first user directly touching or by using a first controller, at least one specific position inside the space, and 　　wherein the designation in the second user operation includes indicating, by the second user directly touching or by using a second controller, at least one position corresponding to each position that has been indicated by the first user.

13. The information processing method according to claim 8, further comprising: 　　displaying, in a selectable mode, a plurality of corner portions present inside the space by each of the first terminal device and the second terminal device, 　　wherein the first user operation includes designating, by the first user, an origin and an axial direction along a corner portion selected from among the plurality of corner portions, and 　　wherein the second user operation includes designating, by the second user, an origin and an axial direction along the corner portion which are identical to the origin and the axial direction that have been designated by the first user.

14. The information processing method according to claim 13, 　　wherein each of the first terminal device and the second terminal device also displays, in the selectable mode, the origin at a vertex of the selected corner portion and at least one axial direction designable along the selected corner portion.

15. The information processing method according to claim 8, further comprising: 　　displaying, by each of the first terminal device and the second terminal device, a virtual space image in which occlusion of the space is reproduced, 　　wherein the first user operation includes matching, by the first user, the virtual space image with the space, and 　　wherein the second user operation includes matching, by the second user, the virtual space image with the space.

16. The information processing method according to claim 15, 　　wherein the first user operation includes moving the first user inside the space and moving the virtual space image, and 　　wherein the second user operation includes moving the second user inside the space and moving the virtual space image.

17. The information processing method according to claim 8, 　　wherein the first user operation includes placing, by the first user, a first controller on a positioning device disposed inside the space, and 　　wherein the second user operation includes placing, by the second user, a second controller on the positioning device.

18. The information processing method according to claim 17, 　　wherein a position and an orientation of each controller placed on the positioning device are predetermined with respect to the positioning device.

19. The information processing method according to claim 8, 　　wherein the space is configured for the first user and the second user to conduct face-to-face communication with each other.

20. The information processing method according to claim 19, further comprising: 　　displaying the information to guide each user to perform each user operation without exchanging data between the first terminal device and the second terminal device.

Images