Information processing method and information processing system

Abstract

Accurately align the coordinate systems. [Solution] The information processing method includes displaying, by a first terminal device used by a first user, information for causing a first user in the space to perform a first user operation to specify a coordinate system within a specific space; displaying, by a second terminal device used by a second user, information for causing a second user in the space to perform a second user operation to specify a coordinate system within the space; and aligning the coordinate system held by the first terminal device and the coordinate system held by the second terminal device based on the results of the first user operation and the second user operation.

Description

[Technical Field]

[0001] The present disclosure relates to an information processing method and an information processing system. [Background technology]

[0002] In AR (Augmented Reality) and the like, in order for multiple users to recognize a virtual object at the same position in space, it is necessary to match the coordinate systems of the terminal devices of each user. For example, Patent Document 1 proposes a technology for matching the coordinate systems of two terminal devices based on the position information of the other terminal device when viewed in the coordinate system of the other terminal device. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2023-98905 [Patent Document 2] International Publication No. 2024 / 090304 Summary of the Invention [Problem to be solved by the invention]

[0004] The accuracy of the coordinate system match is important and there is room for improvement.

[0005] One aspect of the present disclosure is to match coordinate systems with high precision. [Means for solving the problem]

[0006] An information processing method according to one aspect of the present disclosure includes displaying, by a first terminal device used by a first user, information for causing a first user in the space to perform a first user operation to specify a coordinate system within a specific space; displaying, by a second terminal device used by a second user, information for causing a second user in the space to perform a second user operation to specify a coordinate system within the space; and aligning the coordinate system held by the first terminal device and the coordinate system held by the second terminal device based on the results of the first user operation and the second user operation.

[0007] An information processing system according to one aspect of the present disclosure includes a display unit that displays information for causing a first user in a space to perform a first user operation to specify a coordinate system within a specific space, and information for causing a second user in the space to perform a second user operation to specify a coordinate system within the space, and a processing unit that matches the coordinate system of a first terminal device used by the first user with the coordinate system of a second terminal device used by a second user based on the results of the first user operation and the second user operation. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a diagram illustrating an example of a schematic configuration of an information processing system 1 according to an embodiment. [Figure 2] FIG. 2 is a diagram illustrating an example of a schematic configuration of a terminal device 5 and a server device 6. [Figure 3] FIG. 10 illustrates an example of a mismatched coordinate system. [Figure 4] FIG. 10 is a diagram illustrating an example of a matching coordinate system. [Figure 5] FIG. 10 is a diagram illustrating an example of a first technique. [Figure 6] FIG. 10 is a diagram illustrating an example of information display. [Figure 7] FIG. 10 is a diagram illustrating an example of information display. [Figure 8] FIG. 10 is a diagram illustrating an example of information display. [Figure 9] FIG. 10 is a diagram illustrating an example of a position in a coordinate system. [Figure 10]2 is a flowchart showing an example of a process (information processing method) executed in the information processing system 1. [Figure 11] FIG. 10 is a diagram illustrating an example of a second technique. [Figure 12] FIG. 10 is a diagram illustrating an example of information display. [Figure 13] FIG. 10 is a diagram illustrating an example of information display. [Figure 14] FIG. 10 is a diagram illustrating an example of a coordinate system after matching. [Figure 15] 2 is a flowchart showing an example of a process (information processing method) executed in the information processing system 1. [Figure 16] FIG. 10 is a diagram illustrating an example of a third technique. [Figure 17] FIG. 10 is a diagram illustrating an example of a third technique. [Figure 18] FIG. 10 is a diagram illustrating an example of a third technique. [Figure 19] FIG. 10 is a diagram illustrating an example of information display. [Figure 20] FIG. 10 is a diagram illustrating an example of information display. [Figure 21] FIG. 10 is a diagram illustrating an example of information display. [Figure 22] FIG. 10 is a diagram illustrating an example of information display. [Figure 23] 2 is a flowchart showing an example of a process (information processing method) executed in the information processing system 1. [Figure 24] FIG. 10 is a diagram illustrating an example of a fourth technique. [Figure 25] FIG. 10 is a diagram illustrating an example of information display. [Figure 26] FIG. 10 is a diagram illustrating an example of information display. [Figure 27] FIG. 10 is a diagram illustrating an example of information display. [Figure 28] FIG. 2 is a diagram illustrating an example of a schematic configuration of a controller 56. [Figure 29] FIG. 2 is a diagram illustrating an example of a schematic configuration of a controller 56. [Figure 30] FIG. 2 is a diagram illustrating an example of a schematic configuration of a controller 56. [Figure 31]2 is a flowchart showing an example of a process (information processing method) executed in the information processing system 1. [Figure 32] FIG. 10 illustrates an example of remote participation. [Figure 33] FIG. 2 illustrates an example of a hardware configuration of the apparatus. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. In the following embodiments, the same elements are designated by the same reference numerals, and redundant description will be omitted.

[0010] The present disclosure will be described in the following order: 1. Embodiment 1.1 First Method 1.2 Second Method 1.3 Third Method 1.4 The fourth method 2. Variations 3. Hardware configuration example 4. Summary

[0011] 1. Embodiment FIG. 1 is a diagram showing an example of a schematic configuration of an information processing system 1 according to an embodiment. A user of the information processing system 1 is referred to as a user 2. There are multiple users 2, and FIG. 1 shows two users 2 as an example. A first user 2 is referred to as a user 2-1 and illustrated. A second user 2 is referred to as a user 2-2 and illustrated. When there is no particular distinction between these users, they are simply referred to as users 2, etc.

[0012] User 2-1 and user 2-2 use the information processing system 1 in the same space. This particular space is referred to as space 3 and is illustrated in the figure. Space 3 may be large enough to allow user 2-1 and user 2-2 to communicate with each other face-to-face (F2F). An example of space 3 is a room (a conference room, a laboratory, etc.). In space 3, user 2-1 and user 2-2 can communicate with each other by speech, gestures, etc. while looking at the same place in space 3 or seeing each other's appearance.

[0013] Various objects exist in the space 3. Some of the objects are illustrated and referred to as objects 4. Examples of objects 4 include floors, walls, desks, doors, etc.

[0014] The information processing system 1 realizes an integrated space in which a CG virtual object 4v (a car in this example) appears to exist at a specific location within the space 3. Such an integrated space is also called an XR space, and the information processing system 1 can also be called an XR system.

[0015] The information processing system 1 can be used for AR, VR (Virtual Reality), MR (Mixed Reality), SR (Substitutional Reality), etc. Specific uses of the information processing system 1 are not particularly limited, but examples include CAD (Computer Aided Design), communication between users 2, and viewing VR characters.

[0016] The information processing system 1 includes a terminal device 5 and a server device 6.

[0017] The terminal device 5 is used by a user 2 in the space 3. The illustrated terminal device 5 is a head-mounted display (HMD) type device worn on the head of the user 2. However, the terminal device 5 is not limited to an HMD type device. Other examples of the device include a glasses-type device such as AR glasses, a smartphone, a tablet terminal, etc.

[0018] There are multiple terminal devices 5 corresponding to multiple users 2. The terminal device 5 of user 2-1 is referred to and illustrated as terminal device 5-1. The terminal device 5 of user 2-2 is referred to and illustrated as terminal device 5-2. When there is no particular distinction between these, they are simply referred to as terminal devices 5, etc.

[0019] The server device 6 communicates with the terminal devices 5, more specifically, with the terminal devices 5-1 and 5-2. An example of the function of the server device 6 is a function for sharing location information between the terminal devices 5-1 and 5-2, which will be described later. The server device 6 may be provided outside the space 3.

[0020] 2 is a diagram showing an example of a schematic configuration of the terminal device 5 and the server device 6. The terminal device 5 includes a storage unit 50, a sensor unit 51, a display unit 52, and a processing unit 53.

[0021] The storage unit 50 stores information used by the terminal device 5. Information may be interpreted to include data, and may be interpreted as appropriate within the scope of no contradiction. FIG. 2 shows an information processing program 500 as an example of information stored in the storage unit 50. The information processing program 500 is a program (software) for causing a computer to function as the terminal device 5. The storage unit 50 is realized by, for example, a semiconductor memory element such as a RAM (Random Access Memory) or a flash memory, or a storage device such as a hard disk, an SSD (Solid State Drive), or an optical disk.

[0022] The sensor unit 51 includes various sensors. Examples of the sensors are a camera (image sensor), a distance measurement sensor, an acceleration sensor, a microphone (sound sensor), etc. These will be explained in order.

[0023] The camera (image sensor) captures images of the surroundings of the terminal device 5. Image data within the space 3 is obtained. This can be used to grasp the situation within the space 3, which may include the object 4. Multiple cameras may be used depending on the application. Examples of applications include SLAM (Simultaneous Localization and Mapping) and VST (Video See Through). One camera that can be used for multiple applications may also be used. Note that imaging may be interpreted to include shooting, and image may be interpreted to include video. To the extent that there is no contradiction, imaging and image may be interpreted as shooting and video, as appropriate.

[0024] The distance measurement sensor measures the distance from the terminal device 5 to the object 4. An example of a distance measurement sensor is a LiDAR sensor, which can measure the shape of the object 4 in addition to the distance to the object 4.

[0025] The acceleration sensor detects the acceleration of the terminal device 5. Data on the acceleration of the terminal device 5 is obtained. The acceleration sensor may be an IMU (Inertial Measurement Unit) sensor, and may also detect rotation, position, etc., making it possible to grasp the position, attitude, etc. of the terminal device 5 in the space 3. The acceleration sensor may be a gravity sensor. It is possible to detect and automatically grasp the direction of gravity (vertical direction).

[0026] The microphone (sound sensor) detects sounds around the terminal device 5. For example, sound data including the voice of the user 2 can be obtained. This can be used for voice operation of the terminal device 5, etc.

[0027] The sensors are not limited to those described above, and any sensors that can be mounted on the terminal device 5 may be included in the sensor unit 51. For example, a device (such as a button) for operating the terminal device 5 can also be one of the sensors.

[0028] The display unit 52 displays information. For example, a virtual object 4v is displayed. The terminal device 5 illustrated in FIG. 1 is an HMD. The display unit 52 may be a transparent display unit (transparent display), and the user 2 can see the virtual object 4v displayed in a superimposed state in the space 3.

[0029] The processing unit 53 executes various processes necessary for the operation of the terminal device 5. The processing unit 53 is realized by, for example, a central processing unit (CPU), a micro processing unit (MPU), a graphics processing unit (GPU), or the like executing an information processing program 500 stored in the storage unit 50 using a RAM or the like as a working area. The processing unit 53 is also a controller, and may be realized by, for example, an integrated circuit such as an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or a micro controller unit (MCU).

[0030] The processing unit 53 controls each element of the terminal device 5, thereby controlling the entire terminal device 5. For example, the processing unit 53 controls the memory unit 50 to write desired information to the memory unit 50 and read it from the memory unit 50. The processing unit 53 controls the sensor unit 51 to acquire various data such as images, acceleration, and sound, and to process the acquired data. Examples of data processing include image recognition processing, SLAM processing, and sound recognition processing. The processing unit 53 controls the display unit 52 to display various information on the display unit 52.

[0031] In the following description, when it is stated that the terminal device 5 performs some operation, the subject may be interpreted as the storage unit 50, the sensor unit 51, the display unit 52, or the processing unit 53 as appropriate.

[0032] The server device 6 includes a storage unit 60 and a processing unit 61. The storage unit 60 stores information used by the server device 6. The processing unit 61 executes various processes required for the operation of the server device 6.

[0033] In this example, the server device 6 functions as a position sharing server. The terminal device 5-1 transmits position information (information on position, orientation, etc.) of the terminal device 5-1 to the server device 6. The position information of the terminal device 5-1 is generated by the terminal device 5-1, for example, by the above-mentioned SLAM processing. The terminal device 5-2 transmits position information of the terminal device 5-2 to the server device 6. The position information of the terminal device 5-2 is generated by the terminal device 5-2, for example, by the above-mentioned SLAM processing. These pieces of position information are stored in the storage unit 60 of the server device 6 as position information 600. The position information 600 is transmitted from the server device 6 to each of the terminal device 5-1 and the terminal device 5-2. As a result, the position information is shared between the terminal device 5-1 and the terminal device 5-2.

[0034] Returning to FIG. 1, each of the terminal device 5-1 and the terminal device 5-2 has a coordinate system. One example of a coordinate system is the SLAM coordinate system, and unless otherwise specified, the coordinate systems owned by each of the terminal device 5-1 and the terminal device 5-2 are assumed to be the SLAM coordinate system. The coordinate system owned by the terminal device 5-1 is referred to as the coordinate system C1. The coordinate system owned by the terminal device 5-2 is referred to as the coordinate system C2. The terminal device 5-1 displays the virtual object 4v according to the coordinate system C1. The terminal device 5-2 displays the virtual object 4v according to the coordinate system C2.

[0035] Each of the coordinate systems C1 and C2 has three axes that are orthogonal to each other, and therefore has three axis directions, including two axis directions that are orthogonal to each other and extend horizontally (two horizontal axis directions) and one axis direction that is orthogonal to the two horizontal axis directions and extend vertically (one vertical axis direction).

[0036] One of the purposes of the information processing system 1 is to enable the user 2-1 and the user 2-2 to recognize the same virtual object 4v at the same location in the space 3. To achieve this, it is important that the coordinate system C1 of the terminal device 5-1 and the coordinate system C2 of the terminal device 5-2 coincide with each other. This will be described with reference to FIGS. 3 and 4.

[0037] 3 is a diagram showing an example of mismatched coordinate systems. In the example shown in FIG. 3, the coordinate system C1 and the coordinate system C2 do not match. In this case, the position of the virtual object 4v in the space 3 displayed by the terminal device 5-1 does not match the position of the virtual object 4v in the space 3 displayed by the terminal device 5-2. The users 2-1 and 2-2 perceive the same virtual object 4v in different locations within the space 3.

[0038] 4 is a diagram showing an example of coincident coordinate systems. Coordinate system C1 and coordinate system C2 coincide with each other, and this coordinate system is referred to as coordinate system C and illustrated. In this case, the position of virtual object 4v in space 3 displayed by terminal device 5-1 coincides with the position of virtual object 4v in space 3 displayed by terminal device 5-2. User 2-1 and user 2-2 can recognize the same virtual object 4v at the same location in space 3.

[0039] In order to accurately match the coordinate system C1 and the coordinate system C2, in the information processing system 1, each of the user 2-1 and the user 2-2 performs an operation to specify a coordinate system within the space 3. This operation performed by the user 2-1 is referred to as a first user operation. This operation performed by the user 2-2 is referred to as a second user operation.

[0040] In a first user operation, user 2-1 specifies the origin and axis directions of a coordinate system in space 3. In a second user operation, user 2-2 specifies the same origin and axis directions as those specified by user 2-1 in space 3. Some examples of specific techniques will be described below.

[0041] 1.1 First Method 5 is a diagram showing an example of the first technique, in which the imaging range of the terminal device 5 is schematically shown as imaging range 5v.

[0042] The origin and axis directions of the coordinate system are specified by specifying specific positions (points) in space 3. In this example, points P1, P2, and P3 are specified. Point P1 corresponds to the origin. The directions from point P1 to point P2 and from point P1 to point P3 correspond to the axis directions, more specifically, the two horizontal axis directions.

[0043] The first technique uses a corner 4a that exists in the space 3. The corner 4a includes two sides that extend horizontally and are perpendicular to each other. In this example, the corner 4a is located at a corner of a room (an example of the space 3) and is defined by a wall, a floor, etc.

[0044] <First user operation> User 2-1 specifies the origin and axis directions of the coordinate system along corner 4a. Specifically, the vertex of corner 4a is specified as point P1, i.e., the origin of the coordinate system. Furthermore, positions on two sides connected to the vertex of corner 4a are specified as points P2 and P3. As a result, the direction from point P1 toward point P2 and the direction from point P1 toward point P3 are specified as the two horizontal axis directions of the coordinate system.

[0045] Note that only one of the two horizontal axis directions may be specified. For example, user 2-1 may specify only point P2 out of points P2 and P3. This is because once the axis direction from point P1 corresponding to the vertex of corner 4a to point P2 on one side is determined, the axis direction from point P1 to point P3 on the other side is uniquely determined and can be automatically specified. Hereinafter, it is assumed that user 2-1 specifies only two points, points P1 and P2.

[0046] The vertical 1 axis direction is detected by a gravity sensor (an example of the sensor unit 51) of the terminal device 5 and is automatically specified. However, the user 2-1 may specify it manually. Hereinafter, it is assumed that the vertical 1 axis direction is specified automatically.

[0047] The user 2-1 designates points P1 and P2 by touching the positions corresponding to points P1 and P2 directly (for example, with a finger) or by touching them with a controller. In this example, a controller 54 is used. The controller 54 may be a controller for the terminal device 5, and may have a shape that tapers to a tip, such as a pen shape. Any point within the space 3, particularly a narrow place such as the apex of a corner 4a located in the corner of a room, can be easily touched.

[0048] With the controller 54 or a finger touching the positions corresponding to points P1 and P2, an operation (decision operation) for specifying the origin and axis direction is performed. Examples of the decision operation include a button operation on the controller 54, a gesture operation, etc. Examples of the gesture operation include the movement of the user 2-1's finger (a finger other than the finger touching point P1, etc.), eye movement, etc.

[0049] The terminal device 5-1 displays information for causing the user 2-1 to perform the first user operation described above. This will be explained with reference to FIGS.

[0050] 6 to 8 are diagrams showing examples of information display. To the user 2-1, the information displayed on the display unit 52 of the terminal device 5-1 appears to overlap with the space 3. Note that the illustration of the user 2-1 (hands, etc.) holding the controller 54 is omitted.

[0051] As shown in Fig. 6, a message M1 is displayed. In response to the instruction "Please specify the origin" in the message M1, the user 2-1 specifies the origin of the coordinate system. Specifically, the user 2-1 touches the vertex of the corner 4a with the tip of the controller 54. This point is specified as point P1, i.e., the origin of the coordinate system.

[0052] As shown in FIG. 7, a message M2 is displayed. Following the instruction "Please specify a direction" in the message M2, the user 2-1 specifies the axis direction of the coordinate system. Specifically, the user 2-1 touches a point on the edge of the corner 4a with the tip of the controller 54. This point is specified as point P2. The direction from point P1 to point P2 is specified as one of the two horizontal axis directions.

[0053] As mentioned above, point P3 is automatically specified, and the direction from point P1 to point P3 is specified as the other of the two horizontal axis directions. The vertical axis direction is also automatically specified.

[0054] As shown in Figure 8, message M3 is displayed. In response to the message M3 saying "Please share the specified information with other users," user 2-1 conveys the information about points P1, P2, etc. that he or she specified to user 2-2. As mentioned earlier, user 2-1 and user 2-2 are capable of face-to-face communication, and information sharing can be easily performed.

[0055] <Second user operation> Next, user 2-2 performs a second user operation. Referring again to Fig. 5, this time user 2-2 specifies the origin and axis directions of the coordinate system along corner 4a. The origin and axis directions specified by user 2-2 here are the same as those specified by user 2-1 earlier.

[0056] The display unit 52 of the terminal device 5-2 displays information for causing the user 2-2 to perform the second user operation described above. Referring again to FIGS. 6 and 7, in accordance with message M1, the user 2-2 specifies the origin of the coordinate system. Specifically, the user 2-2 touches the vertex of the corner 4a with the tip of the controller 54. This point is specified as point P1, i.e., the origin of the coordinate system. In accordance with message M2, the user 2-2 specifies the axis directions of the coordinate system. Specifically, the user 2-2 touches a point on the side of the corner 4a with the tip of the controller 54. This point is specified as point P2. The direction from point P1 toward point P2 is specified as one of two horizontal axis directions. The other of the two horizontal axis directions and one vertical axis direction are automatically specified.

[0057] <Coordinate system matching> 9 is a diagram showing an example of the coincidence of coordinate systems. Based on the results of the first user operation and the second user operation, each of the terminal device 5-1 and the terminal device 5-2 executes a process for matching the coordinate system C1 and the coordinate system C2 (for coordinate alignment).

[0058] An example of a specific process is coordinate transformation. The terminal device 5-1 and the terminal device 5-2 transform the coordinate system C1 and the coordinate system C2 into the same coordinate system. For example, the terminal device 5-1 transforms the coordinate system C1 into coordinates specified by a first user operation (coordinates having an origin and axis directions specified by the user 2-1). The terminal device 5-2 transforms the coordinate system C2 into coordinates specified by a second user operation (coordinates having an origin and axis directions specified by the user 2-2). Various known operations (matrix operations, etc.) may be used, and necessary parameter settings may also be performed. After the transformation, the coordinate system C1 and the coordinate system C2 coincide with each other and become the coordinate system C.

[0059] The coordinate system C1 after the above-described coordinate transformation may be used as a new coordinate system C1 of the terminal device 5-1. It can also be said that the coordinate system C1 is reset or updated. Alternatively, it can be said that the coordinate system C1 moves to a coordinate system having an origin and axis directions specified by the user 2-1. The same applies to the coordinate system C2.

[0060] Users 2-1 and 2-2 in the same space 3 specify coordinate systems within the space 3, and based on the results, the coordinate system C1 of the terminal device 5-1 and the coordinate system C2 of the terminal device 5-2 can be accurately matched. As a comparative example, a method is also conceivable in which users 2-1 and 2-2 move to specific locations in the space 3 in turn and perform a confirmation operation (such as pressing and holding a button), specifying the positions and orientations of users 2-1 and 2-2 at that time as the origin and axis directions of the coordinate systems. However, with this comparative example method, it is difficult for users 2-1 and 2-2 to perform the confirmation operation while facing the exact same position and direction within the space 3, and therefore it is difficult to accurately match the coordinate systems. According to the information processing system 1 according to the embodiment, users 2-1 and 2-2 can easily and accurately specify the origin and axis directions of the same coordinate system, thereby significantly improving the accuracy of the match between the coordinate systems.

[0061] Since the coordinate system C1 and the coordinate system C2 are the same, the position of the virtual object 4v in the space 3 displayed by the terminal device 5-1 is the same as the position of the virtual object 4v in the space 3 displayed by the terminal device 5-2. The users 2-1 and 2-2 can recognize the same virtual object 4v at the same location in the space 3.

[0062] 10 is a flowchart showing an example of processing (information processing method) executed in the information processing system 1. Description of content that overlaps with the above will be omitted where appropriate.

[0063] In step S1, the terminal device 5-1 displays information for prompting the user 2-1 to perform a first user operation. As shown in step S1a, the user 2-1 specifies the origin and axis directions of the coordinate system along the corner 4a. For example, the user 2-1 touches a specific position in the space 3 (a position corresponding to point P1 or point P2) directly or with the controller 54.

[0064] In step S2, the terminal device 5-2 displays information for prompting the user 2-2 to perform a second user operation. As shown in step S2a, the user 2-2 specifies the same origin and axial direction along the corner 4a as those specified by the user 2-1. For example, the user 2-2 touches the same position as that touched by the user 2-1, either directly or with the controller 54.

[0065] In step S3, the terminal device 5-1 and the terminal device 5-2 each match the coordinates of the coordinate system C1 and the coordinate system C2 based on the results of the first user operation and the second user operation. For example, the terminal device 5-1 converts the coordinate system C1 into a coordinate system having an origin and axis directions specified by the user 2-1. The terminal device 5-2 converts the coordinate system C2 into a coordinate system having an origin and axis directions specified by the user 2-2.

[0066] Each of the terminal device 5-1 and the terminal device 5-2 displays the virtual object 4v based on the matched coordinate systems C1 and C2, i.e., the coordinate system C. The user 2-1 and the user 2-2 can recognize the same object 4 at the same location in the space 3.

[0067] Note that in the above steps S1, S2, and S3, data exchange between the terminal device 5-1 and the terminal device 5-2 is not required. If some data exchange were to be performed between the terminal device 5-1 and the terminal device 5-2 to align the coordinate system C1 and the coordinate system C2, there is a possibility that important information such as accounts would be exchanged during the data exchange, and privacy information would not be protected. According to the above method, the coordinate system C1 and the coordinate system C2 can be aligned without data exchange between the terminal device 5-1 and the terminal device 5-2. This also provides the advantage of protecting privacy information. It also reduces communication loads, etc.

[0068] 1.2 Second Method FIG. 11 is a diagram illustrating an example of the second technique. The second technique also uses a corner 4a that exists in the space 3. In the example shown in FIG. 11, the corner 4a is a corner of a desk (an example of an object 4). Unlike the first technique, the users 2-1 and 2-2 do not need to touch a specific position (a position corresponding to point P1, etc.) in the space 3.

[0069] <First User Operation and Second User Operation> Each of the terminal device 5-1 and the terminal device 5-2 extracts a plurality of corners 4a present in the space 3 and displays them in a selectable manner. Various known image recognition processing techniques may be used. Each of the terminal device 5-1 and the terminal device 5-2 also extracts an origin and an axial direction that can be specified along the corners 4a (point extraction, line extraction) and displays them in a selectable manner. For the extraction, for example, shape recognition processing based on data from a LiDAR sensor (an example of the sensor unit 51) may be used.

[0070] User 2-1 specifies the origin and axis directions of the coordinate system along one corner 4a selected from the multiple corners 4a. User 2-2 specifies the same origin and axis directions as those specified by user 2-1 along the corner 4a selected by user 2-1.

[0071] The terminal device 5-1 displays information for causing the user 2-1 to perform the first user operation. The terminal device 5-2 displays information for causing the user 2-2 to perform the second user operation. This will be described with reference to Figs. 12 and 13.

[0072] 12 and 13 are diagrams showing examples of information display. In this example, the four corners 4a of the desk are displayed (highlighted) in a selectable manner. The origin and axis directions that can be specified along these corners 4a are also displayed in a selectable manner. Corners located in places other than the desk may also be displayed in a selectable manner.

[0073] As shown in Fig. 12, a message M4 is displayed. According to the instruction "Please specify the origin and direction" in the message M4, the user 2-1 and the user 2-2 each specify the origin and the axis direction. This specification is performed by the user 2-1 and the user 2-2 operating the terminal device 5-1 and the terminal device 5-2. The specific operation method is not particularly limited, and various operations such as button operation, controller operation, voice operation, gesture operation, etc. may be performed.

[0074] Specifically, each of user 2-1 and user 2-2 selects one of the multiple corners 4a and designates the vertex of that corner 4a as point P1, i.e., the origin of the coordinate system. Furthermore, one of the two sides of corner 4a is designated as one of two horizontal axis directions. At this time, user 2-1 and user 2-2 select the same corner 4a and designate the same origin and axis directions through FtoF communication. The other of the two horizontal axis directions and the vertical axis direction are automatically designated.

[0075] As shown in Figure 13, message M5 is displayed. In response to the question "Is it the same as other users?" in message M5, if users 2-1 and 2-2 have specified the same origin and axis directions, they select OK to complete the specification. If not (if they are different), they select Cancel and specify again. Also, if the directions of the two horizontal axes have been specified in reverse, selecting "Reverse axis directions" will re-specify them so that their relationship is reversed. The specified axis directions can be easily corrected.

[0076] <Coordinate system matching> Based on the results of the first and second user operations, the terminal device 5-1 and the terminal device 5-2 respectively match the coordinate system C1 and the coordinate system C2. An example of this processing is the coordinate transformation described above.

[0077] 14 is a diagram showing an example of the coordinate systems after matching. The position of the virtual object 4v in the space 3 displayed by the terminal device 5-1 matches the position of the virtual object 4v in the space 3 displayed by the terminal device 5-2. The users 2-1 and 2-2 can recognize the same virtual object 4v at the same location in the space 3.

[0078] 15 is a flowchart showing an example of processing (information processing method) executed in the information processing system 1. Description of content that overlaps with the above will be omitted where appropriate.

[0079] In step S11, the terminal device 5-1 displays information for prompting the user 2-1 to perform a first user operation. Furthermore, the terminal device 5-2 displays information for prompting the user 2-2 to perform a second user operation. For example, each of the terminal devices 5-1 and 5-2 displays a plurality of corners 4a existing in the space 3 in a selectable manner, and further displays an origin and axis directions that can be specified along the corners 4a in a selectable manner. As shown in step S11a, the users 2-1 and 2-2 specify the origin and axis directions of the same coordinate system along the corner 4a. For example, the user 2-1 specifies the origin and axis directions of the coordinate system along one corner 4a selected from the plurality of corners 4a. The user 2-2 specifies the same origin and axis directions as those specified by the user 2-1 along the same corner 4a as the corner 4a selected by the user 2-1.

[0080] In step S12, the terminal device 5-1 and the terminal device 5-2 match the coordinates of the coordinate system C1 and the coordinate system C2.

[0081] Each of the terminal device 5-1 and the terminal device 5-2 displays the virtual object 4v based on the matched coordinate systems C1 and C2, i.e., the coordinate system C. The user 2-1 and the user 2-2 can recognize the same object 4 at the same location in the space 3.

[0082] In addition, in the above steps S11 and S12, data exchange between the terminal device 5-1 and the terminal device 5-2 does not have to be performed.

[0083] 1.3 Third Method 16 to 18 are diagrams illustrating an example of the third technique. In the third technique, a virtual image (CG) having a shape that fits the space 3 is used. This image is referred to as a virtual space image 3v. In the virtual space image 3v, occlusion in the space 3 is reproduced. The occlusion may occur, for example, due to objects 4 that exist within the space 3, particularly objects 4 (walls, floors, ceilings, etc.) that define the shape of the space 3. The portion in the virtual space image 3v that corresponds to such an object 4 is illustrated and referred to as an occlusion portion 4o.

[0084] An origin is also set in the virtual space image 3v. This origin is referred to as the origin 3vp and is illustrated. In this example, the vertex of the corner of the virtual space image 3v is set as the origin 3vp. Two directions extending from the origin 3vp along the two sides of the corner are automatically determined as two horizontal axis directions.

[0085] Each of terminal device 5-1 and terminal device 5-2 displays a virtual space image 3v. Terminal device 5-1 displays the virtual space image 3v so that user 2-1 can move the virtual space image 3v. Terminal device 5-2 displays the virtual space image 3v so that user 2-2 can move the virtual space image 3v. Moving the virtual space image 3v may include, for example, moving the virtual space image 3v on the screen up and down, left and right, or rotating it.

[0086] <First User Operation and Second User Operation> As shown in FIG. 16, initially, the virtual space image 3v displayed by the terminal device 5-1 does not match the space 3. Each of the users 2-1 and 2-2 matches the virtual space image 3v to the space 3. For example, each of the users 2-1 and 2-2 moves within the space 3 or moves the displayed virtual space image 3v so that the occlusion portion 4o in the virtual space image 3v matches the corresponding object 4 in the space 3. As shown in FIG. 17, the virtual space image 3v matches the space 3.

[0087] As shown in Fig. 18, with the virtual space image 3v coinciding with the space 3, the origin 3vp of the virtual space image 3v is designated as the origin of the coordinate system. Furthermore, two directions extending from the origin 3vp are designated as two horizontal axis directions. The vertical axis direction is automatically designated. In this way, the origin and axis directions of the coordinate system are designated.

[0088] Terminal device 5-1 displays information for causing user 2-1 to perform the first user operation. Terminal device 5-2 displays information for causing user 2-2 to perform the second user operation. This will be described with reference to FIGS. 19 to 22. In this description, users 2-1 and 2-2 will not be distinguished from each other and will simply be referred to as user 2. Terminal device 5-1 and terminal device 5-2 will not be distinguished from each other and will simply be referred to as terminal device 5.

[0089] 19 to 22 are diagrams showing examples of information display. From the user 2's perspective, the information displayed by the display unit 52 of the terminal device 5, i.e., the virtual space image 3v (corresponding to the hatched portion), appears to overlap with the space 3. An occlusion portion 4o corresponding to the object 4 causing the occlusion is also displayed. In this example, the virtual space image 3v can be moved by the virtualized user 2's hand touching the virtual space image 3v.

[0090] 19, initially, the virtual space image 3v does not match the space 3, and the object 4 and the corresponding occlusion portion 4o do not match either. As shown in FIGS. 21 and 22, the user 2 moves within the space 3 or moves the virtual space image 3v, thereby matching the occlusion portion 4o with the object 4. In this way, the virtual space image 3v matches the space 3.

[0091] In order to make it easier to match the occlusion portion 4o with the object 4, when they match, the matched portion may be highlighted by changing the color of the matched portion, for example. Data from a LiDAR sensor (an example of the sensor unit 51), for example, may be used to determine whether the occlusion portion 4o and the object 4 match.

[0092] Since the user 2-1 and the user 2-2 match the same virtual space image 3v to the same space 3, it is possible to specify the origin and axis direction of the same coordinate system. In particular, as the accuracy of the virtual space image 3v (of the occlusion part 4o) increases, the accuracy of the match also increases. The origin and axis direction of the coordinate system can be precisely matched.

[0093] <Coordinate system matching> Based on the results of the first and second user operations, the terminal device 5-1 and the terminal device 5-2 respectively match the coordinate system C1 and the coordinate system C2. An example of this processing is the coordinate transformation described above.

[0094] 23 is a flowchart showing an example of processing (information processing method) executed in the information processing system 1. Description of content that overlaps with the above will be omitted where appropriate.

[0095] In step S21, the terminal device 5-1 displays information for causing the user 2-1 to perform a first user operation. The terminal device 5-2 displays information for causing the user 2-2 to perform a second user operation. For example, each of the terminal devices 5-1 and 5-2 displays a virtual space image 3v in which the occlusion of the space 3 is reproduced. As shown in step S21a, the users 2-1 and 2-2 specify the origin and axis directions of the same coordinate system by matching the virtual space image 3v with the space 3. For example, each of the users 2-1 and 2-2 moves within the space 3 or moves the virtual space image 3v.

[0096] In step S22, the terminal device 5-1 and the terminal device 5-2 match the coordinates of the coordinate system C1 and the coordinate system C2.

[0097] Each of the terminal device 5-1 and the terminal device 5-2 displays the virtual object 4v based on the matched coordinate systems C1 and C2, i.e., the coordinate system C. The user 2-1 and the user 2-2 can recognize the same object 4 at the same location in the space 3.

[0098] In addition, in the above steps S21 and S22, data exchange between the terminal device 5-1 and the terminal device 5-2 does not have to be performed.

[0099] 1.4 The fourth method 24 is a diagram showing an example of the fourth method. In the fourth method, each of the user 2-1 and the user 2-2 sets (attaches, fastens, etc.) a controller 56 on a positioning device 55 arranged in the space 3, thereby specifying the origin and axis directions of the same coordinate system. The controller 56 and the positioning device 55 may be the controller of the terminal device 5 and its charging device (also referred to as a charging dock).

[0100] Each of user 2-1 and user 2-2 has a controller 56. More specifically, in this example, the controllers 56 are a pair of controllers. One of the controllers is illustrated as controller 56L. The other controller is illustrated as controller 56R. Each of user 2-1 and user 2-2 holds and uses controller 56 in their left hand and controller 56R in their right hand.

[0101] <First user operation> The user 2-1 sets the controller 56 to be used by the user 2-1 in the positioning device 55. As an example, it is assumed that the controller 56R of the controller 56L and the controller 56R is set.

[0102] The position and orientation of the controller 56R set in the positioning device 55 are fixed relative to the positioning device 55. This position and orientation are defined by, for example, magnets attached to the positioning device 55 and the controller 56R.

[0103] A predetermined position (the position of the tip, center, or any other part) of the controller 56R set in the positioning device 55 is designated as point P1, i.e., the origin of the coordinate system. Two directions along the controller 56R are designated as two horizontal axis directions of the coordinate system. The origin and axis directions designated along the controller 56R are extracted and identified, for example, by image recognition processing of an image including the controller 56R. The vertical axis direction is designated automatically.

[0104] It should be noted that there may be two mutually opposite directions along the controller 56R, and these may be displayed in a selectable manner, for example.

[0105] The terminal device 5-1 displays information for causing the user 2-1 to perform the first user operation described above. This will be described with reference to FIGS.

[0106] 25 to 27 are diagrams showing examples of information displays, in which a positioning device 55 is placed on a desk.

[0107] 25, a message M6 is displayed. The user 2-1 sets the controller 56R in the positioning device 55 in accordance with the instruction "Please set the controller 56R" in the message M6.

[0108] As shown in FIG. 26, a message M7 is displayed. Following the instruction in message M7 to "operate controller 56L," user 2-1 operates controller 56L. The operation of controller 56L here is an operation for specifying the origin and two horizontal axis directions of the coordinate system along controller 56R at this time. By performing such an operation using controller 56L instead of controller 56R, it is possible to specify the origin and axis directions of the coordinate system while preventing the controller 56R set in positioning device 55 from moving.

[0109] As shown in FIG. 27, the origin of the coordinate system (corresponding to point P1) and axial directions (directions from point P1 toward points P2 and P3) are specified along the controller 56R. Then, a message M8 is displayed. In response to the instruction "Please remove the controller 56R" in the message M8, the user 2-1 removes the controller 56R from the positioning device 55. Note that, for example, the next time the user 2-2 performs a similar operation, a message may be displayed to warn the user not to move the positioning device 55 from its current position and orientation.

[0110] <Second user operation> User 2-2 performs a similar operation. User 2-2 sets controller 56R to be used by user 2-2 in positioning device 55 and operates controller 56L. The origin and two horizontal axis directions of the coordinate system are specified along controller 56R set in positioning device 55. The vertical axis direction is automatically specified.

[0111] <Coordinate system matching> Based on the results of the first and second user operations, the terminal device 5-1 and the terminal device 5-2 respectively match the coordinate system C1 and the coordinate system C2. An example of this processing is the coordinate transformation described above.

[0112] <Example of another configuration of the controller 56> When the user 2 holds the controller 56 in his / her hand, it may be difficult to perform another task with the same hand, and the controller 56 may fall when the user lets go of the controller 56. To address these problems, a controller 56 having a different configuration (shape, etc.) from the above may be used. This will be described with reference to FIGS. 28 to 30.

[0113] 28 to 30 are diagrams illustrating an example of a schematic configuration of the controller 56. The controller 56 has a ring portion through which the user 2's finger is inserted, an operation portion (e.g., a slide pad) that the user 2's fingertip touches, and a holding portion located on the opposite side of the ring portion from the holding portion. The user 2 uses the controller 56 by inserting their finger through the ring portion and holding the holding portion. Because the controller 56 does not need to be held in the hand, the user can use the same hand to perform another task, such as operating a device other than the terminal device 5 (e.g., operating a keyboard on a personal computer). The risk of dropping the controller 56 is also significantly reduced. For more specific details of this controller 56, see Patent Document 2 (International Publication No. 2024 / 090304).

[0114] The above-mentioned controllers 56 may also be a pair of controllers corresponding to the left and right hands of the user 2. Furthermore, since there may be two mutually opposite directions along the controller 56, these may be displayed in a manner that allows selection when specifying the axis directions of the coordinate system.

[0115] 29 and 30 show a state in which the controller 56 is set in the positioning device 55. As before, the position and direction of the controller 56 are specified as the origin of the coordinate system (corresponding to point P1) and two horizontal axis directions (directions from point P1 toward points P2 and P3).

[0116] 31 is a flowchart showing an example of processing (information processing method) executed in the information processing system 1. Description of content that overlaps with the above will be omitted where appropriate.

[0117] In step S31, the terminal device 5-1 displays information for prompting the user 2-1 to perform a first user operation. As shown in step S31a, the user 2-1 sets the controller 56 used by the user 2-1 in the positioning device 55, thereby specifying the origin and axis directions of the coordinate system. The origin and axis directions of the coordinate system are specified along with the controller 56 set in the positioning device 55.

[0118] In step S32, the terminal device 5-2 displays information for allowing the user 2-2 to perform a second user operation. As shown in step S32a, the user 2-2 sets the controller 56 used by the user 2-2 in the positioning device 55, thereby specifying the same origin and axis directions as those specified by the user 2-1. The origin and axis directions of the coordinate system are specified along the controller 56 set in the positioning device 55.

[0119] In step S33, the terminal device 5-1 and the terminal device 5-2 match the coordinates of the coordinate system C1 and the coordinate system C2.

[0120] Each of the terminal device 5-1 and the terminal device 5-2 displays the virtual object 4v based on the matched coordinate systems C1 and C2, i.e., the coordinate system C. The user 2-1 and the user 2-2 can recognize the same object 4 at the same location in the space 3.

[0121] In addition, in the above steps S31, S32 and S33, data exchange between the terminal device 5-1 and the terminal device 5-2 does not have to be performed.

[0122] 2. Variations The disclosed technology is not limited to the above-described embodiment, and some modifications will be described.

[0123] In the above embodiment, two users 2, user 2-1 and user 2-2, are described as users 2 in the space 3. However, there may be three or more users 2 in the space 3. The same applies to the terminal device 5.

[0124] In one embodiment, another user from outside the space 3 may participate (remotely participate) in the recognition of the virtual object 4v by the user 2 in the space 3. This will be described with reference to FIG.

[0125] FIG. 32 is a diagram showing an example of remote participation. Another user outside the space 3 is illustrated and referred to as user 7. The information processing system 1 further includes a VR device 8 (an HMD-type device in this example). By using the VR device 8, user 7 can view a virtual object 4v in the space 3 together with users 2-1 and 2-2. The VR device 8 displays both an image of the space 3 (which may include users 2-1 and 2-2) and the virtual object 4v. Information required to generate the image in the space 3 may be prepared in advance or acquired in real time.

[0126] Terminal device 5-1 and terminal device 5-2 display not only virtual object 4v but also virtualized user 7. Virtualized user 7 is referred to as avatar 7v and illustrated. User 7 can move within space 3 as avatar 7v and view virtual object 4v from a viewpoint from a desired position. The position of avatar 7v within space 3 is also shared by server device 6. Users 2-1 and 2-2 and user 7 can recognize virtual object 4v in the same location within space 3, with a sense of realism that transcends space and makes them feel as if they are in the same place.

[0127] 3. Hardware configuration example 33 is a diagram illustrating an example of the hardware configuration of an apparatus. The terminal apparatus 5 and the like according to the embodiments of the present disclosure described above are realized by, for example, a computer 1000 configured as shown in the figure. The computer 1000 includes a processing circuitry 1100, a RAM 1200, a ROM 1300, a secondary storage device 1400, a communication interface 1500, an input / output interface 1600, a display unit 1700, a camera unit 1800, a microphone 1900, and a speaker 2000. The components of the computer 1000 are connected by a bus 1050.

[0128] The processing circuit 1100 operates and controls each unit based on programs stored in the ROM 1300 or the secondary storage device 1400. For example, the processing circuit 1100 loads the programs stored in the ROM 1300 or the secondary storage device 1400 into the RAM 1200 and executes processing corresponding to the various programs.

[0129] The ROM 1300 stores boot programs such as a BIOS (Basic Input Output System) that is executed by the processing circuit 1100 when the computer 1000 is started up, and programs that depend on the hardware of the computer 1000 .

[0130] The secondary storage device 1400 is a computer-readable recording medium that non-temporarily records programs executed by the processing circuit 1100 and data used by such programs. Specifically, the secondary storage device 1400 is a recording medium that records programs (e.g., information processing program 500) for each process of the terminal device 5 according to an embodiment of the present disclosure, which are examples of program data 1450.

[0131] The communication interface 1500 is an interface for connecting the computer 1000 to an external network 1550. For example, the processing circuit 1100 receives data from other devices and transmits data generated by the processing circuit 1100 to other devices via the communication interface 1500.

[0132] The input / output interface 1600 is an interface for connecting the input / output device 1650 and the computer 1000. For example, the processing circuit 1100 receives data from an input device such as a microphone 1900 or a touch panel via the input / output interface 1600. The processing circuit 1100 also transmits data to an output device such as a display unit 1700 or a speaker 2000 via the input / output interface 1600. The input / output interface 1600 may also function as a media interface for reading programs and the like recorded on a predetermined recording medium. Examples of media include optical recording media such as DVDs (Digital Versatile Discs) and PDs (Phase Change Rewritable Disks), magneto-optical recording media such as MOs (Magneto-Optical Disks), tape media, magnetic recording media, and semiconductor memories.

[0133] The display unit 1700 is an interface for displaying information processed by the computer 1000. The display unit 1700 is, for example, a liquid crystal display or an organic electroluminescence display (OLED). The display unit 1700 may also be a touch panel display device or a video projection device. The display unit 1700 may correspond to the display unit 52 (FIG. 2) of the terminal device 5.

[0134] The camera unit 1800 is an interface through which the computer 1000 captures images. The microphone 1900 is an interface through which the computer 1000 captures audio. These correspond to an example of the sensor unit 51 ( FIG. 2 ) of the terminal device 5. The speaker 2000 is an interface through which the computer 1000 outputs audio processed by the computer 1000. The components of the computer 1000 are connected by a bus 1050. The interfaces do not necessarily need to be provided inside the computer 1000, but may be provided outside the computer 1000 via a network or the like. Furthermore, the components constituting the computer 1000 may be controlled by a circuit different from the processing circuit 1100. For example, the display unit 1700 may be controlled not by the processing circuit 1100 but by a circuit dedicated to display processing provided in the display unit 1700.

[0135] For example, when the computer 1000 functions as the terminal device 5 according to an embodiment of the present disclosure, the processing circuit 1100 of the computer 1000 functions as a processing unit 53 ( FIG. 2 ) by executing a program loaded onto the RAM 1200. The secondary storage device 1400 stores an information processing program 500 according to the present disclosure and various data stored in the storage unit 50. The processing circuit 1100 reads and executes program data 1450 from the secondary storage device 1400. Alternatively, the processing circuit 1100 may acquire these programs from another device via an external network 1550. That is, the secondary storage device 1400 need not be located inside the computer 1000, but may be located outside the computer 1000. The processing circuit 1100 is an example of an integrated circuit, and a CPU, an MPU, a GPU, an APU, an ASIC, and an FPGA can all be considered to be integrated circuits.

[0136] 4. Summary The above-described techniques are specified, for example, as follows: One of the techniques disclosed is an information processing method. As described with reference to Figures 1, 2, and 5 to 32, the information processing method includes the steps of: displaying, by a terminal device 5-1 (first terminal device) used by a user 2-1, information for causing a user 2-1 (first user) in the space 3 to perform a first user operation to specify a coordinate system within the space 3 (specific space) (steps S1, S11, S21, S31); displaying, by a terminal device 5-2 (second terminal device) used by a user 2-2, information for causing a user 2-2 (second user) in the space 3 to perform a second user operation to specify a coordinate system within the space 3 (steps S2, S11, S21, S32); and matching the coordinate system C1 held by the terminal device 5-1 with the coordinate system C2 held by the terminal device 5-2 based on the results of the first user operation and the second user operation (steps S3, S12, S22, S33).

[0137] According to the information processing system 1, the users 2-1 and 2-2 who are in the same space 3 specify the coordinate system within the space 3, which increases the possibility of specifying an appropriate coordinate system. Based on this result, the coordinate system C1 of the terminal device 5-1 and the coordinate system C2 of the terminal device 5-2 can be matched with high precision.

[0138] 5 to 7, 10 to 13, 15 to 31, etc., the first user's operation may include user 2-1 specifying an origin and axis directions of a coordinate system in space 3 (steps S1a, S11a, S21a, and S31a), and the second user's operation may include user 2-2 specifying, in space 3, an origin and axis directions that are the same as those specified by user 2-1 (steps S2a, S11a, S21a, and S32a). Based on the same coordinate systems specified by the first user's operation and the second user's operation, coordinate systems C1 and C2 can be matched with high precision.

[0139] As described with reference to Figures 9, 10, 14, 15, 23, and 31, etc., matching the coordinate systems (steps S3, S12, S22, and S33) may include converting the coordinate system C1 of the terminal device 5-1 into a coordinate system having an origin and axis directions specified by the user 2-1, and converting the coordinate system C2 of the terminal device 5-2 into a coordinate system having an origin and axis directions specified by the user 2-2. For example, by such coordinate conversion, the coordinate systems C1 and C2 can be made to match.

[0140] 5 to 8, 10 to 13, 15, etc., the first user operation may include user 2-1 specifying an origin and axis directions of a coordinate system along a corner 4a that exists in space 3 (steps S1a and S11a), and the second user operation may include user 2-2 specifying an origin and axis directions that are the same as the origin and axis directions specified by user 2-1 along the corner 4a (steps S2a and S11a). As described with reference to FIGS. 5 to 8, 10, etc., the specification in the first user operation may include user 2-1 touching a specific position in space 3 directly or with the controller 54 (step S1a), and the specification in the second user operation may include user 2-2 touching the same position touched by user 2-1 directly or with the controller 54 (step S2a). Alternatively, as described with reference to FIGS. 11 to 13 and 15, the information processing method may further include each of the terminal devices 5-1 and 5-2 displaying a plurality of corners 4a present in the space 3 in a selectable manner (step S11), the first user operation including the user 2-1 specifying an origin and axis directions of a coordinate system along one corner 4a selected from the plurality of corners 4a (step S11a), and the second user operation including the user 2-2 specifying an origin and axis directions identical to the origin and axis directions specified by the user 2-1 along the same corner 4a as the corner 4a selected by the user 2-1. In this case, each of the terminal devices 5-1 and 5-2 may also display the origin and axis directions that can be specified along the corner 4a in a selectable manner. For example, the origin and axis directions of the coordinate system can be specified using the corner 4a in this manner.

[0141] As described with reference to FIGS. 16 to 23, the information processing method further includes each of the terminal device 5-1 and the terminal device 5-2 displaying a virtual space image 3v in which the occlusion of the space 3 is reproduced (step S21), and the first user operation may include the user 2-1 matching the virtual space image 3v with the space 3 (step S21a), and the second user operation may include the user 2-2 matching the virtual space image 3v with the space 3 (step S21a). In this case, for example, the first user operation may include the user 2-1 moving within the space 3 and moving the virtual space image 3v (step S21a), and the second user operation may include the user 2-2 moving within the space 3 and moving the virtual space image 3v (step S21a). For example, a coordinate system may be specified using the virtual space image 3v for occlusion in this manner.

[0142] As described with reference to FIGS. 24 to 31, the first user operation may include the user 2-1 setting the controller 56 in the positioning device 55 arranged in the space 3 (step S31a), and the second user operation may include the user 2-2 setting the controller 56 in the positioning device 55 (step S32a). In this case, for example, the position and orientation of the controller 56 set in the positioning device 55 with respect to the positioning device 55 may be predetermined. For example, a coordinate system may be specified using such a positioning device 55 and controller 56.

[0143] 1 and the like, the space 3 may have a size that allows the user 2-1 and the user 2-2 to communicate with each other face-to-face, which further increases the possibility that the user 2-1 and the user 2-2 can specify an appropriate coordinate system.

[0144] As described with reference to Figures 10, 15, 23, and 31, the information processing method does not need to include data exchange between the terminal device 5-1 and the terminal device 5-2. Advantages such as protection of privacy information can also be obtained.

[0145] The information processing system 1 described with reference to FIGS. 1, 2, and 5 to 32 is also one of the disclosed technologies. The information processing system 1 includes a display unit 52 that displays information for causing a user 2-1 (first user) in the space 3 to perform a first user operation for specifying a coordinate system within the space 3 (specific space) and information for causing a user 2-2 (second user) in the space 3 to perform a second user operation for specifying a coordinate system within the space 3, and a processing unit 53 that matches a coordinate system C1 of a terminal device 5-1 (first terminal device) used by the user 2-1 with a coordinate system C2 of a terminal device 5-2 (second terminal device) used by the user 2-2, based on the results of the first user operation and the second user operation. As described above, the information processing system 1 described above can also match the coordinate system C1 of the terminal device 5-1 with the coordinate system C2 of the terminal device 5-2 with high accuracy.

[0146] The effects described in this disclosure are merely examples and are not limited to the disclosed contents. Other effects may also be obtained.

[0147] Although the embodiments of the present disclosure have been described above, the technical scope of the present disclosure is not limited to the above-described embodiments, and various modifications are possible within the scope of the gist of the present disclosure. Furthermore, components of different embodiments and modifications may be combined as appropriate.

[0148] The present technology can also be configured as follows. (1) displaying, by a first terminal device used by a first user, information for causing a first user in a specific space to perform a first user operation for specifying a coordinate system within the space; displaying, by a second terminal device used by a second user, information for causing a second user in the space to perform a second user operation for specifying a coordinate system within the space; Aligning a coordinate system of the first terminal device with a coordinate system of the second terminal device based on results of the first user operation and the second user operation; Including, Information processing methods. (2) the operation of the first user includes the first user specifying an origin and an axis direction of a coordinate system in the space; the operation of the second user includes the second user specifying, in the space, the same origin and axis direction as those specified by the first user; The information processing method described in (1). (3) The matching of the coordinate systems may include: converting a coordinate system of the first terminal device into a coordinate system having an origin and axis directions designated by the first user; converting a coordinate system of the second terminal device into a coordinate system having an origin and axis directions designated by the second user; Including, (2) The information processing method described above. (4) the first user operation includes the first user specifying an origin and an axis direction of a coordinate system along a corner existing in the space; the second user operation includes the second user specifying, along the corner portion, an origin and an axial direction that are the same as the origin and the axial direction specified by the first user; The information processing method according to any one of (1) to (3). (5) the designation in the first user operation includes the first user touching a specific position in the space directly or with a controller; the designation by the second user operation includes the second user touching the same position as the position touched by the first user directly or with a controller; (4) The information processing method described in (4). (6) The method further includes each of the first terminal device and the second terminal device displaying a plurality of corners present in the space in a selectable manner, the first user operation includes the first user specifying an origin and an axis direction of a coordinate system along one corner selected from the plurality of corners; the second user operation includes the second user specifying, along the same corner as the corner selected by the first user, an origin and an axial direction that are the same as the origin and the axial direction that are specified by the first user; The information processing method according to any one of (1) to (4). (7) Each of the first terminal device and the second terminal device displays an origin and an axis direction that can be specified along the corner portion in a selectable manner. (6) The information processing method described in (6). (8) The method further includes each of the first terminal device and the second terminal device displaying a virtual space image in which occlusion of the space is reproduced; the first user operation includes the first user matching the virtual space image to the space; the second user operation includes the second user matching the virtual space image with the space; The information processing method according to any one of (1) to (3). (9) the first user operation includes the first user moving within the space and moving the virtual space image; the second user operation includes the second user moving within the space and moving the virtual space image; (8) The information processing method described in (8). (10) the first user operation includes the first user setting a controller on a positioning device arranged in the space; the second user operation includes the second user setting a controller on the positioning device; The information processing method according to any one of (1) to (3). (11) The position and orientation of the controller set on the positioning device relative to the positioning device are determined. (10) The information processing method described in (10). (12) The space has a size that allows the first user and the second user to communicate with each other face to face. The information processing method according to any one of (1) to (11). (13) does not include data exchange between the first terminal device and the second terminal device; The information processing method according to any one of (1) to (12). (14) a display unit that displays information for causing a first user in a specific space to perform a first user operation for specifying a coordinate system within the space, and information for causing a second user in the space to perform a second user operation for specifying a coordinate system within the space; a processing unit that matches a coordinate system of a first terminal device used by the first user with a coordinate system of a second terminal device used by the second user, based on results of the first user operation and the second user operation; Equipped with Information processing system. (15) the operation of the first user includes the first user specifying an origin and an axis direction of a coordinate system in the space; the operation of the second user includes the second user specifying, in the space, the same origin and axis direction as those specified by the first user; (14) An information processing system according to (14). (16) The processing unit converting a coordinate system of the first terminal device into a coordinate system having an origin and axis directions designated by the first user; converting a coordinate system of the second terminal device into a coordinate system having an origin and axis directions specified by the second user; (15) An information processing system according to (15). (17) the first user operation includes the first user specifying an origin and an axis direction of a coordinate system along a corner existing in the space; the second user operation includes the second user specifying, along the corner portion, an origin and an axial direction that are the same as the origin and the axial direction specified by the first user; The information processing system according to any one of (14) to (16). (18) the designation in the first user operation includes the first user touching a specific position in the space directly or with a controller; the designation by the second user operation includes the second user touching the same position as the position touched by the first user directly or with a controller; (17) An information processing system according to (17). (19) the display unit displays a plurality of corners present in the space in a selectable manner; the first user operation includes the first user specifying an origin and an axis direction of a coordinate system along one corner selected from the plurality of corners; the second user operation includes the second user specifying, along the same corner as the corner selected by the first user, an origin and an axial direction that are the same as the origin and the axial direction that are specified by the first user; The information processing system according to any one of (14) to (17). (20) the display unit displays an origin and an axis direction that can be specified along the corner portion in a selectable manner. (19) An information processing system according to (19). (twenty one) the display unit displays a virtual space image in which the occlusion of the space is reproduced; the first user operation includes the first user matching the virtual space image to the space; the second user operation includes the second user matching the virtual space image with the space; The information processing system according to any one of (14) to (16). (twenty two) the first user operation includes the first user moving within the space and moving the virtual space image; the second user operation includes the second user moving within the space and moving the virtual space image; (21) An information processing system according to (21). (twenty three) the first user operation includes the first user setting a controller on a positioning device arranged in the space; the second user operation includes the second user setting a controller on the positioning device; The information processing system according to any one of (14) to (16). (twenty four) The position and orientation of the controller set on the positioning device relative to the positioning device are determined. (23) An information processing system according to (23). (twenty five) The space has a size that allows the first user and the second user to communicate with each other face to face. The information processing system according to any one of (14) to (24). (26) No data exchange occurs between the first terminal device and the second terminal device; The information processing system according to any one of (14) to (25). [Explanation of symbols]

[0149] 1. Information Processing Systems 2 users 2-1 User 2-2 Users 3 space 3v Virtual Space Image 3vp origin 4. Objects 4a Corner 4o Occlusion section 5 Terminal Devices 5-1 Terminal equipment 5-2 Terminal Equipment 50 Storage section 500 Information Processing Program 51 Sensor unit 52 Display section 53 Processing section 54 Controller 55 Positioning device 56 Controller 56L Controller 56R Controller 5v imaging range 6 Server equipment 60 Storage section 600 Location information 61 Processing section 7 users 7v Avatar 8 VR equipment C coordinate system C1 coordinate system C2 coordinate system M1 Message M2 Messages M3 Messages M4 Messages M5 Message M6 Message M7 Message M8 Message P1 point P2 point P3 points

Claims

1. displaying, by a first terminal device used by a first user, information for causing a first user in a specific space to perform a first user operation for specifying a coordinate system within the space; displaying, by a second terminal device used by a second user, information for causing a second user in the space to perform a second user operation for specifying a coordinate system within the space; and matching a coordinate system of the first terminal device with a coordinate system of the second terminal device based on results of the first user operation and the second user operation; Including, Information processing methods.

2. the operation of the first user includes the first user specifying an origin and an axis direction of a coordinate system in the space; the operation of the second user includes the second user specifying, in the space, an origin and an axis direction that are the same as the origin and the axis direction specified by the first user; The information processing method according to claim 1 .

3. The matching of the coordinate systems may include: converting a coordinate system of the first terminal device into a coordinate system having an origin and axis directions designated by the first user; converting a coordinate system of the second terminal device into a coordinate system having an origin and axis directions designated by the second user; Including, The information processing method according to claim 2 .

4. the first user operation includes the first user specifying an origin and an axis direction of a coordinate system along a corner existing in the space; the second user operation includes the second user specifying, along the corner portion, an origin and an axial direction that are the same as the origin and the axial direction specified by the first user; The information processing method according to claim 1 .

5. the designation in the first user operation includes the first user touching a specific position in the space directly or with a controller; the designation by the second user operation includes the second user touching the same position as the position touched by the first user directly or with a controller; The information processing method according to claim 4.

6. The method further includes each of the first terminal device and the second terminal device displaying a plurality of corners present in the space in a selectable manner; the first user operation includes the first user specifying an origin and an axis direction of a coordinate system along one corner selected from the plurality of corners; the second user operation includes the second user specifying, along the same corner as the corner selected by the first user, an origin and an axial direction that are the same as the origin and the axial direction that are specified by the first user; The information processing method according to claim 1 .

7. each of the first terminal device and the second terminal device displays an origin and an axis direction that can be specified along the corner portion in a selectable manner; The information processing method according to claim 6.

8. The method further includes each of the first terminal device and the second terminal device displaying a virtual space image in which occlusion of the space is reproduced; the first user operation includes the first user matching the virtual space image with the space; the second user operation includes the second user matching the virtual space image with the space; The information processing method according to claim 1 .

9. the first user operation includes the first user moving within the space and moving the virtual space image; the second user operation includes the second user moving within the space and moving the virtual space image; The information processing method according to claim 8.

10. the first user operation includes the first user setting a controller on a positioning device arranged in the space; the second user operation includes the second user setting a controller on the positioning device; The information processing method according to claim 1 .

11. The position and orientation of the controller set on the positioning device relative to the positioning device are determined. The information processing method according to claim 10.

12. The space has a size that allows the first user and the second user to communicate with each other face to face. The information processing method according to claim 1 .

13. does not involve data exchange between the first terminal device and the second terminal device; The information processing method according to claim 1 .

14. a display unit that displays information for causing a first user in a specific space to perform a first user operation for specifying a coordinate system within the space, and information for causing a second user in the space to perform a second user operation for specifying a coordinate system within the space; a processing unit that matches a coordinate system of a first terminal device used by the first user with a coordinate system of a second terminal device used by the second user, based on results of the first user operation and the second user operation; Equipped with Information processing system.

15. the operation of the first user includes the first user specifying an origin and an axis direction of a coordinate system in the space; the operation of the second user includes the second user specifying, in the space, an origin and an axis direction that are the same as the origin and the axis direction specified by the first user; The information processing system according to claim 14.

16. the first user operation includes the first user specifying an origin and an axis direction of a coordinate system along a corner existing in the space; the second user operation includes the second user specifying, along the corner portion, an origin and an axial direction that are the same as the origin and the axial direction specified by the first user; The information processing system according to claim 14.

17. the designation in the first user operation includes the first user touching a specific position in the space directly or with a controller; the designation by the second user operation includes the second user touching the same position as the position touched by the first user directly or with a controller; 17. The information processing system according to claim 16.

18. the display unit displays a plurality of corners present in the space in a selectable manner; the first user operation includes the first user specifying an origin and an axis direction of a coordinate system along one corner selected from the plurality of corners; the second user operation includes the second user specifying, along the same corner as the corner selected by the first user, an origin and an axial direction that are the same as the origin and the axial direction that are specified by the first user; The information processing system according to claim 14.

19. the display unit displays a virtual space image in which the occlusion of the space is reproduced; the first user operation includes the first user matching the virtual space image with the space; the second user operation includes the second user matching the virtual space image with the space; The information processing system according to claim 14.

20. the first user operation includes the first user setting a controller on a positioning device arranged in the space; the second user operation includes the second user setting a controller on the positioning device; The information processing system according to claim 14.

Images