Position estimation device, position estimation method, and computer-readable storage medium

By introducing the estimation method of public point cloud and feature point data in SLAM technology, the problem that SLAM technology is difficult to estimate the location and direction of multiple trackers in a common coordinate system is solved, and the position and direction sharing between trackers is realized. Coordination improves the accuracy and sharing capabilities of location estimation.

CN111788606BActive Publication Date: 2024-12-03SONY INTERACTIVE ENTERTAINMENT LLC
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN201880090197.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2018-03-01
Publication Date
2024-12-03
Estimated Expiration
2038-03-01

AI Technical Summary

Technical Problem

Existing SLAM technologies are difficult to estimate the positions and directions of multiple trackers in a common coordinate system, and cannot map the positions and directions of each tracker into the common coordinate system.

Method used

A position estimation device is designed, including a point cloud storage part and an estimation part. Through the common point cloud and feature point data, the position or direction of each tracker in the common coordinate system is estimated, and if necessary, the data is sent to Tracker, implements position and direction sharing between trackers.

Benefits of technology

It realizes the accurate estimation of the positions and directions of multiple trackers in a common coordinate system, solves the problem of tracker position and direction mapping in applications such as games, and improves the sharing and coordination capabilities of positions and directions.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN111788606B_ABST
    Figure CN111788606B_ABST
Patent Text Reader

Abstract

A position estimation device, a tracker, a position estimation method and a program are provided, wherein the position estimation device is capable of estimating the position or orientation of each of a plurality of trackers in a common coordinate system. A point cloud storage unit (80) stores a common point cloud representing the position by the common coordinate system. A position estimation unit (86) estimates the position or orientation of a first tracker in the common coordinate system based on the common point cloud and first feature data. The position estimation unit (86) estimates the position or orientation of a second tracker in the common coordinate system based on the common point cloud and second feature data.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to a position estimating device, a tracker, a position estimating method and a program. Background Art

[0002] A SLAM (Simultaneous Localization and Mapping) technique for estimating the position of a tracker and generating a map of an environment is known.

[0003] In SLAM technology, feature point data including a plurality of feature points is generated based on sensing data acquired by a tracker (such as an image captured by a camera included in the tracker). Then, based on the result of comparing the generated feature point data with an environment map (which is a pre-stored point cloud), the position or orientation of the tracker is estimated and the environment map is updated. Summary of the invention

[0004] [Technical issues]

[0005] The inventors have been considering applying SLAM technology to games etc., in which multiple users wearing trackers each participate. In such games etc., it is necessary to estimate the position or orientation of each tracker in a common coordinate system shared by the trackers.

[0006] However, the position and orientation of the tracker estimated using SLAM technology is only relative to the position and orientation of the tracker at a certain moment. Therefore, using SLAM technology to estimate the position and orientation of each tracker cannot map the estimated position and orientation to a common coordinate system. Therefore, it is impossible to estimate the position or orientation of each tracker in a common coordinate system.

[0007] The present invention has been made in view of the above-mentioned problems, and an object of the present invention is to provide a position estimation device, a tracker, a position estimation method, and a program that can estimate the position or direction of each of a plurality of trackers in a common coordinate system.

[0008] [Problem Solution]

[0009] In order to solve the above-mentioned problem, the position estimation device according to the present invention includes: a point cloud storage part, which stores a common point cloud that expresses a position using a common coordinate system; and an estimation part, which estimates the position or orientation of the first tracker in the common coordinate system, and the position or orientation of the second tracker in the common coordinate system. The estimation part estimates the position or orientation of the first tracker in the common coordinate system based on the common point cloud and first feature point data, the first feature point data indicating the relative positions of a plurality of feature points and being generated based on the first sensing data acquired by the first tracker. The estimation part estimates the position or orientation of the second tracker in the common coordinate system based on the common point cloud and the second feature point data, the second feature point data indicating the relative positions of a plurality of feature points and being generated based on the second sensing data acquired by the second tracker.

[0010] In one embodiment of the present invention, the position estimation device also includes: a sending part, which sends data indicating the position or orientation of the first tracker in the common coordinate system to the first tracker, and the sending part sends data indicating the position or orientation of the second tracker in the common coordinate system to the second tracker.

[0011] In this embodiment, the sending part may also send data indicating the position or orientation of the second tracker in the public coordinate system to the first tracker, and the sending part may also send data indicating the position or orientation of the first tracker in the public coordinate system to the second tracker.

[0012] In addition, in one embodiment of the present invention, the position estimation device also includes: an adding part, which adds one or more points arranged at one or more positions in the common coordinate system to the common point cloud based on the common point cloud and the first feature point data, and the one or more positions are associated with one or more positions of one or more feature points contained in the first feature point data.

[0013] Optionally, the point cloud storage part also stores a separate point cloud that uses a separate coordinate system different from the common coordinate system to represent the position; the position estimation device also includes: an adding part, when it is determined that some of the multiple points contained in the common point cloud are consistent with some points contained in the separate point cloud, adds points corresponding to some points contained in the separate point cloud to the common point cloud; before executing the addition of the points corresponding to the some points contained in the separate point cloud to the common point cloud, the estimation part estimates the position or orientation of the first tracker in the separate coordinate system based on the separate point cloud and the first feature point data; and after executing the addition of the points corresponding to the some points contained in the separate point cloud to the common point cloud, the estimation part estimates the position or orientation of the first tracker in the common coordinate system based on the common point cloud and the first feature point data.

[0014] In one embodiment of the present invention, the position estimation device also includes: a receiving part, which receives the first feature point data from the first tracker, the receiving part receives the second feature point data from the second tracker, the estimating part estimates the position or orientation of the first tracker in the common coordinate system based on the first feature point data received by the receiving part, and the estimating part estimates the position or orientation of the second tracker in the common coordinate system based on the second feature point data received by the receiving part.

[0015] Optionally, the position estimation device also includes: a receiving part; and a feature point data generating part, which generates the first feature point data and the second feature point data, the receiving part receives the first sensing data from the first tracker, the receiving part receives the second sensing data from the second tracker, the feature point data generating part generates the first feature point data based on the received first sensing data, the feature point data generating part generates the second feature point data based on the received second sensing data, the estimating part estimates the position or orientation of the first tracker in the common coordinate system based on the first feature point data generated by the feature point data generating part, and the estimating part estimates the position or orientation of the second tracker in the common coordinate system based on the second feature point data generated by the feature point data generating part.

[0016] Another position estimating device according to the present invention includes: a point cloud storage part which stores a plurality of point clouds, each of which is associated with an area, a position information acquisition part which acquires position information indicating the position of a tracker, a point cloud selection part which selects a point cloud associated with the area including the position indicated by the position information from the plurality of point clouds, and an estimation part which estimates the position or orientation of the tracker based on the selected point cloud and feature point data which indicates relative positions of a plurality of feature points and is generated based on sensing data acquired by the tracker.

[0017] A tracker according to the present invention includes: a transmitting part that transmits position information indicating the position of the tracker to a server, the server storing a plurality of point clouds, each of which is associated with an area; and a receiving part that receives data from the server, the data indicating the position or orientation of the tracker estimated in the server based on a point cloud and feature point data, the point cloud being selected from the plurality of point clouds and associated with the area including the position indicated by the position information, the feature point data indicating relative positions of a plurality of feature points and being generated based on sensing data acquired by the tracker.

[0018] The position estimation method according to the present invention includes the following steps: estimating the position or orientation of a first tracker in a common coordinate system based on a common point cloud representing a position using the common coordinate system and first feature point data, wherein the first feature point data indicates relative positions of a plurality of feature points and is generated based on first sensing data acquired by the first tracker, and estimating the position or orientation of a second tracker in the common coordinate system based on the common point cloud and second feature point data, wherein the second feature point data indicates relative positions of a plurality of feature points and is generated based on second sensing data acquired by the second tracker.

[0019] The program according to the present invention causes a computer to execute: a step of estimating a position or orientation of the first tracker in the common coordinate system based on a common point cloud representing positions using the common coordinate system and first feature point data, the first feature point data indicating relative positions of a plurality of feature points and being generated based on first sensing data acquired by the first tracker, and a step of estimating a position or orientation of the second tracker in the common coordinate system based on the common point cloud and second feature point data, the second feature point data indicating relative positions of a plurality of feature points and being generated based on second sensing data acquired by the second tracker. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 is a configuration diagram showing an example of a location sharing system according to an embodiment of the present invention.

[0021] Figure 2 is a configuration diagram showing an example of a tracker according to an embodiment of the present invention.

[0022] Figure 3 is a diagram showing an example of a common point cloud.

[0023] Figure 4 is a diagram showing an example of a first individual point cloud.

[0024] Figure 5 is a diagram showing an example of a second independent point cloud.

[0025] Figure 6 is a diagram showing an example of a captured image.

[0026] Figure 7 is a diagram showing an example of feature points.

[0027] Figure 8 is a diagram showing an example of a first individual point cloud.

[0028] Fig. 9 is a diagram showing an example of a first individual point cloud.

[0029] Fig.10 is a diagram showing an example of a common point cloud.

[0030] Fig.11 is a diagram showing an example of a captured image.

[0031] Fig.12 is a diagram showing an example of feature points.

[0032] Fig.13 is a diagram showing an example of a second individual point cloud.

[0033] Fig.14 is a diagram illustrating an exemplary second individual point cloud.

[0034] Fig.15 is a diagram showing an example of a common point cloud.

[0035] Fig.16 is a functional block diagram showing an example of functions implemented in a location sharing system according to an embodiment of the present invention.

[0036] Fig.17 is a diagram showing an example of reference management data.

[0037] Fig.18 is a flowchart showing an example of a process performed in a location sharing system according to an embodiment of the present invention. DETAILED DESCRIPTION

[0038] Hereinafter, one embodiment of the present invention will be described in detail with reference to the accompanying drawings.

[0039] Figure 1 1 is a configuration diagram showing an example of a location sharing system 1 according to an embodiment of the present invention. Figure 1 As shown, the location sharing system 1 according to this embodiment includes a server 10 and a plurality of terminals (eg, trackers 12 in this embodiment). Figure 1 , a tracker 12a and a tracker 12b are shown.

[0040] Each of the server 10, the tracker 12a, and the tracker 12b is connected to a computer network 14, such as the Internet. Therefore, the server 10, the tracker 12a, and the tracker 12b can communicate with each other via the computer network 14.

[0041] like Figure 1 As shown, for example, the server 10 according to the present embodiment is a server computer including a processor 20, a storage section 22, and a communication section 24. Note that the server 10 may be implemented by a cloud system or a server system having a plurality of chassis.

[0042] The processor 20 is a program control device such as a CPU (Central Processing Unit), which operates, for example, according to a program installed in the server 10. The storage section 22 is, for example, a storage element such as a ROM (Read Only Memory) or a RAM (Random Access Memory), a hard disk drive, etc. The program executed by the processor 20 or the like is stored in the storage section 22. The communication section 24 is a communication interface such as a network board or a wireless LAN (Local Area Network) module.

[0043] For example, each tracker 12 according to the present embodiment is a device for tracking the position and direction of the user wearing the tracker 12 in the present embodiment. Figure 1 , a head mounted display (HMD) is shown as an example of the tracker 12.

[0044] like Figure 2 As shown, the tracker 12 according to the present embodiment includes a processor 30 , a storage section 32 , a communication section 34 , an input / output section 36 , a display section 38 , a sensor section 40 , and an audio output section 42 .

[0045] The processor 30 is a program-controlled device such as a microprocessor, for example, and operates according to a program installed in the tracker 12 .

[0046] The storage section 32 is, for example, a storage element such as a ROM or a RAM, etc. A program executed by the processor 30 or the like is stored in the storage section 32 .

[0047] The communication section 34 is, for example, a communication interface such as a wireless LAN module.

[0048] The input / output section 36 is, for example, an input / output port such as an HDMI (registered trademark) (High-Definition Multimedia Interface) port, a USB (Universal Serial Bus) port, or an AUX (Auxiliary) port.

[0049] The display section 38 is, for example, a display such as a liquid crystal display or an organic EL (electroluminescence) display, and is provided on the front surface of the tracker 12. In addition, the display section 38 is housed in the housing of the tracker 12. The display section 38 according to the present embodiment is configured to be able to display a three-dimensional image, for example, by displaying an image for the left eye and an image for the right eye. Note that the display section 38 may alternatively be configured to be able to display only a two-dimensional image and not a three-dimensional image.

[0050] The sensor part 40 includes, for example, sensors such as a camera, an inertial sensor (IMU (Inertial Measurement Unit)), a magnetometer (i.e., a direction sensor), and a GPS (Global Positioning System) module. For example, the camera included in the sensor part 40 captures an image at a predetermined frame rate. In addition, the magnetometer included in the sensor part 40 outputs data indicating the direction in which the tracker 12 is facing to the processor 30 at a predetermined frame rate. In addition, the inertial sensor included in the sensor part 40 outputs data indicating the acceleration, rotation amount, movement amount, etc. of the tracker 12 to the processor 30 at a predetermined frame rate. In addition, the GPS module included in the sensor part 40 outputs data indicating the latitude and longitude of the tracker 12 to the processor 30 at a predetermined frame rate.

[0051] The audio output section 42 is, for example, a headphone, a speaker, or the like, and outputs audio represented by audio data or the like.

[0052] In the present embodiment, for example, estimation of the position and orientation of the tracker 12 using SLAM (Simultaneous Localization and Mapping) technology is performed in the server 10 in the following manner. Then, data indicating the position and orientation thus estimated is provided from the server 10 to the tracker 12.

[0053] Figure 3 1 is a diagram showing an example of a common point cloud 50 managed by the server 10 according to the present embodiment. The common point cloud 50 is used to estimate the position and direction of each of the trackers 12a and 12b. Figure 3 As shown, a plurality of points 52 are included in the public point cloud 50. The position in the public point cloud 50 according to the present embodiment is represented using the X1-Y1-Z1 coordinate system. Therefore, for each of the plurality of points 52 included in the public point cloud 50, a coordinate value represented using the X1-Y1-Z1 coordinate system is set. Hereinafter, the X1-Y1-Z1 coordinate system will be referred to as the public coordinate system. Note here that the coordinate value of each point 52 included in the public point cloud 50 can be represented as a combination of, for example, latitude, longitude, and altitude.

[0054] Figure 4 1 is a diagram showing an example of a first individual point cloud 54 managed by the server 10 according to the present embodiment. The first individual point cloud 54 is used to estimate the position and orientation of the tracker 12a. Figure 4 As shown, a plurality of points 56 are included in the first individual point cloud 54. Coordinate values ​​expressed using the X2-Y2-Z2 coordinate system are set for each of the plurality of points 56. Hereinafter, the X2-Y2-Z2 coordinate system is referred to as the first individual coordinate system.

[0055] Figure 5 1 is a diagram showing an example of a second individual point cloud 58 managed by the server 10 according to the present embodiment. The second individual point cloud 58 is used to estimate the position and orientation of the tracker 12b. Figure 5 As shown, a plurality of points 60 are included in the second individual point cloud 58. Coordinate values ​​expressed using the X3-Y3-Z3 coordinate system are set for each of the plurality of points 60. Hereinafter, the X3-Y3-Z3 coordinate system is referred to as the second individual coordinate system.

[0056] In the present embodiment, it is assumed that the common coordinate system, the first individual coordinate system, and the second individual coordinate system are independent coordinate systems different from each other.

[0057] In this embodiment, the estimation of the position and orientation of the tracker 12a and the tracker 12b is based on Figure 3 The public point cloud 50 shown, Figure 4 The first individual point cloud 54 and Figure 5 This is performed for a second individual point cloud 58 as shown.

[0058] An example of estimating the position and direction of the tracker 12 a based on sensing data generated by the sensor portion 40 of the tracker 12 a will be described below.

[0059] In the present embodiment, an image is captured at a predetermined frame rate by a camera included in the sensor portion 40 of the tracker 12a. Hereinafter, an image captured in this manner will be referred to as a captured image. The captured image corresponds to an example of sensing data generated by the sensor portion 40 of the tracker 12a. Figure 6 , captured image 62 is shown as an example of a captured image captured by a camera included in sensor portion 40 of tracker 12a. Images of object 64a and object 64b are included in Figure 6 In the captured image 62 shown.

[0060] Here, in this embodiment, for example, Figure 7 As shown, a plurality of feature points 66 of the captured image 62 are extracted based on the captured image 62. Figure 7, feature points 66 extracted along the edges of object 64a and object 64b are shown. Then, feature point data indicating the relative positions of the extracted feature points 66 are generated. Here, since the relative positions of the feature points 66 are indicated in the feature point data, the feature point data can be regarded as a kind of point cloud.

[0061] Here, the position of the feature point 66 may be represented by a two-dimensional coordinate value on a two-dimensional plane formed by the captured image 62 (i.e., a two-dimensional coordinate value expressed using a two-dimensional camera coordinate system). Alternatively, the position of the feature point 66 may be represented by, for example, a three-dimensional coordinate value (i.e., a three-dimensional coordinate value expressed using a three-dimensional camera coordinate system), the three-dimensional coordinate value including the two-dimensional coordinate value on the two-dimensional plane formed by the captured image 62 and a coordinate value corresponding to the depth (in the Z direction).

[0062] Then, in this embodiment, if Figure 8 As shown, based on the result of comparing the feature points 66 extracted from the captured image 62 with the points 56 included in the first individual point cloud 54, the SLAM technology is used to estimate the position 68 and direction 70 of the tracker 12a. The estimated position 68 and direction 70 here correspond to, for example, the position and direction of the tracker 12a when the captured image 62 is captured. In addition, the position 68 and direction 70 estimated in the above manner are expressed using the first individual coordinate system.

[0063] Then, in the present embodiment, data indicating the position 68 and the direction 70 expressed using the first separate coordinate system and estimated in the above-described manner are transmitted from the server 10 to the tracker 12a, and the tracker 12a then performs various processes based on the received data.

[0064] Then, in this embodiment, if Fig. 9 As shown, points 56 corresponding to those points of the feature points 66 (whose corresponding points 56 are not included in the first individual point cloud 54) are added to the first individual point cloud 54. Fig. 9 In the example of , the plurality of points 56 in the region R1 are added to the first individual point cloud 54 . Note that at this time, updating of the positions of the points 56 already included in the first individual point cloud 54 may be performed based on the feature points 66 .

[0065] In this case, the addition and update of the point 56 may be performed multiple times for the generated feature points 66. Furthermore, the addition and update of the point 56 may not be performed for the feature point 66 of a point that is a shadow affected by weather or time period, for example.

[0066] Then, in the present embodiment, the points 56 included in the first individual point cloud 54 are compared with the points 52 included in the common point cloud 50. It is assumed here that some of the points 56 included in the first individual point cloud 54 are determined to be consistent with some of the points 52 included in the common point cloud 50. In this case, Fig.10 As shown, points 52 corresponding to those points among the points 56 included in the first individual point cloud 54 (whose corresponding points 52 are not included in the common point cloud 50) are added to the common point cloud 50. Fig.10 In the example of , a plurality of points 52 in region R2 are added to the common point cloud 50 .

[0067] Then, in the present embodiment, after the point 52 is added to the common point cloud 50 as described above, the feature point extracted from the captured image is compared with the point 52 included in the common point cloud 50. Therefore, the position and direction of the tracker 12a to be estimated will be expressed using the common coordinate system. Here, data expressed by latitude, longitude, and altitude can be used as data indicating each position and direction expressed using the common coordinate system.

[0068] In this case, points 52 corresponding to those points among the feature points whose corresponding points 52 are not included in the common point cloud 50 are added to the common point cloud 50. Note that at this time, updating of the positions of the points 52 already included in the common point cloud 50 may be performed based on the feature points.

[0069] In this case, the addition and update of the point 52 may be performed multiple times for the generated feature points. Furthermore, the addition and update of the point 52 may not be performed for feature points such as shadows that are affected by weather or time periods, for example.

[0070] An example of estimating the position and direction of the tracker 12 b based on the sensing data generated by the sensor portion 40 of the tracker 12 b will be described below.

[0071] As in the case of the tracker 12a, in the present embodiment, a capture image is captured at a predetermined frame rate by a camera included in the sensor section 40 of the tracker 12b. The capture image corresponds to an example of sensing data generated by the sensor section 40 of the tracker 12b. Fig.11 , a captured image 72 is shown as an example of an image captured by a camera included in the sensor portion 40 of the tracker 12b. The images of the object 64c and the object 64d are included in Fig.11 In the captured image 72 shown.

[0072] Here, in this embodiment, for example, Fig.12 As shown, a plurality of feature points 74 of the captured image 72 are extracted based on the captured image 72. Fig.12 , feature points 74 extracted along the object 64c and the edge of the object 64c are shown. Then, feature point data indicating the relative positions of the extracted feature points 74 are generated. Here, since the relative positions of the feature points 74 are indicated in the feature point data, the feature point data can be regarded as a kind of point cloud.

[0073] Here, the position of the feature point 74 may be represented by a two-dimensional coordinate value (i.e., a two-dimensional coordinate value expressed using a two-dimensional camera coordinate system) on a two-dimensional plane formed by the captured image 72. Alternatively, the position of the feature point 74 may be represented by, for example, a three-dimensional coordinate value (i.e., a three-dimensional coordinate value expressed using a three-dimensional camera coordinate system), the three-dimensional coordinate value including a two-dimensional coordinate value on a two-dimensional plane formed by the captured image 72 and a coordinate value corresponding to a depth (in the Z direction).

[0074] Then, in this embodiment, if Fig.13 As shown, based on the result of comparing the feature points 74 extracted from the captured image 72 with the points 60 included in the second separate point cloud 58, the SLAM technology is used to estimate the position 76 and direction 78 of the tracker 12b. The estimated position 76 and direction 78 here correspond to, for example, the position and direction of the tracker 12b when the image 72 is captured. In addition, the position 76 and direction 78 estimated in the above manner are expressed using a second separate coordinate system.

[0075] Then, in the present embodiment, data indicating the position 76 and the direction 78 expressed using the second separate coordinate system and estimated in the above-described manner are transmitted from the server 10 to the tracker 12b, and the tracker 12b then performs various processes based on the received data.

[0076] Then, in this embodiment, if Fig.14 As shown, the points 60 corresponding to those points of the feature points 74 (whose corresponding points 60 are not included in the second individual point cloud 58) are added to the second individual point cloud 58. Fig.14 In the example of , the plurality of points 60 in the region R3 are added to the second individual point cloud 58. Note that at this time, updating of the positions of the points 60 already included in the second individual point cloud 58 may be performed based on the feature points 74.

[0077] In this case, the addition and update of the point 60 may be performed multiple times for the generated feature points 74. Furthermore, the addition and update of the point 60 may not be performed for the feature points 74 such as points of shadows affected by weather or time periods.

[0078] Then, in the present embodiment, the points 60 included in the second individual point cloud 58 are compared with the points 52 included in the common point cloud 50. It is assumed here that some of the points 60 included in the second individual point cloud 58 are determined to be consistent with some of the points 52 included in the common point cloud 50. In this case, Fig.15 As shown, points 52 corresponding to those points among the points 60 included in the second individual point cloud 58 (whose corresponding points 52 are not included in the common point cloud 50) are added to the common point cloud 50. Fig.15 In the example of , a plurality of points 52 in region R4 are added to the common point cloud 50 .

[0079] Then, in the present embodiment, after the point 52 is added to the common point cloud 50 as described above, the feature point extracted from the captured image is compared with the point 52 included in the common point cloud 50. Therefore, the position and direction of the tracker 12b to be estimated will be expressed using the common coordinate system. Here, data expressed by latitude, longitude, and altitude can be used as data indicating each position and direction expressed using the common coordinate system.

[0080] In this case, points 52 corresponding to those points among the feature points whose corresponding points 52 are not included in the common point cloud 50 are added to the common point cloud 50. Note that at this time, updating of the positions of the points 52 already included in the common point cloud 50 may be performed based on the feature points.

[0081] In this case, the addition and update of the point 52 may be performed multiple times for the generated feature points. Furthermore, the addition and update of the point 52 may not be performed for feature points such as shadow points that are affected by weather or time periods, for example.

[0082] As described above, in the present embodiment, the estimation of the position and orientation of the tracker 12 is performed based on the first individual point cloud 54 or the second individual point cloud until, for example, the addition of the point 52 to the common point cloud 50 is performed.

[0083] Then, in this case, as described above, for example, data indicating the position and orientation expressed using the first separate coordinate system is provided to the tracker 12a. At the same time, data indicating the position and orientation expressed using the second separate coordinate system is provided to the tracker 12b. Therefore, in this case, the relationship between the position and orientation of the tracker 12a and the position and orientation of the tracker 12b cannot be recognized.

[0084] However, after adding point 52 to common point cloud 50, each of tracker 12a and tracker 12b is provided with data indicating the position and orientation expressed using the common coordinate system. Therefore, according to the present embodiment, tracker 12 can be provided with an estimated position or orientation of tracker 12 in the common coordinate system.

[0085] Here, after adding the point 52 to the common point cloud 50, data indicating the position and orientation of the tracker 12b expressed using the common coordinate system may be provided to the tracker 12a. Furthermore, after adding the point 52 to the common point cloud 50, data indicating the position and orientation of the tracker 12a expressed using the common coordinate system may be provided to the tracker 12b. This will enable processing to be performed involving the position and orientation of the two trackers 12a expressed in each of the tracker 12a and the tracker 12b using the coordinate system shared by the two trackers 12.

[0086] Furthermore, in the present embodiment, in the case where estimation of the position and orientation of the tracker 12 is performed based on the common point cloud 50, for example, even when the position and orientation of the tracker 12 are estimated for the first time, the position and orientation expressed using the common coordinate system are provided to the tracker 12. Here, as described above, data indicating the position of the tracker 12 expressed in latitude, longitude, and altitude may be provided to the tracker 12. This will enable estimation of the absolute position and orientation of the tracker 12 even when the position and orientation of the tracker 12 are estimated for the first time.

[0087] Furthermore, in the present embodiment, the number of points 52 included in the common point cloud 50 is increased by each tracker 12 performing estimation of the position and orientation of the tracker 12. Therefore, according to the present embodiment, when the process of estimating the position and orientation of the tracker 12 is performed, the area covered by the common point cloud 50 shared by the trackers 12 can be expanded. Therefore, in the present embodiment, the administrator or the like does not have to bother to perform the process for expanding the area covered by the common point cloud 50.

[0088] In addition, for example, the direction of the tracker 12 may be estimated based on the direction of the tracker 12 measured by the magnetometer included in the sensor portion 40 of the tracker 12. Then, the position and direction of the tracker 12 may be estimated on the basis of the result of the direction estimation based on the direction, the feature point data, and the common point cloud 50. This will result in the accuracy of the position and direction estimation of the tracker 12 being improved.

[0089] In addition, the public point cloud 50 may be managed by a plurality of databases in the server 10. Then, in each database, a public point cloud 50 corresponding to an area (e.g., a range of latitude and longitude) of the database may be stored. Here, the areas corresponding to the public point clouds 50 stored in the respective databases may partially overlap with each other.

[0090] Here, for example, based on the approximate latitude and longitude of the tracker 12 measured by the GPS module included in the sensor part 40 of the tracker 12, a database corresponding to the area including the position represented by the latitude and the longitude can be identified. Then, the position and direction of the tracker 12 can be estimated based on the common point cloud 50 and the feature point data stored in the identification database. This will enable the position and direction of the tracker 12 to be estimated more accurately than when the position and direction of the tracker 12 are estimated based only on the measurement results of the GPS module. Note that the approximate latitude and longitude of the tracker 12 are not necessarily measured by the GPS module. The approximate latitude and longitude of the tracker 12 can be identified based on, for example, identification information (SSID (Service Set Identifier)) of a Wi-Fi (registered trademark) access point, information of a base station for a mobile phone, information provided by a location service, and the like.

[0091] In addition, in the server 10, the public point cloud 50 may be managed, for example, by a database associated with a time period, a season, a weather condition, etc. Then, in this case, for example, the position and direction of the tracker 12 may be estimated based on the public point cloud 50 stored in the database associated with the time period, the season, the weather condition, etc. at the time of estimation.

[0092] In addition, the captured images can be stored in the server 10 as textures of the common point cloud 50, the first individual point cloud 54, and the second individual point cloud 58, for example. For example, these textures can be used for estimation using machine learning. In addition, based on these textures, other textures can also be generated.

[0093] Furthermore, three-dimensional shape data may be generated from the common point cloud 50, the first individual point cloud 54, and the second individual point cloud 58. For example, the three-dimensional shape data may be used for estimation using machine learning.

[0094] Furthermore, in conjunction with the data indicative of the estimated position and orientation of the tracker 12 , the tracker 12 may be provided with additional information (eg, tags) that is stored in the server 10 so as to be associated with the estimated position and orientation.

[0095] Furthermore, the tracker 12 can perform processing to cause the display section 38 to display a captured road image in the navigation system, such as an arrow for guidance appropriately superimposed along the road, based on the received data indicating the position and direction of the tracker 12. According to the present embodiment, it is expected that the arrow for guidance can be displayed with sufficient accuracy even for a visually impaired person.

[0096] In addition, for example, the tracker 12 can perform a superimposition process of metadata such as an advertisement, rating and word-of-mouth evaluation of a restaurant, or a supplementary description of a sightseeing spot based on the received data indicating the position and direction of the tracker 12. In addition, the tracker 12 can perform, for example, a process of displaying a captured image in which a virtual object is superimposed at an appropriate position and in an appropriate direction, or a process of eliminating unnecessary human graphics from a captured image or a captured moving image. Here, for example, the graphics of a person included in the captured image and the background image transmitted from the server 10 can be superimposed.

[0097] In addition, for example, the tracker 12 can perform processing in an AR (augmented reality) game, such as a treasure hunt game, a monster collection game, or a location acquisition game, based on the received data indicating the position and orientation of the tracker 12. In this case, for example, items or monsters can be arranged to appear mainly in places with a small amount of texture and a registered public point cloud 50. This will make it easier to collect feature point data from places with a small amount of texture and a public point cloud 50.

[0098] In addition, the position sharing system 1 according to the present embodiment can also be used in VR (virtual reality) theme parks, where various textures are displayed while maintaining the shape of a three-dimensional model, and for automatically generating real-world content based on three-dimensional models, textures, etc.

[0099] In addition, the position sharing system 1 according to the present embodiment can be used for transaction situations in the virtual world. For example, a user who owns a place in the virtual world can add information such as advertisements to the place.

[0100] Other possible applications of the position sharing system 1 according to the present embodiment include, for example, surveying, land management, disaster simulation, VR (virtual reality) travel, flight experience, automatic walking, automatic driving, and automatic flight.

[0101] As described above, the tracker 12 according to the present embodiment is able to perform various processes based on the received data indicating the position and direction of the tracker 12 .

[0102] Next, the functions of the position sharing system 1 according to the present embodiment and the processing performed in the position sharing system 1 will be further described.

[0103] Fig.16 is a functional block diagram showing an example of functions implemented in the location sharing system 1 according to the present embodiment. Note that the location sharing system 1 according to the present embodiment does not necessarily implement Fig.16 All functions shown and can be realized Fig.16 Functions other than those shown.

[0104] like Fig.16 As shown, the server 10 includes, for example, a point cloud storage section 80, a reference management data storage section 82, a feature point data receiving section 84, a position estimation section 86, a position transmission section 88, and an addition / update section 90 in terms of functions. Each of the point cloud storage section 80 and the reference management data storage section 82 is mainly implemented by the storage section 22. Each of the feature point data receiving section 84 and the position transmission section 88 is mainly implemented by the communication section 24. Each of the position estimation section 86 and the addition / update section 90 is mainly implemented by the processor 20. The server 10 according to the present embodiment plays the role of a position transmission device, which transmits data indicating the position or orientation of the tracker 12 in the public coordinate system to the tracker 12.

[0105] The above functions can be implemented by the processor 20 executing a program installed in the server 10 as a computer, the program including commands corresponding to the above functions. The program can be provided to the server 10, for example, via a computer-readable information storage medium (such as an optical disk, a magnetic disk, a magnetic tape, a magneto-optical disk, or a flash memory) or via the Internet.

[0106] In addition, if Fig.16 As shown, the tracker 12 includes, for example, a sensing data acquisition section 92, a feature point data generation section 94, a feature point data transmission section 96, a position receiving section 98, and a processing execution section 100 in terms of functions. The sensing data acquisition section 92 is mainly implemented by the processor 30 and the sensor section 40. Each of the feature point data generation section 94 and the processing execution section 100 is mainly implemented by the processor 30. Each of the feature point data transmission section 96 and the position receiving section 98 is mainly implemented by the communication section 34.

[0107] The above functions can be implemented by the processor 30 executing a program installed in the tracker 12 as a computer, and the program includes commands corresponding to the above functions. The program can be provided to the tracker 12, for example, via a computer-readable information storage medium (such as an optical disk, a magnetic disk, a magnetic tape, a magneto-optical disk, or a flash memory) or via the Internet.

[0108] In the present embodiment, for example, the point cloud storage section 80 stores point clouds to be referred to when estimating the position or orientation of the tracker 12 , such as the common point cloud 50 , the first individual point cloud 54 , and the second individual point cloud 58 .

[0109] Here, as described above, the common point cloud 50 is generated based on the sensing data obtained by each of the trackers 12. In addition, as described above, the positions of the points 52 included in the common point cloud 50 are expressed using the common coordinate system shared by the trackers 12. The positions of the points 56 included in the first individual point cloud 54 are expressed using the first individual coordinate system. The positions of the points 60 included in the second individual point cloud 58 are expressed using the second individual coordinate system.

[0110] In this embodiment, for example, the reference management data storage section 82 stores Fig.17 Reference management data as shown. Fig.17 As shown, the reference management data according to this embodiment includes a tracker ID and reference point cloud data.

[0111] The tracker ID is identification information of the tracker 12. It is assumed here that, for example, the tracker ID of the tracker 12a is 001, and the tracker ID of the tracker 12b is 002.

[0112] The reference point cloud data is data representing a point cloud to be referred to when estimating the position or orientation of the tracker 12 identified by the tracker ID associated with the reference point cloud data. Fig.17 The reference management data shown in indicates that the tracker 12 a references the first individual point cloud 54 , and the tracker 12 b references the second individual point cloud 58 .

[0113] In the present embodiment, for example, the feature point data receiving section 84 receives feature point data indicating the relative position of the feature point from the tracker 12. The feature point data receiving section 84 receives, for example, first feature point data indicating the relative position of the feature point from the tracker 12a. Here, the first feature point data may be represented using a two-dimensional camera coordinate system defined on a two-dimensional plane formed by the captured image 62, or a three-dimensional camera coordinate system defined by the two-dimensional plane and the depth direction. In addition, the feature point data receiving section 84 receives second feature point data indicating the relative position of the feature point from the tracker 12b. Here, the second feature point data may be represented using a two-dimensional camera coordinate system defined on a two-dimensional plane formed by the captured image 72, or a three-dimensional camera coordinate system defined by the two-dimensional plane and the depth direction.

[0114] In the present embodiment, for example, the position estimating section 86 estimates the position or orientation of the tracker 12 based on the point cloud and the feature point data as described above. Here, the position estimating section 86 may identify, in the reference management data, the reference point cloud data associated with the tracker ID of the tracker 12 that has transmitted the feature point data received by the feature point data receiving section 84. Then, the position estimating section 86 may estimate the position or orientation of the tracker 12 based on the feature point data transmitted from the tracker 12 and the point cloud represented by the identified reference point cloud data.

[0115] In this case, the coordinate system in which the referenced point cloud is arranged may be used to represent the position or orientation of the tracker 12. For example, in the case where the common point cloud 50 is referenced when estimating the position or orientation of the tracker 12a, the common coordinate system may be used to represent the position or orientation of the tracker 12a. Meanwhile, for example, in the case where the first individual point cloud 54 is referenced when estimating the position or orientation of the tracker 12a, the first individual coordinate system may be used to represent the position or orientation of the tracker 12a.

[0116] For example, when the common point cloud 50 is referenced when estimating the position or orientation of the tracker 12 b, the common coordinate system may be used to represent the position or orientation of the tracker 12 b. At the same time, for example, when the second individual point cloud 58 is referenced when estimating the position or orientation of the tracker 12 b, the second individual coordinate system may be used to represent the position or orientation of the tracker 12 b.

[0117] Furthermore, as described above, the position estimating part 86 may estimate the position or direction of the tracker 12 using the direction of the tracker 12 measured by the magnetometer included in the sensor part 40 and / or the approximate latitude and longitude of the tracker 12 measured by the GPS module.

[0118] In the present embodiment, for example, the position transmitting section 88 transmits data indicating the position or orientation of the tracker 12 estimated by the position estimating section 86 to the tracker 12. Here, the position transmitting section 88 transmits data indicating the position or orientation of the tracker 12a estimated by the position estimating section 86 to the tracker 12a, for example. Furthermore, the position transmitting section 88 transmits data indicating the position or orientation of the tracker 12b estimated by the position estimating section 86 to the tracker 12b, for example.

[0119] Here, the position transmitting section 88 may additionally transmit data indicating the position or orientation of the tracker 12b in the public coordinate system to the tracker 12a. In addition, the position transmitting section 88 may additionally transmit data indicating the position or orientation of the tracker 12a in the public coordinate system to the tracker 12b.

[0120] In the present embodiment, for example, the adding / updating section 90 performs addition and update of points included in the point cloud based on the feature point data received by the feature point data receiving section 84 as described above. Fig.10 As shown, based on the common point cloud 50 and the first feature point data, a point 52 arranged at a position associated with the position of the feature point included in the first feature point data in the common coordinate system may be added to the common point cloud 50. Fig.15 As shown, based on the common point cloud 50 and the second feature point data, points 52 arranged at positions associated with the positions of the feature points included in the second feature point data in the common coordinate system may be added to the common point cloud 50 .

[0121] Furthermore, the adding / updating section 90 may determine whether some of the points 52 included in the common point cloud 50 coincide with some of the points 56 included in the first individual point cloud 54. Then, when it is determined that there is such a coincidence, the points 52 corresponding to the some of the points 56 included in the first individual point cloud 54 may be added to the common point cloud 50.

[0122] Here, for example, when the above-mentioned addition of point 52 to the common point cloud 50 has been performed, the adding / updating part 90 may update the value of the reference point cloud data in the reference management data including the tracker ID of the tracker 12a to the "common point cloud". In this case, before the above-mentioned addition of point 52 to the common point cloud 50 is performed, the position estimating part 86 estimates the position or orientation of the tracker 12a in the first individual coordinate system based on the first individual point cloud 54 and the first feature point data. Furthermore, after the above-mentioned addition of point 52 to the common point cloud 50 is performed, the position estimating part 86 estimates the position or orientation of the tracker 12a in the common coordinate system based on the common point cloud 50 and the first feature point data. Therefore, the coordinate system representing the estimated position or orientation of the tracker 12a is switched from the first individual coordinate system to the common coordinate system without explicit operation by the user.

[0123] Similarly, the adding / updating section 90 may determine whether some of the points 52 included in the common point cloud 50 coincide with some of the points 60 included in the second individual point cloud 58. Then, when it is determined that there is such a coincidence, the points 52 corresponding to the some of the points 60 included in the second individual point cloud 58 may be added to the common point cloud 50.

[0124] Here, for example, when the above-mentioned addition of point 52 to the common point cloud 50 has been performed, the adding / updating part 90 may update the value of the reference point cloud data in the reference management data including the tracker ID of the tracker 12b to the "common point cloud". In this case, before the above-mentioned addition of point 52 to the common point cloud 50 is performed, the position estimating part 86 estimates the position or orientation of the tracker 12b in the second individual coordinate system based on the second individual point cloud 58 and the second feature point data. Furthermore, after the above-mentioned addition of point 52 to the common point cloud 50 is performed, the position estimating part 86 estimates the position or orientation of the tracker 12b in the common coordinate system based on the common point cloud 50 and the second feature point data. Therefore, the coordinate system representing the estimated position or orientation of the tracker 12b is switched from the second individual coordinate system to the common coordinate system without explicit operation by the user.

[0125] In the present embodiment, for example, the sensed data acquisition section 92 acquires sensed data generated by the sensor section 40. Here, the sensed data acquisition section 92 may acquire a captured image, direction data, data indicating latitude and longitude, and the like.

[0126] In the present embodiment, for example, the feature point data generating section 94 generates feature point data based on the sensing data acquired by the sensing data acquiring section 92. Here, as described above, the feature point data generating section 94 may extract a plurality of feature points from a captured image and generate feature point data indicating relative positions of the extracted feature points.

[0127] In the present embodiment, for example, the feature point data transmission section 96 transmits the feature point data generated by the feature point data generation section 94 to the server 10. Here, the feature point data transmission section 96 may also transmit to the server 10 a captured image, direction data, data indicating latitude and longitude, and the like.

[0128] In the present embodiment, for example, the position receiving section 98 receives data indicating a position or a direction transmitted by the position transmitting section 88 of the server 10 .

[0129] In the present embodiment, for example, the process execution section 100 executes a process based on the position or direction indicated by the data received by the position reception section 98. As described above, various processes can be executed in the process execution section 100, such as processes in a game and navigation processes.

[0130] Furthermore, as described above, the point cloud storage portion 80 may store a plurality of point clouds, each point cloud being associated with a region.

[0131] Then, the feature point data transmission part 96 may acquire position information indicating the position of the tracker 12 and transmit the position information to the server 10. Here, the position information may be, for example, information indicating the latitude and longitude of the tracker 12 measured by the GPS module included in the sensor part 40 of the tracker 12. In addition, the position information may be generated based on identification information (SSID) of a Wi-Fi (registered trademark) access point, information of a base station for a mobile phone, information provided by a location service, or the like.

[0132] Then, the position estimation section 86 may acquire the position information transmitted from the tracker 12. Then, the position estimation section 86 may select a point cloud associated with the area including the position indicated by the position information transmitted from the tracker 12 from among the plurality of point clouds stored in the point cloud storage section 80.

[0133] Then, the position estimating portion 86 may estimate the position or orientation of the tracker 12 based on the selected point cloud and feature point data indicating relative positions of feature points generated based on the sensing data acquired by the tracker 12 .

[0134] The position transmitting portion 88 may then transmit data indicating the estimated position or orientation of the tracker 12 to the tracker 12. The position receiving portion 98 of the tracker 12 may then receive the data indicating the position or orientation of the tracker 12.

[0135] Here, the following will refer to Fig.18 The illustrated flowchart describes an example of a process performed by the tracker 12a and the server 10 according to the present embodiment. In this process example, the processing of S101 to S112 to be described below is repeatedly performed at a predetermined frame rate.

[0136] First, the sensing data acquisition section 92 of the tracker 12 a acquires a captured image captured by the camera of the sensor section 40 ( S101 ).

[0137] Then, the feature point data generation section 94 of the tracker 12a extracts a plurality of feature points from the captured image acquired during the processing of S101, and generates feature point data indicating the respective positions of the extracted feature points (S102).

[0138] Then, the feature point data transmission section 96 of the tracker 12a transmits the feature point data generated in the process of S102 to the server 10. Then, the feature point data reception section 84 of the server 10 receives the transmitted feature point data (S103). It is assumed here that the tracker ID of the tracker 12a that has transmitted the feature point data is associated with the transmitted feature point data.

[0139] Then, the position estimation section 86 of the server 10 identifies the tracker ID associated with the feature point data received in the process of S103. Then, the position estimation section 86 of the server 10 identifies the reference point cloud data associated with the identified tracker ID in the reference management data stored in the reference management data storage section 82 (S104).

[0140] Then, the position estimation section 86 of the server 10 estimates the position and orientation of the tracker 12a based on the point cloud represented by the reference point cloud data identified in the process of S104 and the feature point data received in the process of S103 (S105). The estimated position and orientation here are expressed using a coordinate system of a virtual space in which the point cloud represented by the reference point cloud data identified in the process of S104 is arranged.

[0141] Then, the position transmitting section 88 of the server 10 transmits data indicating the position and direction of the tracker 12a estimated in the process of S105 to the tracker 12a. Then, the position receiving section 98 of the tracker 12a receives the data (S106).

[0142] Then, the process execution section 100 of the tracker 12a executes a predetermined process based on the data indicating the position and direction of the tracker 12a received in the process of S106 (S107).

[0143] Then, the adding / updating section 90 of the server 10 adds points to the point cloud used in the process of S105 based on the feature point data received in the process of step S103, and updates the points included in the point cloud (S108). Note that the process of S108 may be performed after the process of S106 is performed.

[0144] Then, the adding / updating section 90 of the server 10 determines whether the point cloud represented by the reference point cloud data identified in the process of S104 is the public point cloud 50 ( S109 ).

[0145] If it is determined that the point cloud represented by the reference point cloud data identified in the process of S104 is the common point cloud 50 ( S109 : YES), the processing loop in this process example is ended.

[0146] Assume that the point cloud represented by the reference point cloud data identified in the process of S104 is determined not to be the public point cloud 50 (S109: No). In this case, the adding / updating section 90 of the server 10 compares the point cloud processed by S108 with the public point cloud 50 to determine whether the point cloud can be integrated into the public point cloud 50 (S110). Here, for example, it can be determined whether some points 56 included in the public point cloud 50 are consistent with some points included in the point cloud processed by S108. Here, if it is determined that there is such a consistency, it can be determined that integration is possible, and if it is determined that there is no such consistency, it can be determined that integration is impossible.

[0147] Here, if it is determined that integration into the common point cloud 50 is impossible ( S110 : NO), the processing loop in this process example is ended.

[0148] Meanwhile, it is assumed that it is determined that it can be integrated into the common point cloud 50 (S110: Yes). In this case, the adding / updating section 90 adds points corresponding to some points included in the point cloud processed by S108 to the common point cloud 50 (S111). At this time, the updating of the points included in the common point cloud 50 can be performed.

[0149] Then, the adding / updating section 90 updates the value of the reference point cloud data in the reference management data including the tracker ID of the tracker 12a to "public point cloud" (S112), and ends the processing loop in this process example.

[0150] Note that in the process of S105 in the above-described process example, the position estimation section 86 fails to estimate the position and orientation of the tracker 12a based on the first individual point cloud 54 and the feature point data received in the process of S103. In this case, the position estimation section 86 may estimate the position and orientation of the tracker 12a based on the common point cloud 50 and the feature point data received in the process of S103.

[0151] The processing in the above-described process example is also performed by the tracker 12 b and the server 10 .

[0152] Note that the present invention is not limited to the above-described embodiments.

[0153] For example, the roles of the server 10 and the tracker 12 may be divided therebetween in a manner different from that described above. For example, the feature point data generating section 94 of the tracker 12 may be implemented in the server 10 instead. In this case, the feature point data transmitting section 96 of the tracker 12 may transmit the sensing data such as the captured image acquired by the sensing data acquiring section 92 to the server 10. Then, the feature point data generating section 94 in the server 10 may generate feature point data based on the sensing data received from the tracker 12.

[0154] Furthermore, for example, the tracking device 12 may be implemented in place of the server 10. Fig.16 The functions of the point cloud storage section 80, the reference management data storage section 82, the position estimation section 86 and the adding / updating section 90 are shown.

[0155] It should also be noted that the specific character strings and numerical values ​​described above and those shown in the drawings are merely examples, and these character strings and numerical values ​​are not limitative.

Claims

1. A position estimation device, include: A point cloud storage part, which stores a public point cloud using a public coordinate system to represent the position; as well as an estimating portion, which estimates the position or orientation of the first tracker in the common coordinate system and the position or orientation of the second tracker in the common coordinate system, wherein The estimating section estimates the position or orientation of the first tracker in the common coordinate system based on the common point cloud and first feature point data, the first feature point data indicating relative positions of a plurality of feature points and generated based on first sensing data acquired by the first tracker, and the estimating section estimates the position or orientation of the second tracker in the common coordinate system based on the common point cloud and second feature point data, the second feature point data indicating relative positions of a plurality of feature points and generated based on second sensing data acquired by the second tracker; wherein the point cloud storage portion further stores a separate point cloud representing a position using a separate coordinate system different from the common coordinate system, The position estimation device further comprises: an adding part, when it is determined that some of the plurality of points included in the common point cloud are consistent with some of the points included in the separate point cloud, adding points corresponding to the some of the points included in the separate point cloud to the common point cloud, Before performing adding the points corresponding to the some points included in the separate point cloud to the common point cloud, the estimating section estimates the position or orientation of the first tracker in the separate coordinate system based on the separate point cloud and the first feature point data, and After performing adding the points corresponding to the some points included in the individual point clouds to the common point cloud, the estimating section estimates a position or orientation of the first tracker in the common coordinate system based on the common point cloud and the first feature point data.

2. The position estimation device according to claim 1, further comprising: include: Sending part, where The transmitting section transmits data indicating the position or orientation of the first tracker in the common coordinate system to the first tracker, and The transmitting section transmits data indicating the position or orientation of the second tracker in the common coordinate system to the second tracker.

3. The position estimation device according to claim 2, in, The transmitting portion also transmits data indicating the position or orientation of the second tracker in the common coordinate system to the first tracker, and The transmitting portion also transmits data indicating the position or orientation of the first tracker in the common coordinate system to the second tracker.

4. The position estimation device according to claim 1, in: The adding part adds one or more points arranged at one or more positions in the common coordinate system to the common point cloud based on the common point cloud and the first feature point data, and the one or more positions are associated with one or more positions of one or more of the feature points included in the first feature point data.

5. The position estimation device according to any one of claims 1 to 4, further comprising: include: The receiving part, The receiving section receives the first feature point data from the first tracker, The receiving section receives the second feature point data from the second tracker, The estimating section estimates the position or orientation of the first tracker in the common coordinate system based on the first feature point data received by the receiving section, and The estimating section estimates the position or orientation of the second tracker in the common coordinate system based on the second feature point data received by the receiving section.

6. The position estimation device according to any one of claims 1 to 4, further comprising: include: Receiving part; as well as a feature point data generating section, which generates the first feature point data and the second feature point data, wherein The receiving part receives the first sensing data from the first tracker, The receiving part receives the second sensing data from the second tracker, The feature point data generating section generates the first feature point data based on the received first sensing data, The feature point data generating section generates the second feature point data based on the received second sensing data, The estimating section estimates the position or orientation of the first tracker in the common coordinate system based on the first feature point data generated by the feature point data generating section, and The estimating section estimates the position or orientation of the second tracker in the common coordinate system based on the second feature point data generated by the feature point data generating section.

7. A method for estimating a position, The following steps are involved: estimating a position or orientation of a first tracker in a common coordinate system based on a common point cloud representing a position using the common coordinate system and first feature point data, the first feature point data indicating relative positions of a plurality of feature points and generated based on first sensing data acquired by the first tracker; estimating a position or orientation of a second tracker in the common coordinate system based on the common point cloud and second feature point data, the second feature point data indicating relative positions of a plurality of feature points and generated based on second sensing data acquired by the second tracker; storing separate point clouds representing positions using separate coordinate systems different from the common coordinate system, When it is determined that some of the plurality of points included in the common point cloud coincide with some of the points included in the separate point clouds, adding points corresponding to the some of the points included in the separate point clouds to the common point cloud, Before performing the adding of the points corresponding to the some points contained in the separate point cloud to the common point cloud, estimating the position or orientation of the first tracker in the separate coordinate system based on the separate point cloud and the first feature point data, and After adding the points corresponding to the some points included in the individual point clouds to the common point cloud, a position or orientation of the first tracker in the common coordinate system is estimated based on the common point cloud and the first feature point data.

8. A non-transitory computer-readable storage medium having a program stored thereon, the program causing a computer to execute the following steps: estimating a position or orientation of a first tracker in a common coordinate system based on a common point cloud representing a position using the common coordinate system and first feature point data, the first feature point data indicating relative positions of a plurality of feature points and generated based on first sensing data acquired by the first tracker; estimating a position or orientation of a second tracker in the common coordinate system based on the common point cloud and second feature point data, the second feature point data indicating relative positions of a plurality of feature points and generated based on second sensing data acquired by the second tracker; storing separate point clouds representing positions using separate coordinate systems different from the common coordinate system, When it is determined that some of the plurality of points included in the common point cloud coincide with some of the points included in the separate point clouds, adding points corresponding to the some of the points included in the separate point clouds to the common point cloud, Before performing the adding of the points corresponding to the some points contained in the separate point cloud to the common point cloud, estimating the position or orientation of the first tracker in the separate coordinate system based on the separate point cloud and the first feature point data, and After adding the points corresponding to the some points included in the individual point clouds to the common point cloud, a position or orientation of the first tracker in the common coordinate system is estimated based on the common point cloud and the first feature point data.