Position estimation device, information terminal, position estimation program, and position estimation method

Abstract

To provide a position estimation device for estimating the position of an information terminal, in which expenses required for creation of point cloud arrangement information to be used for position estimation and expenses due to the capacity of a memory into which the point cloud arrangement information is written are suppressed.SOLUTION: The position estimation device includes: a communication circuit for performing data communication to an information terminal over a network; a local map database for storing data on a plurality of local maps; a local map selection unit for selecting one local map from among the plurality of local maps in accordance with information obtained based on the wave motion received from external apparatuses by the information terminal; a point cloud search unit for extracting a plurality of feature points from measurement data obtained from a sensor unit of the information terminal and searching for a point cloud which the plurality of feature points match from among the plurality of point clouds arranged in the selected local map; and a position estimation unit for estimating the position of the information terminal on the basis of the positional relationship between the plurality of feature points and the searched point cloud.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present invention relates to a position estimation device, an information terminal, a position estimation program, a position estimation method, and the like, used in a position estimation system that estimates the position of a portable or mobile information terminal. [Background technology]

[0002] For example, there are cases where it is necessary to know the location of an information terminal such as a smartphone carried by a security guard patrolling inside a building for security purposes. GNSS (Global Navigation Satellite System) is used as a position measurement technology for smartphones, and it is also an essential element technology for autonomous driving of moving objects such as automobiles. However, in fields where radio waves transmitted from positioning satellites are blocked by buildings, deterioration of the accuracy of position measurement is unavoidable.

[0003] Furthermore, in logistics operations such as transporting parts and materials within factories and transporting goods from warehouses to trucks, there is an increasing demand for mobile robots called AGVs (Automated Guided Vehicles) from the perspective of labor saving and efficiency. Conventional AGVs mainly use a guide system in which a running path is formed by attaching magnetic tape or the like to the floor, and introducing AGVs requires a great deal of effort and cost.

[0004] Therefore, by using a technology called SLAM (Simultaneous Localization and Mapping), which uses a mobile object equipped with a sensor to simultaneously estimate its own position and understand the structure of the surrounding environment, a guide becomes unnecessary and it becomes possible to easily change the route. SLAM technology mainly includes Visual SLAM, which uses a camera as a sensor, and LiDAR SLAM, which uses a laser sensor called LiDAR (Light Detection and Ranging) as a sensor.

[0005] Visual SLAM, which uses a camera, has the problem of being easily affected by brightness. On the other hand, LiDAR SLAM, which uses a laser sensor, measures the shape of surrounding structures by shining a laser beam on them, so it is less affected by brightness and can measure in dark places. However, it generally uses a sensor that scans only horizontally, and the information obtained is limited to two-dimensional position information. Three-dimensionalization is also possible, but a special mechanism is required.

[0006] As a related technique, the following Patent Document 1 discloses a position estimation device capable of improving the accuracy of self-position estimation while suppressing an increase in cost. This position estimation device includes a first position estimation unit that acquires image data output from a camera mounted on a moving body on a frame-by-frame basis, extracts feature points corresponding to other frames from the image data for each frame, and estimates the three-dimensional coordinates of the moving body in the frame based on the extracted feature points, a second position estimation unit that acquires depth data continuously output from a two-dimensional depth sensor mounted on the moving body and estimates the two-dimensional coordinates of the moving body using the acquired depth data, and a correction processing unit that determines whether a setting condition for the estimated two-dimensional coordinates of the moving body is satisfied, and corrects the three-dimensional coordinates of the moving body using the two-dimensional coordinates of the moving body if the setting condition is satisfied. Here, the depth means the distance from the two-dimensional depth sensor to the target object. [Prior art documents] [Patent documents]

[0007] [Patent Document 1] JP 2023-4517 A (paragraphs 0011-0012, 0024, 0066, Figures 1-2) DISCLOSURE OF THEINVENTION [Problem to be solved by the invention]

[0008] The position estimation device of Patent Document 1 does not require the use of a costly three-dimensional depth sensor, for example, a three-dimensional LiDAR sensor, and therefore reduces the cost increase compared to the case of using such a sensor. However, in order to estimate the position of a moving body based on depth data output from a two-dimensional depth sensor mounted on the moving body, it is necessary to create in advance a map for autonomous driving (two-dimensional environmental map) of the area where the position estimation is to be performed.

[0009] Generally, to estimate the position of a moving object or the like in a certain area (e.g., a rectangular area), it is necessary to create point cloud arrangement information (environment map) for that area in advance and write the point cloud arrangement information into memory when performing position estimation. Therefore, as the area in which position estimation is performed becomes larger, the amount of point cloud arrangement information used for position estimation also increases, which increases the cost required to create the point cloud arrangement information and the cost associated with the memory capacity for writing the point cloud arrangement information, resulting in an increase in facility costs.

[0010] In view of the above, a first object of the present invention is to reduce facility costs in a position estimation device that estimates the position of a portable or mobile information terminal by suppressing the cost required for creating point cloud arrangement information used to estimate the position of the information terminal and the cost associated with the memory capacity for writing the point cloud arrangement information. A second object of the present invention is to reduce the rate of error in estimated position that may occur when estimating the position of an information terminal. A third object of the present invention is to provide an information terminal that enables position estimation while reducing facility costs in such a position estimation device. [Means for solving the problem]

[0011] In order to solve at least part of the above problems, a position estimation device according to a first aspect of the present invention is a position estimation device that estimates a position of an information terminal having a sensor unit including an image sensor or a laser range sensor and capable of receiving waves transmitted within a predetermined range from an external device, and includes: a communication circuit that performs data communication with the information terminal via a network; a local map database that stores data of a plurality of local maps including point cloud arrangement information in a plurality of specific areas; at least one local map selection unit including a local map selection unit that selects one local map from a plurality of local maps stored in the local map database in accordance with information obtained based on the waves received by the information terminal from the external device; a point cloud search unit that extracts a plurality of feature points from measurement data obtained from a sensor unit of the information terminal and searches for a point cloud to which the plurality of feature points match from a plurality of point clouds arranged on the selected local map; and a position estimation unit that estimates the position of the information terminal based on a positional relationship between the plurality of feature points extracted from the measurement data obtained from the sensor unit of the information terminal and the point cloud searched by the point cloud search unit.

[0012] Here, the position estimation device may further include a local map setting unit that reads out from the local map database one local map selected by the at least one local map selection unit, or reads out one local map from among the multiple local maps selected by the at least one local map selection unit in accordance with a priority order, and writes the read local map into a memory, and the point cloud search unit may search for a point cloud that matches the multiple feature points from among the multiple point clouds located on the local map written into the memory.

[0013] Furthermore, a position estimation program according to a first aspect of the present invention is a position estimation program used to estimate a position of an information terminal in a position estimation device that has a sensor unit including an image sensor or a laser range sensor and that performs data communication via a network with an information terminal capable of receiving waves transmitted within a predetermined range from an external device, and causes a CPU to execute the following steps: a step (a) of selecting one local map from a plurality of local maps that are stored in a local map database and include point cloud arrangement information in a plurality of specific areas, in accordance with information obtained based on the waves received by the information terminal from the external device; a step (b) of extracting a plurality of feature points from measurement data obtained from the sensor unit of the information terminal, and searching for a point cloud to which the plurality of feature points match from a plurality of point clouds arranged on the selected local map; and a step (c) of estimating the position of the information terminal based on the positional relationship between the plurality of feature points extracted from the measurement data obtained from the sensor unit of the information terminal and the point cloud searched for in step (b).

[0014] Furthermore, a position estimation method according to a first aspect of the present invention is a position estimation method for estimating a position of an information terminal in a position estimation device that has a sensor unit including an image sensor or a laser range sensor and is capable of receiving waves transmitted within a predetermined range from an external device and that communicates data via a network with an information terminal, the method comprising: a step (a) of selecting one local map from a plurality of local maps that are stored in a local map database and include point cloud arrangement information in a plurality of specific areas, in accordance with information obtained based on the waves received by the information terminal from the external device; a step (b) of extracting a plurality of feature points from measurement data obtained from the sensor unit of the information terminal, and searching for a point cloud to which the plurality of feature points match from a plurality of point clouds arranged on the selected local map; and a step (c) of estimating the position of the information terminal based on a positional relationship between the plurality of feature points extracted from the measurement data obtained from the sensor unit of the information terminal and the point cloud searched for in step (b).

[0015] An information terminal according to a second aspect of the present invention is an information terminal that performs data communication with a position estimation device, and includes a sensor unit that includes an image sensor or a laser range sensor and measures surrounding objects to generate measurement data, a short-range communication circuit that receives a beacon wave transmitted from a beacon device and outputs a beacon signal, a wireless LAN communication circuit that receives wireless LAN radio waves transmitted from a wireless LAN access point and outputs a wireless LAN signal, a GNSS receiving circuit that receives radio waves transmitted from three or more positioning satellites and outputs a received signal, and a barometer that measures air pressure and outputs an air pressure measurement signal, and an inertial measurement circuit that detects three-dimensional inertial motion of the information terminal and outputs a detection signal. the position estimation device, and a control circuit that controls the sensor unit to perform continuous measurements when the position estimation device performs a position estimation operation, and controls the wireless communication circuit to transmit to the position estimation device at least one selected from the measurement data obtained from the sensor unit, ID information of a beacon device obtained from the beacon signal, ID information of a wireless LAN access point obtained from the wireless LAN signal, position information obtained from a received signal of the GNSS receiving circuit, attitude information obtained from a detection signal of the inertial measurement device, and atmospheric pressure information obtained from the atmospheric pressure measurement signal. Effect of the Invention

[0016] According to a first aspect of the present invention, in a position estimation device that estimates the position of a portable or mobile information terminal, point cloud arrangement information (environmental map) used to estimate the position of the information terminal is divided into multiple local maps of specific areas and stored in a database, and the information terminal selects one local map based on waves (environmental information) received from an external device, thereby suppressing the costs required to create the point cloud arrangement information and the costs associated with the memory capacity for writing the point cloud arrangement information, thereby reducing equipment costs.

[0017] For example, if point cloud arrangement information is required for buildings A and B, which are located some distance from each other, creating a local map of building A and a local map of building B rather than creating an environmental map of a large rectangular area that includes buildings A and B can reduce the costs required to create the point cloud arrangement information and the costs associated with the memory capacity for writing the point cloud arrangement information.

[0018] In addition, by narrowing down the search area from a wide area to a relatively narrow area based on the measurement data obtained from the sensor unit of the information terminal, it is possible to reduce the rate of error in the estimated position that may occur when estimating the position of the information terminal. Furthermore, according to the second aspect of the present invention, it is possible to provide an information terminal that enables position estimation while reducing the equipment cost in such a position estimation device. [Brief description of the drawings]

[0019] [Figure 1] 1 is a block diagram showing an example of the configuration of a position estimation system including a position estimation device according to an embodiment of the present invention. [Diagram 2] 2 is a block diagram showing an example of the configuration of a first information terminal shown in FIG. 1. [Diagram 3] 2 is a block diagram showing an example of the configuration of the position estimation device shown in FIG. 1. [Figure 4] FIG. 11 is a diagram showing an example of information included in local map data. [Diagram 5] 4 is a flowchart showing an example of the operation of a first local map selection unit shown in FIG. 3. [Figure 6] 4 is a schematic diagram for explaining an example of the operation of a first local map selection unit shown in FIG. 3; FIG. [Figure 7] 4 is a flowchart showing an example of the operation of a second local map selection unit shown in FIG. 3. [Figure 8] 10 is a flowchart showing an example of the operation of a fourth local map selection unit shown in FIG. 3. [Figure 9] 4 is a flowchart showing an example of the operation of a local map setting unit shown in FIG. 3; [Figure 10]4 is a flowchart showing an example of the operation of a point cloud searching unit shown in FIG. 3. [Figure 11] 4 is a schematic diagram for explaining an example of the operation of the point cloud searching unit shown in FIG. 3. [Figure 12] 4 is a flowchart showing an example of the operation of a position estimation unit shown in FIG. 3. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0020] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. Note that the same components are given the same reference numerals and duplicated explanations will be omitted.

[0021] <Location Estimation System> Fig. 1 is a block diagram showing an example of the configuration of a position estimation system including a position estimation device according to an embodiment of the present invention. As shown in Fig. 1, this position estimation system includes at least one information terminal (in Fig. 1, a first information terminal 10a and a second information terminal 10b are shown as examples), a position estimation device 20 according to an embodiment of the present invention, and a position information processing device 30. The first information terminal 10a to the position information processing device 30 have a function of communicating with each other via a network such as a mobile communication network or the Internet.

[0022] The first information terminal 10a has a sensor unit including an image sensor or a laser range sensor, and is capable of receiving waves (radio waves, light, etc.) transmitted from an external device within a predetermined range. For example, the first information terminal 10a may be a mobile device such as a smartphone or tablet terminal that has an imaging function and a communication function and is easy to carry. Alternatively, the first information terminal 10a may be mounted on a moving object such as an unmanned guided vehicle, an automatic delivery vehicle, a self-propelled vehicle, or a robot used in the manufacture of goods at a manufacturing company.

[0023] The position estimation device 20 is, for example, a position estimation server installed in a data center or the like to provide a position estimation service. In the example shown in Fig. 1, the position estimation device 20 estimates the current position of the first information terminal 10a based on measurement data and various measurement information transmitted from the first information terminal 10a. At that time, the position estimation device 20 may use the measurement information transmitted from the second information terminal 10b to estimate the current position of the first information terminal 10a.

[0024] The position information processing device 30 is installed, for example, in a control center of a security company or a manufacturing company and is used to monitor the position estimation results. The position information processing device 30 may be configured with a general PC (personal computer) and display, or may be configured similarly to the position estimation device 20 and include various databases used for position estimation. In the latter case, the position information processing device 30 also becomes capable of performing position estimation.

[0025] For example, a security guard belonging to a security company carries the first information terminal 10a when guarding a building or the like, and the position estimation device 20 estimates the current position of the security guard. In this case, the estimated current position is displayed on the display of the position information processing device 30 installed in a control center or the like of the security company.

[0026] Furthermore, by having another security guard carry the second information terminal 10b, the location information of two security guards may be displayed simultaneously on the display. Alternatively, the second information terminal 10b may be placed on a specific floor of a specific building and used as a reference device for measuring a reference atmospheric pressure at a reference altitude.

[0027] As another example, the position estimation device 20 may estimate the current position of the first information terminal 10a mounted on a moving body such as an automatic guided vehicle or an automatic delivery vehicle. In this case, the estimated current position is displayed on a display of a position information processing device 30 installed in a control center of a manufacturing company or a shipping company.

[0028] <Information terminal> Fig. 2 is a block diagram showing a configuration example of the first information terminal shown in Fig. 1. As shown in Fig. 2, the first information terminal 10a includes a sensor unit 11, a short-range communication circuit 12, a wireless LAN (Local Area Network) communication circuit 13, a GNSS (Global Navigation Satellite System) receiving circuit 14, an inertial measurement unit (IMU) 15, a wireless communication circuit 16, and a control circuit 17.

[0029] The second information terminal 10b may be configured similarly to the first information terminal 10a. Some of the components shown in Figures 1 and 2 may be omitted or changed, or other components may be added to the components shown in Figures 1 and 2.

[0030] The sensor unit 11 is, for example, configured as the camera portion of a smartphone, and includes a plurality of lenses that collect light to form an image, and an image sensor having a number of pixel sensors that convert the formed image into an electrical signal and extract it. The camera captures an object such as a structure existing in the vicinity, and generates image data that represents the brightness or RGB value of each pixel. Note that two cameras (stereo cameras) may be used to generate three-dimensional image data.

[0031] Alternatively, the sensor unit 11 may include a laser range sensor instead of a camera. For example, a laser range sensor mounted on a moving object projects a laser beam within a predetermined angle range from the moving direction of the moving object to measure the positions and shapes of objects such as structures present in the vicinity, thereby generating angle data indicating the direction in which the object exists and distance data indicating the distance to the object.

[0032] For example, a laser range sensor uses a laser diode to generate a pulsed laser beam and projects the laser beam into the surrounding space via a rotating mirror such as a polygon mirror, thereby scanning the surrounding space in an angular direction. The projection direction of the laser beam is changed at a constant angular step using an angular encoder. The laser range sensor also uses a photodiode to detect the reflected light.

[0033] The direction in which an object exists is calculated based on the output signal of the angle encoder when the reflected light is detected, and angle data is generated that represents the direction in which the object exists. The distance to the object is calculated based on the time between the projection of the pulsed laser beam and the detection of the reflected light, and distance data is generated that represents the distance to the object. Note that when two laser range sensors are used, even if multiple objects are arranged overlapping each other as viewed from one of the laser range sensors, it may be possible to separate and measure those objects.

[0034] The short-distance communication circuit 12 includes a transmission / reception circuit, a modem, and the like having a short-distance wireless data communication function that allows wireless data communication with another device only when the device is within a predetermined distance from the device. For this short-distance wireless data communication, a communication method such as Bluetooth (registered trademark), RFID (Radio Frequency Identifier), or infrared communication is used. With such a communication method, wireless data communication is possible within a maximum distance of about 10 m.

[0035] The wireless LAN communication circuit 13 includes a transmission / reception circuit and a modem having a wireless LAN communication function for transmitting and receiving wireless LAN radio waves to and from an external device, and is capable of wireless LAN communication with a wireless LAN access point. In this wireless LAN communication, for example, a communication method such as Wi-Fi (Wireless Fidelity) is used.

[0036] The GNSS receiver circuit 14 receives radio waves transmitted from three or more (generally four) positioning satellites of the global positioning system. The radio waves transmitted from the positioning satellites contain information on the time of transmission, so the radio wave propagation time can be determined and the distance from the positioning satellite can be calculated. Therefore, the current position of the first information terminal 10a can be calculated based on the radio waves received by the GNSS receiver circuit 14.

[0037] The inertial measurement unit 15 includes, for example, a three-axis angular velocity sensor (gyro sensor) and a three-direction acceleration sensor, and detects three-dimensional inertial motion (translational motion and rotational motion in three orthogonal axial directions) of the first information terminal 10a. By using the results of measuring the angular velocity and acceleration of the first information terminal 10a with high accuracy, it is possible to grasp or estimate the attitude and trajectory of the first information terminal 10a. In this embodiment, the inertial measurement unit 15 has a barometer 151 to improve reliability, and the barometer 151 can also be used to estimate the position of the first information terminal 10a.

[0038] The wireless communication circuit 16 performs data communication via a network with the position estimation device 20 and the position information processing device 30 by performing wireless communication with, for example, a mobile phone base station. The control circuit 17 includes, for example, a CPU (Central Processing Unit), a storage constituted by a flash memory or the like, and a cache memory constituted by a RAM (Random Access Memory) or the like.

[0039] The CPU performs various calculations and data processing according to various software (including a position estimation client program used for position estimation) stored in the storage. When the position estimation device 20 performs a position estimation operation, the control circuit 17 cooperates with the position estimation device 20 to control the sensor unit 11 to the wireless communication circuit 16.

[0040] Under the control of the control circuit 17, the sensor unit 11 measures surrounding objects, generates measurement data, and outputs the measurement data to the control circuit 17. For example, the sensor unit 11 continuously measures objects such as structures present in the vicinity of the first information terminal 10a, and generates video data representing the position and shape of the objects, or angle data and distance data.

[0041] The short-range communication circuit 12, under the control of the control circuit 17, receives a beacon wave transmitted from a beacon device installed in or around the building, and outputs a beacon signal obtained by demodulating the wave to the control circuit 17. For example, radio waves such as BLE (Bluetooth Low Energy) Beacon or light such as infrared light are used as the beacon wave. The beacon signal includes ID information of the beacon device as device information for identifying each device, and the control circuit 17 obtains the ID information of the beacon device from the beacon signal.

[0042] Under the control of the control circuit 17, the wireless LAN communication circuit 13 receives wireless LAN radio waves transmitted from a wireless LAN access point installed in or around the building, demodulates the radio waves, and outputs the resulting wireless LAN signal to the control circuit 17. The wireless LAN signal includes ID information of the wireless LAN access point as device information for identifying each device, and the control circuit 17 obtains the ID information of the wireless LAN access point from the wireless LAN signal.

[0043] For example, Wi-Fi is used as a communication method for wireless LAN. A Wi-Fi access point has a function of wirelessly connecting to Wi-Fi compatible devices as a base station of Wi-Fi radio waves. The Wi-Fi radio waves contain an SSID (Service Set Identifier) ​​or BSSID (Basic Service Set Identifier) ​​as ID information of the Wi-Fi access point. The SSID is an identifier for distinguishing a wireless LAN from other networks, and is also used as a hotspot name.

[0044] A wireless LAN can also be divided into multiple sub-networks that require different authentication methods. In the infrastructure mode of a wireless LAN, a network consisting of one access point and the subordinate wireless LAN client terminals within the radio waves of that access point is called a BSS. The BSSID is the identifier of the BSS. Note that since both the SSID and the BSSID can be set arbitrarily, they are not completely unique identifiers, but they can function to distinguish a wireless LAN or sub-network from surrounding networks, either individually or in combination.

[0045] The GNSS receiving circuit 14, under the control of the control circuit 17, receives radio waves transmitted from three or more (generally four) positioning satellites of the global positioning system, and outputs the received signals to the control circuit 17. The control circuit 17 calculates position information relating to the current position of the first information terminal 10a based on the received signals.

[0046] Under the control of the control circuit 17, the inertial measurement device 15 detects three-dimensional inertial motion of the first information terminal 10a and outputs a detection signal to the control circuit 17. The control circuit 17 generates attitude information representing the attitude or trajectory of the first information terminal 10a based on the detection signal. In addition, under the control of the control circuit 17, the barometer 151 measures the air pressure at the current position of the first information terminal 10a and outputs an air pressure measurement signal to the control circuit 17. The control circuit 17 generates air pressure information representing the air pressure at the current position of the first information terminal 10a based on the air pressure measurement signal.

[0047] In this way, the control circuit 17 controls the sensor unit 11 to perform continuous measurements when the position estimation device 20 performs a position estimation operation, and controls the wireless communication circuit 16 to transmit to the position estimation device 20 at least one selected from the measurement data obtained from the sensor unit 11, the ID information of the beacon device obtained from the beacon signal, the ID information of the wireless LAN access point obtained from the wireless LAN signal, the position information obtained from the received signal of the GNSS receiving circuit 14, the attitude information obtained from the detection signal of the inertial measurement device 15, and the atmospheric pressure information obtained from the atmospheric pressure measurement signal of the barometer 151.

[0048] When a smartphone is used as the first information terminal 10a or the second information terminal 10b, a smartphone attachment device for self-location estimation may be attached to the smartphone so that a security guard or the like belonging to a security company can operate the smartphone normally without using his or her own hands. This smartphone attachment device includes a belt part that is attached around the waist of the human body and a storage part fixed to the belt part.

[0049] The smartphone faces the front of the human body when stored in the storage section. When the position estimation device 20 performs a position estimation operation, the control circuit 17 performs a control operation according to the position estimation client program, and the camera part of the smartphone operates to capture an image of an object such as a structure in front of the human body.

[0050] <Position estimation device> Fig. 3 is a block diagram showing an example of the configuration of the position estimation device shown in Fig. 1. As shown in Fig. 3, the position estimation device 20 includes an operation unit 21, a display unit 22, a voice input / output unit 23, a communication circuit 24, an interface 25, a CPU (Central Processing Unit) 26, a storage unit 27, and a cache memory 28. The interface 25 to the cache memory 28 are connected to each other via a bus line. Note that some of the components shown in Fig. 3 may be omitted or modified, or other components may be added to the components shown in Fig. 3.

[0051] The operation unit 21 is composed of, for example, a keyboard, a mouse, etc., and is used to input various commands and data to the position estimation device 20. The display unit 22 includes, for example, an LCD (liquid crystal display), etc., and displays an operation screen, etc. The audio input / output unit 23 includes, for example, a microphone, an amplifier, a speaker, etc., and converts an audio signal into an electrical signal and converts an electrical signal into an audio signal.

[0052] The communication circuit 24 has the function of performing data communication between the first information terminal 10a, the second information terminal 10b, and the position information processing device 30 via a network such as a wired LAN, a wireless LAN, a mobile communication network, or the Internet.

[0053] The interface 25 is connected to the operation unit 21 to the communication circuit 24, and transmits various commands and data between them and the CPU 26. The CPU 26 performs various calculations and data processing in accordance with various software (including a position estimation server program used for position estimation) stored in the storage unit 27. In addition, the position estimation device 20 or the position information processing device 30 can integrate or link its own application with applications or web services of other providers using an API (Application Programming Interface).

[0054] A hard disk, a flexible disk, an optical magnetic disk, a magnetic tape, a ROM (read only memory), a CD-ROM, a DVD-ROM, or the like can be used as a recording medium (storage medium) in the storage unit 27. The cache memory 28 is composed of a RAM, or the like. In addition to the above software, the storage unit 27 stores various data used for position estimation, and the data is read out and written to the cache memory 28 as necessary.

[0055] Here, the CPU 26 and the location estimation server program stored in the storage unit 27 configure a communication control unit 260, a first local map selection unit 261, a second local map selection unit 262, a third local map selection unit 263, a fourth local map selection unit 264, a local map setting unit 265, a point cloud search unit 266, and a location estimation unit 267 as functional blocks.

[0056] The communication control unit 260 controls the communication operation of the communication circuit 24. When the position estimation device 20 performs a position estimation operation, the communication circuit 24 outputs the measurement data, the ID information of the beacon device, the ID information of the wireless LAN access point, the position information based on the radio wave reception of the GNSS, the attitude information based on the inertial measurement, or the atmospheric pressure information based on the atmospheric pressure measurement received from the first information terminal 10a or the second information terminal 10b to the communication control unit 260.

[0057] The first local map selection unit 261 to the position estimation unit 267 estimate the position of the first information terminal 10a by using measurement data and various information acquired from the first information terminal 10a or the second information terminal 10b by the communication circuit 24 and the communication control unit 260.

[0058] The position estimation method used in the position estimation system shown in FIG. 1 uses a global map that includes geographical information on a wide area together with latitude and longitude coordinate information (global coordinate system), and a local map that includes point cloud arrangement information (local coordinate system) arranged in a specific area smaller than the area of ​​the global map.

[0059] For example, the global map may be Google Maps including global geographical information, and the local map may be a local map including point cloud arrangement information in a specific area such as the interior or site of a specific building. Furthermore, image data representing a map image (e.g., a floor plan or illustration of a building) corresponding to the local map may be used to display the estimated position on the local map.

[0060] 4 is a diagram showing an example of information included in the local map data. For example, the local map data may include the following information: (1) Image coordinate system (IX, IY) (unit: pixel) Image width IW, image length IL, local coordinate system origin position on the image (IX1, IY1) (2) Local coordinate system (LX, LY) (unit: m) Local map width LW, local map length LL (3) Global coordinate system Latitude and longitude on the global map of the origin of the local coordinate system The angle of the global map when the north in the Y-axis direction of the local coordinate system is zero (4)Altitude information Local map altitude information with the ground surface at zero For example, the floor of a building (1F, 2F, 3F, etc.) (5) Point cloud placement information Local coordinates or altitude information of points for position estimation that are pre-placed on a local map For example, the three-dimensional coordinates (Xij, Yij, Zij) of points A to E (i=1 to 5) (6) Local Map Identifier An identifier used to classify multiple local maps or to identify a specific local map For example, Building A-3F, connecting passage, etc. (It does not necessarily have to be textual information)

[0061] In this embodiment, the point cloud arrangement information is obtained by measuring a plurality of predetermined regions in advance using a sensor of the same type as the sensor unit of the first information terminal 10a or the second information terminal 10b, while identifying the position and orientation based on the movement direction and movement distance of the sensor. When an image sensor is used, the point cloud means a collection of feature points in an image of an object represented by image data generated by the image sensor. When a laser range sensor is used, the point cloud means a collection of feature points in the position and shape of an object represented by angle data and distance data generated by the laser range sensor.

[0062] 3, data of a plurality of local maps including point cloud arrangement information in a plurality of specific regions is stored in the local map database DBL of the storage unit 27. The first local map selection unit 261 to the third local map selection unit 263 select one local map from the plurality of local maps stored in the local map database DBL according to information obtained based on waves received by the first information terminal 10a from an external device.

[0063] For example, the first local map selection unit 261 and the second local map selection unit 262 acquire device information that identifies each device that transmits a wave received by the first information terminal 10a, and select one local map corresponding to the acquired device information. Also, the third local map selection unit 263 acquires location information obtained based on the GNSS radio waves received by the first information terminal 10a, and selects one local map according to the acquired location information.

[0064] On the other hand, the fourth local map selection unit 264 selects one local map according to altitude information of the first information terminal 10a obtained based on atmospheric pressure information obtained from the barometers of the first information terminal 10a and the second information terminal 10b, or selects one local map according to location information regarding latitude and longitude obtained based on the GNSS radio waves received by the first information terminal 10a in addition to the altitude information of the first information terminal 10a.

[0065] <First local map selection section> The first local map selection unit 261 acquires ID information of a beacon device obtained from radio waves or light of a beacon received by the first information terminal 10a, and selects one local map corresponding to the acquired ID information of the beacon device.

[0066] Fig. 5 is a flowchart showing an example of the operation of the first local map selection unit shown in Fig. 3. When the process starts in Fig. 5, the position information, flags, etc. previously set by the first local map selection unit 261 are reset.

[0067] In step S11, the first local map selection unit 261 acquires from the communication control unit 260 ID information of the beacon device obtained from the radio waves or light of the beacon received by the first information terminal 10a from the beacon device.

[0068] The first location information database DBP1 stores information on which beacon devices are installed in which locations in which buildings or sites. For example, the first location information database DBP1 stores local map identifiers that identify the local maps of the areas in which the beacon devices are installed, and the local coordinates (LX, LY) of the beacon devices on the local maps, corresponding to the ID information of the multiple beacon devices, as the location information of the beacon devices.

[0069] In step S12, the first local map selection unit 261 determines whether the ID information of the beacon device acquired in step S11 satisfies the local map selection condition. For example, the first local map selection unit 261 searches for the ID information of the beacon device stored in advance in the first location information database DBP1 using the acquired ID information of the beacon device as a search keyword. If the search is successful, it is determined that the ID information of the beacon device satisfies the local map selection condition.

[0070] If it is determined that the ID information of the beacon device does not satisfy the local map selection condition, the process proceeds to step S13. In step S13, the first local map selection unit 261 sets a map selection failure flag F1, which indicates that the local map selection based on the ID information of the beacon device has failed. Then, the process ends.

[0071] If it is determined that the ID information of the beacon device satisfies the local map selection condition, the process proceeds to step S14. In step S14, the first local map selection unit 261 reads out the local map identifier corresponding to the ID information of the beacon device from the first position information database DBP1, and selects one local map from among the multiple local maps stored in the local map database DBL.

[0072] In step S15, the first local map selection unit 261 outputs a local map identifier that identifies the local map selected in step S14, and outputs the local coordinates (LX, LY) of the beacon device read from the first location information database DBP1 as provisional location information. Then, the process ends.

[0073] Fig. 6 is a schematic diagram for explaining an example of the operation of the first local map selection unit shown in Fig. 3. Fig. 6 shows schematic perspective views of buildings A and B, and a schematic plan view of the fourth floor of building A. Beacon devices 41 and 42 that transmit radio waves of BLE (Bluetooth Low Energy) beacons are installed on the fourth floor of building A, and the ranges that the radio waves of the beacons can reach are shown by dashed lines.

[0074] The first information terminal 10a receives radio waves of a beacon transmitted from the beacon device 41, acquires ID information of the beacon device 41 from the received radio waves of the beacon, and transmits the ID information of the beacon device 41 to the position estimation device 20.

[0075] In the position estimation device 20, the first local map selection unit 261 reads one local map identifier from the first position information database DBP1 according to the ID information of the beacon device 41 acquired from the first information terminal 10a. Since each local map identifier stored in the first position information database DBP1 specifies one of the local maps stored in the local map database DBL, if the local map identifier can be read from the first position information database DBP1, the selection of the local map is successful.

[0076] In addition, since the first information terminal 10a receives a beacon radio wave from the beacon device 41, it can be estimated that the first information terminal 10a is located within the range of the beacon radio wave from the beacon device 41. Therefore, if the local coordinates (LX, LY) of the beacon device 41 can be read from the first location information database DBP1, the tentative location of the first information terminal 10a has been successfully estimated.

[0077] <Second local map selection section> The second local map selection unit 262 acquires ID information of a wireless LAN access point obtained from the wireless LAN radio waves received by the first information terminal 10a, and selects one local map corresponding to the acquired ID information of the wireless LAN access point.

[0078] Fig. 7 is a flowchart showing an example of the operation of the second local map selection unit shown in Fig. 3. When the process starts in Fig. 7, the position information, flags, and the like previously set by the second local map selection unit 262 are reset.

[0079] In step S21, the second local map selection unit 262 acquires, from the communication control unit 260, ID information of the wireless LAN access point obtained from the wireless LAN radio waves received by the first information terminal 10a from the wireless LAN access point.

[0080] The second location information database DBP2 stores in advance information about which wireless LAN access points are installed in which locations of which buildings or sites. For example, the second location information database DBP2 stores, in correspondence with the ID information of a plurality of wireless LAN access points, local map identifiers that identify the local maps of the areas in which the wireless LAN access points are installed, and the local coordinates (LX, LY) of the wireless LAN access points on those local maps as location information of the wireless LAN access points.

[0081] In step S22, the second local map selection unit 262 determines whether the ID information of the wireless LAN access point acquired in step S21 satisfies the local map selection condition. For example, the second local map selection unit 262 uses the acquired ID information of the wireless LAN access point as a search keyword to search for ID information of the wireless LAN access point stored in advance in the second location information database DBP2. If the search returns a hit, it is determined that the ID information of the wireless LAN access point satisfies the local map selection condition.

[0082] If it is determined that the ID information of the wireless LAN access point does not satisfy the local map selection condition, the process proceeds to step S23. In step S23, the second local map selection unit 262 sets a map selection failure flag F2, which indicates that the local map selection based on the ID information of the wireless LAN access point has failed. Then, the process ends.

[0083] If it is determined that the ID information of the wireless LAN access point satisfies the local map selection condition, the process proceeds to step S24. In step S24, the second local map selection unit 262 reads out a local map identifier corresponding to the ID information of the wireless LAN access point from the second location information database DBP2, and selects one local map from among the multiple local maps stored in the local map database DBL.

[0084] In step S25, the second local map selection unit 262 outputs a local map identifier that identifies the local map selected in step S24, and outputs the local coordinates (LX, LY) of the wireless LAN access point read from the second location information database DBP2 as provisional location information. Then, the process ends.

[0085] <Third local map selection section> The third local map selection unit 263 acquires location information based on the GNSS radio waves received by the first information terminal 10a, and selects one local map according to the acquired location information. When the process starts in the third local map selection unit 263, previously set location information, flags, and the like are reset.

[0086] The third local map selection unit 263 acquires from the communication control unit 260 location information relating to the current location of the first information terminal 10a, which is obtained from radio waves received by the first information terminal 10a from GNSS positioning satellites.

[0087] For example, the third position information database DBP3 stores local map identifiers that identify a plurality of local maps in association with information that identifies the coordinate ranges of those local maps.

[0088] The third local map selection unit 263 judges whether the acquired position information of the first information terminal 10a satisfies the local map selection condition. For example, when the position represented by the acquired position information of the first information terminal 10a is included in the coordinate range of any local map stored in the third position information database DBP3, it is judged that the position information of the first information terminal 10a satisfies the local map selection condition.

[0089] If it is determined that the location information of the first information terminal 10a does not satisfy the local map selection conditions, the third local map selection unit 263 sets a map selection failure flag F3, which indicates that local map selection based on GNSS radio wave reception has failed.

[0090] If it is determined that the location information of the first information terminal 10a satisfies the local map selection condition, the third local map selection unit 263 selects one local map from among the multiple local maps stored in the local map database DBL by reading out a local map identifier corresponding to the location information of the first information terminal 10a from the third location information database DBP3.

[0091] The third local map selection unit 263 outputs a local map identifier that identifies the selected local map, and also outputs the acquired location information of the first information terminal 10a as provisional location information.

[0092] <4th Local Map Selection Section> The fourth local map selection unit 264 calculates the relative altitude of the first information terminal 10a with respect to the second information terminal 10b based on atmospheric pressure information obtained from the barometer of the first information terminal 10a and atmospheric pressure information obtained from the barometer of the second information terminal 10b, and selects one local map according to the altitude information of the first information terminal 10a.

[0093] For example, the altitude of the first information terminal 10a is calculated based on the difference between the atmospheric pressure measured by the first information terminal 10a and the reference atmospheric pressure measured by the second information terminal 10b placed at the reference altitude, and altitude information on the floor where the first information terminal 10a is located is obtained. Alternatively, the fourth local map selection unit 264 selects one local map according to position information on latitude and longitude obtained based on the GNSS radio waves received by the first information terminal 10a, in addition to the altitude information of the first information terminal 10a.

[0094] Fig. 8 is a flowchart showing an example of the operation of the fourth local map selection unit shown in Fig. 3. When the process starts in Fig. 8, the position information, flags, and the like previously set by the fourth local map selection unit 264 are reset.

[0095] In step S41, the fourth local map selection unit 264 acquires from the communication control unit 260 atmospheric pressure information obtained from the barometer of the first information terminal 10a and atmospheric pressure information obtained from the barometer of the second information terminal 10b.

[0096] In step S42, the fourth local map selection unit 264 calculates the relative altitude of the first information terminal 10a with respect to the second information terminal 10b based on the atmospheric pressure information of the first information terminal 10a and the atmospheric pressure information of the second information terminal 10b acquired in step S41. As a result, the fourth local map selection unit 264 obtains altitude information related to the range of the floor in which the first information terminal 10a is located, and sets it as an altitude condition for selecting a local map.

[0097] Furthermore, if the altitude of the second information terminal 10b is known, the fourth local map selection unit 264 may calculate the absolute altitude of the first information terminal 10a based on the altitude of the second information terminal 10b and the relative altitude of the first information terminal 10a with respect to the second information terminal 10b. As a result, the fourth local map selection unit 264 obtains altitude information related to the floor where the first information terminal 10a is located, and sets it as an altitude condition for selecting a local map.

[0098] In step S43, the fourth local map selection unit 264 determines whether or not a floor satisfying the altitude condition set in step S42 exists in any of the local maps stored in the local map database DBL. For example, if the range of floors satisfying the altitude condition set is the fourth floor or higher, a local map having the fourth floor becomes a candidate for selection.

[0099] If no floor that satisfies the set altitude condition exists in any of the local maps, the process proceeds to step S44. In step S44, the fourth local map selection unit 264 sets a map selection failure flag F4, which indicates that the local map selection based on the atmospheric pressure information of the first information terminal 10a and the second information terminal 10b has failed. Then, the process ends.

[0100] If it is determined that a floor satisfying the set altitude condition exists in any of the local maps, the process proceeds to step S45. In step S45, the fourth local map selection unit 264 determines whether a floor satisfying the set altitude condition exists in multiple local maps.

[0101] If it is determined that a floor satisfying the set altitude condition exists in only one local map, the process proceeds to step S46. In step S46, the fourth local map selection unit 264 selects one local map in which a floor satisfying the set altitude condition exists, and outputs a local map identifier that identifies the selected local map, and also outputs, for example, the origin coordinates of the local map and altitude information related to the floor where the first information terminal 10a is located as provisional position information. Then, the process ends.

[0102] If it is determined that a floor satisfying the set altitude condition exists in a plurality of local maps, the process proceeds to step S47. In step S47, the fourth local map selection unit 264 acquires, as auxiliary location information, location information related to latitude and longitude obtained based on the GNSS radio waves received by the first information terminal 10a from a GNSS positioning satellite from the third local map selection unit 263.

[0103] The fourth local map selection unit 264 selects one local map from among a plurality of local maps that satisfy the set altitude condition according to the position information on latitude and longitude. For example, the fourth local map selection unit 264 selects one local map that includes an area closest to the position represented by the position information on latitude and longitude.

[0104] In step S48, the fourth local map selection unit 264 outputs a local map identifier that identifies the local map selected in step S47, and also outputs, for example, the origin coordinates of the local map and altitude information related to the floor where the first information terminal 10a is located as provisional position information. Then, the process ends.

[0105] 6 again, it is assumed that a local map of building A, which is the only building with four or more floors, and a local map of building B, which has three floors, are stored in the local map database DBL. For example, if the floor that satisfies the set altitude condition is the fourth floor, the local map of building A, which has the fourth floor (Building A-4F), is selected.

[0106] On the other hand, if the floor that satisfies the set altitude condition is the second floor, a local map of building A having a second floor (building A-2F) and a local map of building B having a second floor (building B-2F) are determined as two candidates. In this case, the first information terminal 10a acquires location information related to latitude and longitude based on the GNSS radio waves, and one local map including an area closest to the location represented by the information is selected from the two candidates.

[0107] <Local map settings section> In the above, the map selection processes in the first local map selection unit 261 to the fourth local map selection unit 264 may be performed sequentially or in parallel. After that, the local map setting unit 265 reads one local map selected by the first local map selection unit 261 to the fourth local map selection unit 264 from the local map database DBL, and writes the read local map into the cache memory 28.

[0108] Alternatively, when multiple local maps are selected by the first local map selection unit 261 to the fourth local map selection unit 264, the local map setting unit 265 reads one local map from the multiple selected local maps from the local map database DBL in accordance with the priority order, and writes the read local map to the cache memory 28.

[0109] For example, when the first information terminal 10a is located indoors, radio waves transmitted from the GNSS positioning satellites are blocked by a building, and deterioration of the accuracy of position measurement is unavoidable. Therefore, the priority order may be determined in the order of the first local map selection unit 261, the second local map selection unit 262, and the fourth local map selection unit 264.

[0110] On the other hand, when the first information terminal 10a is located outdoors, it is considered that radio waves transmitted from the GNSS positioning satellites can be received well, so the priority of the third local map selection unit 263 may be increased.

[0111] Fig. 9 is a flowchart showing an example of the operation of the local map setting unit shown in Fig. 3. In this example, it is assumed that the first information terminal 10a is mainly located indoors. When the process starts in Fig. 9, flags and the like previously set by the local map setting unit 265 are reset.

[0112] In step S51, the local map setting unit 265 acquires the selection result from the first local map selection unit 261. In step S52, the local map setting unit 265 determines whether or not the acquisition of the selection result in step S51 has been successful.

[0113] If the acquisition of the selection result in step S51 is successful, the process proceeds to step S53. In step S53, the local map setting unit 265 determines that the selection of the local map is successful, reads data of the local map identified by the local map identifier acquired from the first local map selection unit 261 from the local map database DBL, and writes the data to the cache memory 28. In addition, the local map setting unit 265 writes the provisional position information acquired from the first local map selection unit 261 to the cache memory 28. Then, the process ends.

[0114] If the acquisition of the selection result in step S51 fails, the process proceeds to step S54. In step S54, the local map setting unit 265 acquires the selection result from the second local map selection unit 262. In step S55, the local map setting unit 265 determines whether or not the acquisition of the selection result in step S54 was successful.

[0115] If the acquisition of the selection result in step S54 is successful, the process proceeds to step S53. In step S53, the local map setting unit 265 determines that the setting of the local map is successful, reads data of the local map identified by the local map identifier acquired from the second local map selection unit 262 from the local map database DBL, and writes the data to the cache memory 28. In addition, the local map setting unit 265 writes the provisional position information acquired from the second local map selection unit 262 to the cache memory 28. Thereafter, the process ends.

[0116] If the acquisition of the selection result in step S54 fails, the process proceeds to step S56. In step S56, the local map setting unit 265 acquires the selection result from the fourth local map selection unit 264. In step S57, the local map setting unit 265 determines whether or not the acquisition of the selection result in step S56 was successful.

[0117] If the acquisition of the selection result in step S56 is successful, the process proceeds to step S53. In step S53, the local map setting unit 265 determines that the setting of the local map is successful, reads data of the local map identified by the local map identifier acquired from the fourth local map selection unit 264 from the local map database DBL, and writes the data to the cache memory 28. In addition, the local map setting unit 265 writes the provisional position information acquired from the fourth local map selection unit 264 to the cache memory 28. Thereafter, the process ends.

[0118] If the acquisition of the selection result in step S56 fails, the process proceeds to step S58. In step S58, the local map setting unit 265 sets a map setting failure flag indicating that the setting of the local map has failed. Then, the process ends.

[0119] <Point cloud search section> On the other hand, when measurement data obtained from the sensor unit of the first information terminal 10a (e.g., image data obtained from the camera portion of a smartphone) is transmitted to the position estimation device 20, the point cloud search unit 266 shown in Figure 3 acquires the measurement data from the communication control unit 260 and starts the point cloud search process.

[0120] The point cloud search unit 266 extracts a plurality of feature points from the measurement data, and searches for a point cloud to which the extracted feature points match from among a plurality of point clouds selected by any one of the first local map selection unit 261 to the fourth local map selection unit 264 and placed on the local map set by the local map setting unit 265.

[0121] Fig. 10 is a flowchart showing an example of the operation of the point cloud searching unit shown in Fig. 3. When the process starts in Fig. 10, flags and the like previously set by the point cloud searching unit 266 are reset.

[0122] In step S61, the point cloud search unit 266 determines whether or not a local map selected by any one of the first local map selection unit 261 to the fourth local map selection unit 264 has already been set. For example, if the local map has been written in the cache memory 28, it is determined that the local map has already been set.

[0123] If a local map has not been set, the process proceeds to step S62. In step S62, the point cloud search unit 266 sets a point cloud search failure flag indicating that the search for the point cloud has failed. In this case, the local map selection process in the first local map selection unit 261 to the fourth local map selection unit 264 is restarted, and the newly selected local map is set by the local map setting unit 265.

[0124] If a local map has been set, the process proceeds to step S63. In step S63, the point cloud search unit 266 acquires a list of a plurality of point clouds in the local map set by the local map setting unit 265. For example, five point clouds A to E are set in the local map shown in Fig. 4 (the number of point clouds n = 5).

[0125] In step S64, the point cloud search unit 266 starts searching for a point cloud. Here, the point cloud count value i is reset to zero (i=0). In step S65, the point cloud search unit 266 selects a point cloud from the list of multiple point clouds acquired in step S63, thereby reading the next point cloud data (point cloud arrangement information) from the cache memory 28. Here, the point cloud count value i is incremented (i=i+1).

[0126] In step S66, the point cloud search unit 266 acquires measurement data obtained by the sensor unit of the first information terminal 10a measuring the surroundings from the communication control unit 260, and extracts multiple feature points from the acquired measurement data. In step S67, the point cloud search unit 266 determines whether the multiple feature points extracted from the measurement data match the point cloud data read out from the cache memory 28.

[0127] If the multiple feature points extracted from the measurement data match the point cloud data, the process proceeds to step S68. In step S68, the point cloud search unit 266 determines that the point cloud search has been successful and outputs information identifying the point cloud to which the multiple feature points match, and outputs the position information of the point cloud as provisional position information. The point cloud search unit 266 also changes the flag indicating the acquisition status of the point cloud to "acquired". Then, the process ends.

[0128] If the multiple feature points extracted from the measurement data do not match the point cloud data, the process proceeds to step S69. In step S69, the point cloud search unit 266 determines whether the point cloud count value i has reached the number n of point clouds.

[0129] If the point cloud count value i has not reached the number n of point clouds, the process returns to step S65, and the next point cloud is searched for. On the other hand, if the point cloud count value i has reached the number n of point clouds, the process proceeds to step S70. In step S70, the point cloud search process by the point cloud search unit 266 ends without successfully searching the point cloud.

[0130] In step S71, the point cloud search unit 266 determines whether the selection result has been changed in the other local map selection unit. If the selection result has not been changed in the other local map selection unit, the process returns to step S64, and the point cloud search is repeated for new measurement data. On the other hand, if the selection result has been changed in the other local map selection unit, the process proceeds to step S72. In step S72, the point cloud search unit 266 changes the flag indicating the point cloud acquisition status to "not acquired." Then, the process proceeds to step S62.

[0131] Fig. 11 is a schematic diagram for explaining an example of the operation of the point cloud searching unit shown in Fig. 3. Fig. 11 shows schematic perspective views of buildings A and B, and a local map of the fourth floor of building A. Five point clouds A to E are set on the local map of the fourth floor of building A.

[0132] The point cloud search unit 266 may read out the point cloud arrangement information of one point cloud in any order from the point cloud arrangement information of the point clouds A to E arranged on the local map, and determine whether or not a plurality of feature points match the read out point cloud arrangement information. For example, as shown in (a) of FIG. 11, the point cloud search unit 266 may perform the point cloud search process in the order of point cloud A, point cloud B, point cloud C, point cloud D, and point cloud E.

[0133] On the other hand, when position information (provisional position information) is obtained by at least one of the first local map selection unit 261 to the fourth local map selection unit 264, the point cloud search unit 266 may search for a point cloud with which a plurality of feature points match by reading out point cloud arrangement information in order of proximity from the position represented by the obtained position information. For example, as shown in (b) of FIG. 11, when position information indicating that the first information terminal 10a is located near point cloud B is obtained, the point cloud search unit 266 may perform point cloud search processing in the order of point cloud B, point cloud E, point cloud A, point cloud D, and point cloud C.

[0134] <Position estimation part> 3 acquires the measurement data from the communication control unit 260 and starts a position estimation process when measurement data (e.g., image data obtained from a camera part of a smartphone) obtained from a sensor unit of the first information terminal 10a is transmitted to the position estimation device 20. The position estimation unit 267 estimates the position of the first information terminal 10a based on the positional relationship between a plurality of feature points extracted from the measurement data and the point cloud searched by the point cloud search unit 266.

[0135] Fig. 12 is a flowchart showing an example of the operation of the position estimation unit shown in Fig. 3. When the process starts in Fig. 12, flags and the like previously set by the position estimation unit 267 are reset. In step S81, the position estimation unit 267 determines whether or not a point cloud has been acquired.

[0136] If the point cloud has not been acquired, the process proceeds to step S82. In step S82, the position estimation unit 267 determines whether or not provisional position information has been acquired. If provisional position information has been acquired, the process proceeds to step S83, where the position estimation unit 267 outputs provisional position information as a position estimation result. On the other hand, if provisional position information has not been acquired, the process proceeds to step S84, where the position estimation unit 267 sets a position estimation failure flag indicating that the position estimation has failed. Then, the process ends.

[0137] If the point cloud has been acquired, the process proceeds to step S85. In step S85, the position estimation unit 267 acquires measurement data obtained from the sensor unit of the first information terminal 10a from the communication control unit 260, and estimates the position of the first information terminal 10a based on the positional relationship between a plurality of feature points extracted from the measurement data and the point cloud searched by the point cloud search unit 266.

[0138] In step S86, the position estimation unit 267 determines whether the position estimation is successful. If the position estimation is unsuccessful, the process proceeds to step S83, where the position estimation unit 267 outputs provisional position information as the position estimation result. On the other hand, if the position estimation is successful, the process proceeds to step S87.

[0139] In step S87, the position estimation unit 267 calculates the coordinates of the local coordinate system of the estimated position based on the point cloud arrangement location on the local map. Furthermore, in step S88, the position estimation unit 267 outputs the coordinates of the local coordinate system of the estimated position as a position estimation result. Then, the process ends.

[0140] 12, in order to display the estimated position of the first information terminal 10a on the local map, the position estimation unit 267 may read image data of a map image (e.g., a floor plan or illustration of a building) corresponding to the local map from the image database DBI and generate image data in which a display of the estimated position is added to the map image. Based on the image data generated by the position estimation unit 267, a map image indicating the estimated position of the first information terminal 10a is displayed on the display unit 22.

[0141] Furthermore, the image data generated by the position estimation unit 267 may be transmitted from the position estimation device 20 to the position information processing device 30. In that case, for example, it becomes possible to confirm the position of a security guard guarding a building or the like at a control center of a security company. Also, it becomes possible to confirm the position of an automated guided vehicle transporting parts or materials at a control center of a manufacturing company, and to confirm the position of an automated delivery vehicle transporting goods at a control center of a transportation company.

[0142] <Example of point cloud search and position estimation algorithm> In the following, as an example, a case where a camera is used as a sensor unit of the first information terminal 10a will be described. The point cloud search unit 266 of the position estimation device 20 performs image recognition processing on image data obtained by capturing an image of the surroundings with the camera of the first information terminal 10a, extracts multiple feature points from the image data, and obtains the coordinates of those feature points.

[0143] In the above image recognition processing, for example, an image of a target object can be divided into shape and texture (appearance), and each of these can be dimensionally compressed using principal component analysis, allowing a model to be used that can represent changes in the shape and texture of the target with a small number of parameters.

[0144] In this model, the shape vector of all feature points is expressed using an average shape vector and an eigenvector matrix obtained by principal component analysis of deviations from the average shape vector. This allows the shape and texture information of an object to be expressed with low-dimensional parameters, so that the shape and texture of the object can be changed by changing these parameters. Furthermore, parameters related to global changes in where, what size, and what orientation the object is in the image are taken into account.

[0145] Searching the point cloud involves changing the above parameters to change the model locally and globally to generate an image of the target, comparing the generated image with the point cloud image, and determining the parameter values ​​that minimize the error. If the error is equal to or less than a threshold, the point cloud search unit 266 determines that the multiple feature points extracted from the image data match the point cloud being searched.

[0146] Here, since the appearance of the object changes depending on the camera position, the above parameters can also be used to estimate the relative position of the camera with respect to the searched point group. That is, the feature points of the object can be recognized, and the distance between the camera and the feature points can be calculated based on the change in the image caused by the change in the distance and angle from the camera.

[0147] The position estimation unit 267 can obtain the positional relationship between the multiple feature points extracted from the image data and the searched point cloud based on the point cloud arrangement information and the parameter value when the error is minimized, and estimate the position of the first information terminal 10a based on the positional relationship. Typically, the three-dimensional position of the first information terminal 10a is estimated using the principle of triangulation.

[0148] In the above point cloud search and position estimation, SLAM technology may be used, which simultaneously estimates the position and grasps the structure of the surrounding environment (creates an environmental map). When a camera is used as the sensor unit of the first information terminal 10a, Visual SLAM is used. Visual SLAM algorithms can be broadly classified into two types: a feature based method that matches image feature points, and a direct method that uses the brightness of the entire image.

[0149] The feature-based method is a method for estimating the 3D shape of the object and the external parameters of the camera by matching multiple feature points and their feature vectors obtained from images taken from different viewpoints using a specified algorithm.The direct method is a method for estimating the external parameters of the camera by comparing the brightness values ​​of images taken from different viewpoints to minimize the error.

[0150] When using a monocular camera, the unit of distance (scale) becomes indefinite, so it is desirable to detect known objects such as an AR (Augmented Reality) marker, AR tag, QR code (registered trademark), or checkerboard, and know the physical quantity from the image, or to use attitude information obtained from a sensor that can directly measure physical quantities such as the inertial measurement unit 15 shown in Figure 2.

[0151] On the other hand, when a laser sensor is used as the sensor unit of the first information terminal 10a, LiDAR SLAM is used. The point cloud obtained by the laser sensor has high distance accuracy and is very effective for map construction of SLAM. In matching between a plurality of feature points extracted from the measurement data and the point cloud being searched, an ICP (Iterative Closest Point) algorithm, an NDT (Normal Distributions Transform) algorithm, or the like is used.

[0152] The ICP algorithm determines the closest combination of points between multiple feature points and the point cloud being searched, and iteratively updates the rotation and translation of one side so that the sum of the squares of the distances between them is minimized.The NDT algorithm divides the search space into a grid of a certain size, calculates the average and variance of the distribution of the point cloud within the grid, and iteratively updates the rotation and translation of one side so that they match optimally.

[0153] <Effects of this embodiment> According to this embodiment, in a position estimation device 20 that estimates the position of a first information terminal 10a that can be transported or moved, point cloud arrangement information (environmental map) used to estimate the position of the first information terminal 10a is divided into multiple local maps of specific areas and stored in a local map database DBL. The first information terminal 10a selects one local map based on waves (environmental information) received from an external device, thereby suppressing the costs required for creating the point cloud arrangement information and the costs associated with the memory capacity for writing the point cloud arrangement information, thereby reducing equipment costs.

[0154] For example, if point cloud arrangement information is required for buildings A and B, which are located some distance from each other, creating a local map of building A and a local map of building B rather than creating an environmental map of a large rectangular area that includes buildings A and B can reduce the costs required to create the point cloud arrangement information and the costs associated with the memory capacity for writing the point cloud arrangement information.

[0155] In addition, by narrowing down the search area based on the measurement data obtained from the sensor unit of the first information terminal 10a from a wide area to a relatively narrow area, it is possible to reduce the rate of error in the estimated position that may occur when estimating the position of the first information terminal 10a. Furthermore, according to this embodiment, it is possible to provide a first information terminal 10a that enables position estimation while reducing the equipment cost in such a position estimation device 20.

[0156] The present invention is not limited to the above-described embodiment, and many modifications can be made within the technical concept of the present invention by those skilled in the art. For example, some of the operations in the above-described embodiment can be omitted. [Industrial Applicability]

[0157] The present invention can be used in a position estimation system that estimates the position of a portable or mobile information terminal. [Explanation of symbols]

[0158] 10a...first information terminal, 10b...second information terminal, 11...sensor unit, 12...short-range communication circuit, 13...wireless LAN communication circuit, 14...GNSS receiving circuit, 15...inertial measurement unit, 151...barometer, 16...wireless communication circuit, 17...control circuit, 20...position estimation device, 21...operation unit, 22...display unit, 23...audio input / output unit, 24...communication circuit, 25...interface, 26...CPU, 260...communication control unit, 261...first local map selection unit, 262...second local map selection unit, 263...third local map selection unit, 264...fourth local map selection unit, 265...local map setting unit, 266...point cloud search unit, 267...position estimation unit, 27...storage unit, 28...cache memory, 30...position information processing device

Claims

1. A position estimation device that estimates the position of an information terminal having a sensor unit including an image sensor or a laser range sensor and capable of receiving a wave transmitted within a predetermined range from an external device, a communication circuit for performing data communication with the information terminal via a network; A local map database that stores data of a plurality of local maps including point cloud arrangement information in a plurality of specific areas; at least one local map selection unit including a local map selection unit that selects one local map from a plurality of local maps stored in the local map database according to information obtained based on a wave signal received by the information terminal from an external device; a point cloud search unit that extracts a plurality of feature points from measurement data obtained from a sensor unit of the information terminal, and searches for a point cloud to which the plurality of feature points match among a plurality of point clouds arranged on a selected local map; a position estimation unit that estimates a position of the information terminal based on a positional relationship between a plurality of feature points extracted from measurement data obtained from a sensor unit of the information terminal and the point cloud searched by the point cloud search unit; A position estimation device comprising:

2. 2. The position estimation device according to claim 1, wherein the at least one local map selection unit includes a local map selection unit that acquires device information identifying an individual device that transmits waves received by the information terminal, and selects one local map corresponding to the acquired device information.

3. The position estimation device according to claim 1, wherein the at least one local map selection unit includes a local map selection unit that acquires ID information of a beacon device obtained from radio waves or light of a beacon received by the information terminal, and selects one local map corresponding to the acquired ID information of the beacon device.

4. 2. The position estimation device according to claim 1, wherein the at least one local map selection unit includes a local map selection unit that acquires ID information of a wireless LAN access point obtained from wireless LAN radio waves received by the information terminal, and selects one local map corresponding to the acquired ID information of the wireless LAN access point.

5. the communication circuit performs data communication between the information terminal and a second information terminal having the respective barometers via a network; 5. The position estimation device according to claim 1, wherein the at least one local map selection unit calculates a relative altitude of the information terminal with respect to the second information terminal based on atmospheric pressure information obtained from a barometer of the information terminal and atmospheric pressure information obtained from a barometer of the second information terminal, and selects one local map according to the altitude information of the information terminal, or selects one local map according to position information regarding latitude and longitude obtained based on radio waves of a global positioning system received by the information terminal in addition to the altitude information of the information terminal.

6. a local map setting unit that reads out one local map selected by the at least one local map selection unit from the local map database, or reads out one local map from among a plurality of local maps selected by the at least one local map selection unit in accordance with a priority order, and writes the read local map into a memory; 5. The position estimation device according to claim 1, wherein the point cloud search unit searches for a point cloud to which the plurality of feature points match from among a plurality of point clouds arranged on a local map written in the memory.

7. 5. The position estimation device according to claim 1, wherein the point cloud search unit searches for a point cloud to which the plurality of feature points match by reading out point cloud arrangement information from among a plurality of point clouds arranged on a selected local map in order of proximity to a position represented by position information obtained by the at least one local map selection unit.

8. An information terminal that performs data communication with a position estimation device, A sensor unit including an image sensor or a laser range sensor, which measures surrounding objects and generates measurement data; a short-range communication circuit that receives a beacon wave transmitted from a beacon device and outputs a beacon signal; a wireless LAN communication circuit that receives wireless LAN radio waves transmitted from a wireless LAN access point and outputs a wireless LAN signal; a GNSS receiving circuit for receiving radio waves transmitted from three or more positioning satellites and outputting a received signal; an inertial measurement device including a barometer that measures atmospheric pressure and outputs an atmospheric pressure measurement signal, and that detects three-dimensional inertial motion of the information terminal and outputs a detection signal; a wireless communication circuit for performing data communication with the position estimation device via a network; a control circuit that controls the sensor unit to perform continuous measurement when the position estimation device performs a position estimation operation, and controls the wireless communication circuit to transmit to the position estimation device at least one selected from the measurement data obtained from the sensor unit, ID information of a beacon device obtained from the beacon signal, ID information of a wireless LAN access point obtained from the wireless LAN signal, position information obtained from a reception signal of the GNSS reception circuit, attitude information obtained from a detection signal of the inertial measurement device, and atmospheric pressure information obtained from the atmospheric pressure measurement signal; An information terminal comprising:

9. A position estimation device that has a sensor unit including an image sensor or a laser range sensor and performs data communication via a network with an information terminal capable of receiving a wave transmitted within a predetermined range from an external device, the position estimation program being used to estimate the position of the information terminal, A step (a) of selecting one local map from among a plurality of local maps that are stored in a local map database and include point cloud arrangement information in a plurality of specific areas according to information obtained based on a wave signal received by the information terminal from an external device; a step (b) of extracting a plurality of feature points from measurement data obtained from a sensor unit of the information terminal, and searching for a point cloud to which the plurality of feature points match among a plurality of point clouds arranged on the selected local map; a step (c) of estimating a position of the information terminal based on a positional relationship between a plurality of feature points extracted from measurement data obtained from a sensor unit of the information terminal and the point group searched for in the step (b); A position estimation program that causes a CPU to execute the above.

10. A position estimation method for estimating a position of an information terminal in a position estimation device that has a sensor unit including an image sensor or a laser range sensor and that performs data communication via a network with an information terminal that is capable of receiving a wave transmitted within a predetermined range from an external device, comprising: (a) selecting one local map from among a plurality of local maps that are stored in a local map database and include point cloud arrangement information in a plurality of specific areas according to information obtained based on a wave signal received by the information terminal from an external device; (b) extracting a plurality of feature points from measurement data obtained from a sensor unit of the information terminal, and searching for a point cloud to which the plurality of feature points match among a plurality of point clouds arranged on the selected local map; a step (c) of estimating a position of the information terminal based on a positional relationship between a plurality of feature points extracted from measurement data obtained from a sensor unit of the information terminal and the point group searched for in step (b); A location estimation method comprising:

Images