Autonomous mobile device and self-position estimation device

Abstract

To provide an autonomous mobile device and a self-position estimation device that are able to improve the estimation accuracy of a self-position.SOLUTION: An autonomous mobile device includes a camera and an information processing device that estimates its own position. The information processing device performs: image cut-out in which, based on building information indicating a position and a height of a building on a map, an image including a region of the building is cut out from an image captured by the camera; a feature-point extraction in which a feature point of the building is extracted from the cut-out image; and self-position estimation in which a time-series movement of a feature point is calculated from the feature point of the building extracted at a first time point and the feature point of the building extracted at a second time point before the first time point, and the self-position at the first time point is estimated based on the self-position at the second time point and the time-series movement of the feature point.SELECTED DRAWING: Figure 1A

Claims

1. An autonomous mobile device that travels along a planned route, comprising an external or internal information processing device for estimating its own position, a control device configured to send and receive information from each other, and a storage device, An imaging device that acquires an image by imaging the surroundings, A moving device that is controlled by the control device and capable of moving the autonomous moving device in a desired direction, Equipped with, The storage device stores a map showing the areas where objects exist and areas where they do not exist along the planned travel route, and building information indicating whether the area where the objects exist on the map is the area of ​​a building. The information processing device generates an image cropped image containing the area of ​​the building by cropping an image containing the area of ​​the building indicated by the building information from the captured image based on the building information stored in the storage device. The aforementioned cropped image is subjected to feature point extraction, which extracts the characteristic points of the building from the cropped image. Self-position estimation is performed by calculating the time-series movement of the feature points from the feature points of the building extracted at the first time point and the feature points of the building extracted at the second time point prior to the first time point, and estimating the self-position at the first time point based on the self-position at the second time point and the time-series movement of the feature points, Movement control that causes the control device to control the autonomous mobile device so that it moves along the planned movement path, based on the self-position and the planned movement path that passes through an area on the map where the object does not exist. To do It is configured in such a way. Autonomous mobile device.

2. In the autonomous mobile device according to claim 1, The storage device further stores sensor information relating to the field of view and installation position of the imaging device. The aforementioned building information indicates whether the area on the map where the object exists is the area of ​​the building, and the height of the building. The information processing device is configured to identify the area of ​​the building indicated by the building information based on the building information and sensor information stored in the storage device during image extraction, and to store the feature points of the building extracted by feature point extraction and the self-position estimated by self-position estimation in the storage device. The information processing device is configured to use, in the self-position estimation, the feature points of the building extracted at the second time point and the self-position at the second time point, which are stored in the memory device. Autonomous mobile device.

3. In the autonomous mobile device according to claim 2, The aforementioned information processing device is The information processing device is configured to be able to input at least one data set of drawing data and image data that can identify the building information. The information processing device stores the building information identified from the input data, at least one of the data, in the storage device. It is configured in such a way. Autonomous mobile device.

4. In the autonomous mobile device according to claim 2, A display device is connected to the aforementioned information processing device. The aforementioned information processing device is A GUI screen for inputting information is displayed on the display device. Multiple pieces of building information can be input to the information processing device via the GUI screen. It is configured in such a way, The information processing device stores the input information on multiple buildings in the storage device. It is configured in such a way, In the image extraction process, if the captured image includes multiple areas of the building indicated by each of the multiple building information items, the information processing device generates multiple extracted images corresponding to each of the multiple areas of the building. In the feature point extraction process, the feature points of each of the multiple buildings are extracted from each of the multiple cropped images. It is configured in such a way. Autonomous mobile device.

5. In the autonomous mobile device according to claim 1, The aforementioned information processing device is The aforementioned image extraction, Based on the segmentation results obtained from the difference in brightness between the building and the sky in the captured image, an image containing the area of ​​the building is extracted, thereby generating the extracted image containing the area of ​​the building. It can be selectively executed. It is configured in such a way. Autonomous mobile device.

6. In the autonomous mobile device according to claim 1, The aforementioned information processing device is The aforementioned image extraction, Other image extraction methods generate the extracted image containing the region of the building by extracting an image containing the region of the building based on deep learning-based segmentation of the building that is visible in the captured image, It can be selectively executed. It is configured in such a way. Autonomous mobile device.

7. In the autonomous mobile device according to claim 1, Includes a movable mechanism for moving the orientation of the imaging device and / or the lens of the imaging device, The aforementioned information processing device is By controlling the aforementioned movable mechanism, The orientation of the imaging device and / or the lens of the imaging device is moved to a position where the imaging range of the imaging device includes a portion in which a feature point easily extracted, including at least one of the edges and corners of the building, is easily extracted. It is configured in such a way. Autonomous mobile device.

8. In the autonomous mobile device according to claim 2, The aforementioned information processing device is The system includes an ambient sensor that detects the reflection points of the irradiation waves on an object as measurement points by irradiating the surrounding area with an irradiation wave, and acquires position information indicating the relative position of the measurement points with respect to the autonomous mobile device. The aforementioned information processing device is By moving through the area to be mapped, and based on the position information of the measurement points acquired by the surrounding sensors, the location where the object exists is recorded in map coordinates corresponding to the area to be mapped, thereby creating a map that represents the area where the object exists and the area where the object does not exist, and storing the created map in the storage device. It is configured in such a way. Autonomous mobile device.

9. In the autonomous mobile device according to claim 1, A display device is connected to the aforementioned information processing device. The aforementioned information processing device is A GUI screen for inputting information is displayed on the display device. The system is configured to allow input of the starting and ending points of movement on the map via a GUI screen. When the information processing device receives input for the starting point and the ending point of movement, it sets a route as the planned movement route between the starting point and the ending point of movement such that the building corresponding to the building information is included in the imaging range of the imaging device when the autonomous mobile device moves. It is configured in such a way. Autonomous mobile device.

10. An information processing device that acquires an image from an imaging device mounted on a moving body traveling along a planned route, which captures the surroundings, and estimates the self-position of the moving body based on the image. A storage unit that stores a map showing areas where an object exists and areas where an object does not exist along the planned travel route, and building information indicating whether the area where the object exists on the map is the area of ​​a building. Equipped with, The aforementioned information processing device is Image cropping generates a cropped image containing the area of ​​the building by cropping an image containing the area of ​​the building indicated by the building information from the captured image based on the building information stored in the memory unit. The aforementioned cropped image is subjected to feature point extraction, which extracts the characteristic points of the building from the cropped image. Self-position estimation is performed by calculating the time-series movement of the feature points from the feature points of the building extracted at the first time point and the feature points of the building extracted at the second time point prior to the first time point, and estimating the self-position at the first time point based on the self-position at the second time point and the time-series movement of the feature points, To do It is configured in such a way. Self-location estimation device.

Images