Position estimation system, position estimation device, position estimation method, and position estimation program

Abstract

Provided is a position estimation system comprising: a map information acquisition means for acquiring map information in which the position and posture of each of a plurality of trees in the vicinity of a road are associated with each other; a posture recognition means for recognizing the posture of a tree appearing in an image which has been captured by an imaging means that is mounted on a moving body which travels on the road; an extraction means for extracting, from among the plurality of trees included in the map information, a tree which has a posture that coincides with that of the tree appearing in the image; and a position estimation means for using the extracted position of the tree to estimate the position of the moving body. The position estimation system may use depth information which has been acquired using a sensor that is mounted on the moving body to recognize the posture of a tree.

Description

Position estimation system, position estimation device, position estimation method, and position estimation program

[0001] The present disclosure relates to a position estimation system, a position estimation device, a position estimation method, and a position estimation program.

[0002] Patent Document 1 discloses a method for estimating the self-position of a moving object based on the distribution of tree trunks in an environment where multiple trees grow thickly.

[0003] International Publication No. 2022 / 107587

[0004] The method disclosed in Patent Document 1 for detecting tree trunks with few features is unable to match individual trees with environmental map data, and instead requires matching a row of multiple trees with environmental map data. In other words, the method disclosed in Patent Document 1 can only be used in environments where multiple trees are lined up, and cannot be used in environments where trees are sparse. One of the purposes of the present disclosure is to provide a position estimation system, etc. that uses trees to more effectively estimate the position of a moving object.

[0005] The position estimation system disclosed herein comprises a map information acquisition means for acquiring map information that associates the position and posture of each of a plurality of trees near a road; a posture recognition means for recognizing the posture of trees that appear in an image captured by an imaging means mounted on a mobile body traveling on a road; an extraction means for extracting trees from the plurality of trees included in the map information whose posture matches that of the trees that appear in the image; and a position estimation means for estimating the position of the mobile body using the positions of the extracted trees.

[0006] The position estimation device disclosed herein is a position estimation device that includes a map information acquisition means that acquires map information that associates the position and posture of each of a plurality of trees near a road; a posture recognition means that recognizes the posture of trees that appear in an image captured by an imaging means mounted on a mobile body traveling on a road; an extraction means that extracts trees from the plurality of trees included in the map information whose posture matches that of the trees that appear in the image; and a position estimation means that estimates the position of the mobile body using the positions of the extracted trees.

[0007] The position estimation method disclosed herein is a position estimation method that acquires map information that associates the position and posture of each of a plurality of trees near a road, recognizes the postures of the trees that appear in an image captured by an imaging means mounted on a mobile object traveling on the road, extracts trees from the plurality of trees included in the map information whose postures match those of the trees that appear in the image, and estimates the position of the mobile object using the positions of the extracted trees.

[0008] The position estimation program disclosed herein is a position estimation program that causes an information processing device to perform the following operations: acquire map information that associates the position and posture of each of multiple trees near a road; recognize the posture of the trees that appear in an image captured by an imaging means of a mobile body traveling on the road; extract trees from the multiple trees included in the map information whose posture matches that of the trees that appear in the image; and estimate the position of the mobile body using the positions of the extracted trees.

[0009] The present disclosure provides a position estimation system and the like that uses trees to more effectively estimate the position of a moving object.

[0010] Fig. 1 is a block diagram showing a configuration of a position estimation system according to the present disclosure; Fig. 2 is a block diagram showing a configuration of a position estimation device according to the present disclosure; Fig. 3 is a flowchart showing a position estimation method according to the present disclosure; Fig. 4 is a diagram showing an example of an image for recognizing the posture of a tree according to the present disclosure; Fig. 5 is a block diagram showing a configuration of an information processing device according to the present disclosure.

[0011] Embodiment 1 Fig. 1 is a block diagram showing the configuration of a position estimation system according to the present disclosure. An example configuration of a position estimation system 100 will be described below with reference to Fig. 1. The position estimation system 100 is a system that is installed in, for example, a mobile object and estimates its own position. In particular, the position estimation system 100 is a system that captures images of trees in the environment surrounding the mobile object and estimates the position of the mobile object from the posture of the trees.

[0012] As shown in FIG. 1, the position estimation system 100 includes a map information acquisition unit 101 , a posture recognition unit 102 , an extraction unit 103 , and a position estimation unit 104 .

[0013] The map information acquisition unit 101 acquires map information that associates the position and posture of each of a plurality of trees near a road. For example, a road is a public road or a private road, and trees near the road are roadside trees. The posture of a tree is information that indicates the shape of the tree. For example, the posture of a tree is expressed as information that indicates the skeleton including the trunk and branches of the tree. The map information may include tree position information that associates the identification information of the tree with the position of the tree, and tree posture information that associates the identification information of the tree with the posture of the tree. The map information acquisition unit 101 acquires map information that associates the positions of the trunks and branches of roadside trees with a map.

[0014] The posture recognition unit 102 recognizes the posture of trees in an image captured by an imaging device mounted on a mobile object traveling on a road. Mobile objects traveling on roads include, for example, passenger vehicles such as automobiles and buses, and work vehicles such as tractors. The mobile object is, for example, an autonomous mobile object equipped with a prime mover or motor and capable of autonomous movement using gasoline or electricity as power.

[0015] The position estimation system 100 is attached to a mobile object and estimates the position of the mobile object. For example, the position estimation system 100 may be executed by an ECU (Electronic Control Unit). Alternatively, the position estimation system 100 may be provided remotely from the mobile object and estimate the position of the mobile object via electrical communication or the like. For example, the position estimation system 100 may be executed by an external cloud server. The mobile object can estimate its own position while moving using the position estimation system 100. An autonomous mobile object attached with a position estimation system is capable of autonomous driving.

[0016] The mobile body is equipped with an imaging device that is an imaging means. For example, an RGB sensor is used for the imaging device. Alternatively, an infrared sensor may be used for the imaging device. Alternatively, the mobile body may be equipped with a distance sensor. The distance sensor may be a depth sensor, LiDAR (Light Detection and Ranging), millimeter-wave radar, laser, or stereo camera. The distance sensor is used to measure the distance to natural objects. The distance sensor acquires depth information.

[0017] FIG. 4 is a diagram showing an example of an image for recognizing the posture of a tree according to the present disclosure. As shown in FIG. 4, the posture of a tree is information indicating, for example, the skeleton including the trunk and branches of the tree. The posture of a tree is represented, for example, by the coordinates of each point constituting the skeleton and the connection relationship between the points. The posture recognition unit 102 may recognize the posture of a tree as shown in FIG. 4 from an image captured by an imaging device mounted on a moving object. The posture recognition unit 102 may recognize the posture from an image using any method. For example, the posture recognition unit 102 may recognize the posture from an image using a posture estimation AI created by machine learning.

[0018] The posture recognition unit 102 may also recognize the posture of a tree as shown in FIG. 4 from depth information acquired by a distance sensor mounted on a moving object. In this case, for example, point cloud data including the trunk and branches of the tree is acquired by the distance sensor, and the posture of the tree is recognized using this point cloud data. Here, the point cloud data is data representing the three-dimensional shape of an object. The point cloud data may include color information in addition to the three-dimensional shape of the object.

[0019] The extraction unit 103 extracts trees from the map information whose postures match those of the trees depicted in the image. For example, the extraction unit 103 compares the coordinates of each point constituting the skeleton of the tree depicted in the image and the connection relationships between each point with the coordinates of each point constituting the skeleton of each tree included in the map information and the connection relationships between each point, and extracts trees whose coordinates of each point and the connection relationships between each point match or are similar.

[0020] Because the trunks and branches of trees change little over time, by using information indicating the skeleton including the trunks and branches of the tree as the tree posture, matching trees can be extracted with greater accuracy.

[0021] The position estimation unit 104 estimates the position of the moving object using the positions of the extracted trees. First, the position estimation unit 104 acquires the positions of the extracted trees from map information. If the map information includes tree position information that associates tree identification information with tree positions and tree orientation information that associates tree identification information with tree orientations, the position estimation unit 104 may acquire the identification information of the extracted trees and acquire the positions of the trees associated with the identification information.

[0022] Next, the position estimation unit 104 calculates the relative position of the moving object with respect to the trees. For example, the position estimation unit 104 estimates the relative position of the moving object with respect to the trees from the orientation of the moving object and the distance between the moving object and the trees. The orientation of the moving object may be calculated from information from a position information sensor or acceleration sensor mounted on the moving object, or may be calculated based on the orientation of trees, buildings, and other structures obtained by image recognition of images captured by an imaging device. The distance between the moving object and the trees may be calculated from depth information acquired by a distance sensor mounted on the moving object, or may be calculated by image recognition of images captured by an imaging device.

[0023] The position estimation unit 104 estimates the position of the moving body based on the acquired tree positions and the calculated relative position of the moving body with respect to the trees. In this way, the position of the moving body is estimated from the trees near the road.

[0024] In addition to trees, buildings such as buildings can also be used to estimate the location. For example, in an area with many buildings, such as an urban area, the location of a moving object may be estimated based on buildings, and in an area with few buildings, such as a suburb, the location may be estimated based on trees.

[0025] With this configuration, a position estimation system is provided that extracts trees whose postures match those of trees captured in an image and estimates the position of a moving object.

[0026] The map information acquisition unit 101, the attitude recognition unit 102, the extraction unit 103, and the position estimation unit 104 may be read as a map information acquisition means, an attitude recognition means, an extraction means, and a position estimation means, respectively.

[0027] The position estimation system 100 is realized by an information processing device 500 such as an ECU. FIG. 5 is a block diagram showing the configuration of the information processing device of the present disclosure. As shown in FIG. 5, the information processing device 500 includes a processor 501 that processes a program to execute processing, and a memory 502 that stores the program. The information processing device 500 may be configured as a single device or may be configured across multiple devices. The information processing device 500 may also be configured as a cloud server that distributes some or all of its functions.

[0028] When the information processing device 500 is configured as a single device, it can be referred to as a position estimation device 200. FIG. 2 is a block diagram showing the configuration of the position estimation device according to the present disclosure. As shown in FIG. 2, the position estimation device includes a map information acquisition unit 201, a posture recognition unit 202, an extraction unit 203, and a position estimation unit 204. The map information acquisition unit 201, the posture recognition unit 202, the extraction unit 203, and the position estimation unit 204 have the same functions as the map information acquisition unit 101, the posture recognition unit 102, the extraction unit 103, and the position estimation unit 104, respectively. The map information acquisition unit 201, the posture recognition unit 202, the extraction unit 203, and the position estimation unit 204 may be interpreted as map information acquisition means, posture recognition means, extraction means, and position estimation means, respectively.

[0029] (Description of Self-Location Estimation Method According to an Embodiment) Fig. 3 is a flowchart of a location estimation method according to the present disclosure. The location estimation method will be described below with reference to the flowchart of Fig. 3. The location estimation method according to the embodiment is executed, for example, by the location estimation system 100 of Fig. 1. As shown in Fig. 3, first, the map information acquisition unit 101 acquires map information (step S301). The map information acquisition unit 101 acquires map information that associates the positions and postures of multiple trees near a moving road.

[0030] Next, the posture recognition unit 102 recognizes the posture of the tree (step S302). The posture recognition unit 102 recognizes the posture of the tree in the image captured by the imaging means mounted on the mobile object traveling on the road. Next, the extraction unit 103 extracts the tree (step S303). The extraction unit 103 extracts the tree whose posture matches the tree in the image from among the multiple trees included in the map information. Next, the position estimation unit 104 estimates the position of the mobile object (step S304). The position estimation unit 104 estimates the position of the mobile object using the positions of the extracted trees.

[0031] With this configuration, a position estimation method is provided that extracts trees whose postures match those of trees in an image and estimates the position of a moving object. The position estimation method is executed by the information processing device 500, and a position estimation program is installed.

[0032] Furthermore, part or all of the processing in the above-described position estimation system 100 and position estimation device 200 can be realized as a computer program. Such a program can be stored and supplied to a computer using various types of non-transitory computer-readable media. Non-transitory computer-readable media include various types of tangible recording media. Examples of non-transitory computer-readable media include magnetic recording media (e.g., flexible disks, magnetic tapes, hard disk drives), magneto-optical recording media (e.g., magneto-optical disks), CD-ROMs (Read Only Memory), CD-Rs, CD-R / Ws, and semiconductor memories (e.g., mask ROMs, PROMs (Programmable ROMs), EPROMs (Erasable PROMs), flash ROMs, and RAMs (Random Access Memory)). Furthermore, the program may be supplied to a computer by various types of temporary computer-readable media. Examples of temporary computer-readable media include electrical signals, optical signals, and electromagnetic waves. The temporary computer-readable medium can supply the program to the computer via a wired communication path such as an electric wire or an optical fiber, or via a wireless communication path.

[0033] Although the present disclosure has been described above with reference to the embodiments, the present disclosure is not limited to the above-described embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present disclosure within the scope of the present disclosure. Furthermore, each embodiment can be combined with other embodiments as appropriate.

[0034] Each drawing is merely an example for describing one or more embodiments. Each drawing may not relate to only one particular embodiment, but may also relate to one or more other embodiments. As will be understood by those skilled in the art, various features or steps described with reference to any one drawing can be combined with features or steps shown in one or more other drawings to create, for example, an embodiment not explicitly shown or described. Not all features or steps shown in any one drawing are necessary to describe an exemplary embodiment, and some features or steps may be omitted. The order of steps described in any drawing may be changed as appropriate.

[0035] Some or all of the above embodiments can be described as, but are not limited to, the following supplementary notes. (Supplementary Note 1) A position estimation system comprising: map information acquisition means for acquiring map information associating the position and posture of each of a plurality of trees near a road; posture recognition means for recognizing the postures of trees appearing in an image captured by an imaging means mounted on a mobile object traveling on the road; extraction means for extracting trees whose postures match those of the trees appearing in the image from the plurality of trees included in the map information; and position estimation means for estimating the position of the mobile object using the positions of the extracted trees. (Supplementary Note 2) The position estimation system according to Supplementary Note 1, which recognizes the postures of trees using depth information acquired by a sensor mounted on the mobile object. (Supplementary Note 3) The position estimation system according to Supplementary Note 1, wherein the map information includes tree position information associating identification information of the trees with the positions of the trees, and tree posture information associating identification information of the trees with the postures of the trees. (Supplementary Note 4) The position estimation system according to Supplementary Note 2, wherein the sensor mounted on the mobile body is a LiDAR (Light Detection and Ranging). (Supplementary Note 5) The position estimation system according to any one of Supplements 1 to 4, wherein the mobile body is an automobile, a bus, or a tractor. (Supplementary Note 6) A position estimation device comprising: a map information acquisition means for acquiring map information associating the position and posture of each of a plurality of trees near a road; a posture recognition means for recognizing the posture of the tree appearing in an image captured by an imaging means mounted on the mobile body traveling on the road; an extraction means for extracting, from the plurality of trees included in the map information, a tree whose posture matches that of the tree appearing in the image; and a position estimation means for estimating the position of the mobile body using the positions of the extracted trees. (Supplementary Note 7) The position estimation device according to Supplementary Note 6, wherein the posture of the tree is recognized using depth information acquired by a sensor mounted on the mobile body. (Supplementary Note 8) The position estimation device according to Supplementary Note 6, wherein the map information includes tree position information that associates identification information of the tree with a position of the tree, and tree posture information that associates identification information of the tree with a posture of the tree. (Supplementary Note 9) The position estimation device according to Supplementary Note 7, wherein the sensor mounted on the moving object is a LiDAR.(Supplementary Note 10) The position estimation device according to any one of Supplementary Notes 6 to 9, wherein the mobile body is an automobile, a bus, or a tractor. (Supplementary Note 11) A position estimation method comprising: acquiring map information associating the position and posture of each of a plurality of trees near a road; recognizing the postures of the trees in an image captured by an imaging means mounted on a mobile body traveling on the road; extracting trees whose postures match those of the trees in the image from the plurality of trees included in the map information; and estimating the position of the mobile body using the positions of the extracted trees. (Supplementary Note 12) The position estimation method according to Supplementary Note 11, wherein the postures of the trees are recognized using depth information acquired by a sensor mounted on the mobile body. (Supplementary Note 13) The position estimation method according to Supplementary Note 11, wherein the map information includes tree position information associating identification information of the trees with the positions of the trees, and tree posture information associating identification information of the trees with the postures of the trees. (Supplementary Note 14) The position estimation method according to Supplementary Note 12, wherein the sensor mounted on the moving body is a LiDAR. (Supplementary Note 15) The position estimation method according to any one of Supplementary Notes 11 to 14, wherein the moving body is an automobile, a bus, or a tractor. (Supplementary Note 16) A position estimation program causing an information processing device to execute the following steps: acquiring map information in which the position and orientation of each of a plurality of trees near a road are associated; recognizing the orientation of the trees appearing in an image captured by an imaging means mounted on a moving body traveling on the road; extracting trees whose orientations match those of the trees appearing in the image from the plurality of trees included in the map information; and estimating the position of the moving body using the positions of the extracted trees. (Supplementary Note 17) The position estimation program according to Supplementary Note 16, which recognizes the orientation of the trees using depth information acquired by a sensor mounted on the moving body. (Supplementary Note 18) The position estimation program according to Supplementary Note 16, wherein the map information includes tree position information associating identification information of the tree with the position of the tree, and tree attitude information associating identification information of the tree with the attitude of the tree. (Supplementary Note 19) The position estimation program according to Supplementary Note 17, wherein the sensor mounted on the moving object is a LiDAR. (Supplementary Note 20) The position estimation program according to any one of Supplementary Notes 16 to 19, wherein the moving object is an automobile, a bus, or a tractor.

[0036] Some or all of the elements (e.g., configurations and functions) described in Supplementary Notes 2 to 5 that are dependent on Supplementary Note 1 (e.g., system) may also be dependent on Supplementary Note 6 (e.g., device), Supplementary Note 11 (e.g., method), and Supplementary Note 16 (e.g., program) in the same dependency relationship as Supplementary Note 2 to Supplementary Note 5. Some or all of the elements described in any Supplementary Note may be applied to various hardware, software, recording means for recording software, systems, and methods.

[0037] 100 Position estimation system, 101 Map information acquisition unit, 102 Attitude recognition unit, 103 Extraction unit, 104 Position estimation unit, 200 Position estimation device, 201 Map information acquisition unit, 202 Attitude recognition unit, 203 Extraction unit, 204 Position estimation unit, 500 Information processing device, 501 Processor, 502 Memory

Claims

1. A position estimation system comprising: a map information acquisition means for acquiring map information associating the positions and postures of a plurality of trees near a road; a posture recognition means for recognizing the posture of a tree shown in an image captured by an imaging means mounted on a moving body traveling on the road; an extraction means for extracting a tree whose posture matches that of the tree shown in the image from among the plurality of trees included in the map information; and a position estimation means for estimating the position of the moving body using the position of the extracted tree.

2. The position estimation system according to claim 1, wherein the posture of the tree is recognized using depth information acquired using a sensor mounted on the moving body.

3. The position estimation system according to claim 1, wherein the map information includes tree position information associating the identification information of the tree with the position of the tree, and tree posture information associating the identification information of the tree with the posture of the tree.

4. The position estimation system according to claim 2, wherein the sensor mounted on the moving body is a LiDAR (Light Detection And Ranging).

5. The position estimation system according to any one of claims 1 to 4, wherein the moving body is an automobile, a bus, or a tractor.

6. A position estimation device comprising: a map information acquisition means for acquiring map information associating the positions and postures of a plurality of trees near a road; a posture recognition means for recognizing the posture of a tree shown in an image captured by an imaging means mounted on a moving body traveling on the road; an extraction means for extracting a tree whose posture matches that of the tree shown in the image from among the plurality of trees included in the map information; and a position estimation means for estimating the position of the moving body using the position of the extracted tree.

7. The position estimation device according to claim 6, wherein the posture of the tree is recognized using depth information acquired using a sensor mounted on the moving body.

8. The position estimation device according to claim 6, wherein the map information includes tree position information associating the identification information of the tree with the position of the tree, and tree posture information associating the identification information of the tree with the posture of the tree.

9. The position estimation device according to claim 7, wherein the sensor mounted on the moving body is a LiDAR.

10. The position estimation device according to any one of claims 6 to 9, wherein the moving body is an automobile, a bus, or a tractor.

11. A position estimation method for acquiring map information associating the positions and postures of a plurality of trees near a road, recognizing the posture of a tree shown in an image captured by imaging means mounted on a moving body traveling on the road, extracting a tree whose posture matches that of the tree shown in the image from among the plurality of trees included in the map information, and estimating the position of the moving body using the position of the extracted tree.

12. The position estimation method according to claim 11, wherein the posture of the tree is recognized using depth information acquired using a sensor mounted on the moving body.

13. The position estimation method according to claim 11, wherein the map information includes tree position information associating the identification information of the tree with the position of the tree, and tree posture information associating the identification information of the tree with the posture of the tree.

14. The position estimation method according to claim 12, wherein the sensor mounted on the moving body is a LiDAR.

15. The position estimation method according to any one of claims 11 to 14, wherein the moving body is an automobile, a bus, or a tractor.

16. A position estimation program for causing an information processing apparatus to acquire map information associating the positions and postures of a plurality of trees near a road, recognize the posture of a tree shown in an image captured by imaging means mounted on a moving body traveling on the road, extract a tree whose posture matches that of the tree shown in the image from among the plurality of trees included in the map information, and estimate the position of the moving body using the position of the extracted tree.

17. The position estimation program according to claim 16, wherein the posture of the tree is recognized using depth information acquired using a sensor mounted on the moving body.

18. The position estimation program according to claim 16, wherein the map information includes tree position information associating the identification information of the tree with the position of the tree, and tree posture information associating the identification information of the tree with the posture of the tree.

19. The position estimation program according to claim 17, wherein the sensor mounted on the moving body is a LiDAR.

20. The position estimation program according to any one of claims 16 to 19, wherein the moving body is an automobile, a bus, or a tractor.

Images