Information processing method, information processing device, and program

Abstract

The present invention relates to an information processing method, information processing device, and program facilitating improvement of the robustness of self-position estimation processing. An information processing unit of the present invention generates first distance information and second distance information from sensor data output by a ranging sensor mounted on a moving body, executes first self-position estimation processing, which is based on the first distance information, and executes second self-position estimation processing, which is based on the second distance information, as self-position estimation processing of the moving body, and outputs self-position information representing the self-position of the moving body on the basis of the processing results of the first self-position estimation processing and the second self-position estimation processing. The present disclosure can be applied to a moving body that performs self-position estimation in an environment where three-dimensional structures change dramatically.

Description

Information processing method, information processing device, and program

[0001] The present disclosure relates to an information processing method, an information processing device, and a program, and more particularly to an information processing method, an information processing device, and a program that enable the robustness of a self-location estimation process to be improved.

[0002] Conventionally, autonomously moving objects are known that estimate their own position based on the detection results of the direction and distance to surrounding objects using a distance sensor and the position information of the objects in pre-stored map information of the moving area.

[0003] For example, Patent Document 1 discloses a self-propelled robot equipped with a height-adjustable range sensor that can set the height of the range sensor to a height corresponding to the shelf boards in each area in a warehouse where shelves of different heights are lined up in each area, and estimates its own position using the measurement results.

[0004] JP 2017-102705 A

[0005] In the technology of Patent Document 1, which detects shelf boards of a fixed height, in an environment where three-dimensional structures change dramatically, the environmental changes have a significant impact on the self-position estimation process.

[0006] The present disclosure has been made in consideration of such circumstances, and aims to improve the robustness of self-location estimation processing.

[0007] The information processing method disclosed herein generates first distance information and second distance information from sensor data output by a ranging sensor mounted on a moving body, and as a self-position estimation process of the moving body, executes a first self-position estimation process based on the first distance information and a second self-position estimation process based on the second distance information, and outputs self-position information representing the self-position of the moving body based on the processing results of each of the first self-position estimation process and the second self-position estimation process.

[0008] The information processing device of the present disclosure is an information processing device that includes a distance information generation unit that generates first distance information and second distance information from sensor data output by a ranging sensor mounted on a moving body, a self-position estimation unit that performs a first self-position estimation process based on the first distance information and a second self-position estimation process based on the second distance information as self-position estimation processes of the moving body, and a self-position information output unit that outputs self-position information representing the self-position of the moving body based on the processing results of each of the first self-position estimation process and the second self-position estimation process.

[0009] The program disclosed herein is a program for causing a computer to generate first distance information and second distance information from sensor data output by a ranging sensor mounted on a moving body, and as a self-position estimation process of the moving body, execute a first self-position estimation process based on the first distance information and a second self-position estimation process based on the second distance information, and execute a process of outputting self-position information representing the self-position of the moving body based on the processing results of each of the first self-position estimation process and the second self-position estimation process.

[0010] In the present disclosure, first distance information and second distance information are generated from sensor data output by a ranging sensor mounted on a moving body, and as a self-position estimation process of the moving body, a first self-position estimation process based on the first distance information is executed, and a second self-position estimation process based on the second distance information is executed, and self-position information representing the self-position of the moving body is output based on the processing results of each of the first self-position estimation process and the second self-position estimation process.

[0011] FIG. 1 is a diagram showing an example of sensing in 3D LiDAR SLAM. FIG. 2 is a diagram showing an example of sensing in 2D LiDAR SLAM. FIG. 3 is a diagram showing a first example of self-localization processing using the technology according to the present disclosure. FIG. 4 is a diagram showing a second example of self-localization processing using the technology according to the present disclosure. A block diagram showing an example configuration of a moving body. A block diagram showing an example functional configuration of an information processing unit. A diagram explaining updating / recreating an environment map based on a score. A diagram explaining position correction between submaps. A flowchart explaining multi-layer self-localization processing. A block diagram showing another example functional configuration of an information processing unit. A flowchart explaining layer search processing. A block diagram showing yet another example functional configuration of an information processing unit. A block diagram showing an example hardware configuration of a computer.

[0012] Modes for carrying out the present disclosure (hereinafter referred to as embodiments) will be described below in the following order.

[0013] 1. Problems with the prior art and an overview of the technology according to the present disclosure 2. Configuration of a moving body 3. First embodiment (configuration for executing multi-layer self-location estimation processing) 4. Second embodiment (configuration for executing layer search processing) 5. Third embodiment (configuration including multiple ranging sensors) 6. Example of computer hardware configuration

[0014] 1. Issues of the Prior Art and Overview of the Technology Relating to the Present Disclosure> (Prior Art and Issues Thereof) LiDAR SLAM (Simultaneous Localization and Mapping) is known, in which a highly accurate environmental map is created using sensor data acquired from a laser sensor (distance sensor) called LiDAR (Light Detection and Ranging) mounted on a mobile body such as a self-propelled robot, and the self-location is estimated by comparing and referencing the map with a map prepared in advance. Representative LiDAR SLAM techniques include 2D LiDAR SLAM, which creates a two-dimensional map using 2D LiDAR or the like to identify the self-location, and 3D LiDAR SLAM, which uses all point cloud information obtained by 3D LiDAR to estimate the three-dimensional self-location.

[0015] In general, 3D LiDAR SLAM uses more features to estimate self-location, and is therefore said to be more robust and less susceptible to environmental changes. In contrast, 2D LiDAR SLAM is more susceptible to environmental changes. Specifically, since 2D LiDAR SLAM senses only a certain height, if there is a change in the environment at that height, self-location estimation may not be possible or the accuracy of self-location estimation may be significantly reduced.

[0016] On the other hand, in environments that differ from normal living environments, such as warehouses where three-dimensional structures change dramatically, the impact of these changes becomes significant even with 3D LiDAR SLAM.

[0017] For example, in the warehouse shown in Figure 1, the three-dimensional environment changes dramatically in a short period of time compared to a normal living environment. 3D LiDAR SLAM uses point cloud information obtained by sensing three-dimensional structures (such as loaded luggage) indicated by the dashed line R1 in the figure, so the impact of environmental changes is significant.

[0018] On the other hand, 2D LiDAR SLAM is less susceptible to environmental changes if it can sense the appropriate height. However, because the sensor height is fixed for each moving object, it may not be applicable to all environments. For example, in the warehouse shown in Figure 2, if the height of a tray on which cargo is loaded, as indicated by the dashed line R2, is sensed, the sensor will be affected by environmental changes such as the tray itself moving or other cargo being loaded around the tray.

[0019] (Overview of Technology According to the Present Disclosure) In the technology according to the present disclosure, for example, point cloud information obtained by 3D LiDAR is divided into layers in the height direction, and self-location estimation processing is performed on multiple layers. Layers with large environmental changes are then avoided (the processing results of the self-location estimation processing are not used), thereby enabling stable output of self-location information.

[0020] Specifically, as shown in Fig. 3, self-localization processing is performed in two layers: 2D LiDAR SLAM using two-dimensional point cloud information of a subject at height L11, and 2D LiDAR SLAM using two-dimensional point cloud information of a subject at height L12. Furthermore, as shown in Fig. 4, self-localization processing may be performed in two layers: 2D LiDAR SLAM using two-dimensional point cloud information of a subject at height L21, and 3D LiDAR SLAM using three-dimensional point cloud information of a subject in height range L22. Furthermore, although not shown, self-localization processing may be performed in three or more layers, including either 2D LiDAR SLAM or 3D LiDAR SLAM.

[0021] In addition, the technology disclosed herein improves robustness against environmental changes by providing feedback between layers through self-location estimation processing in multiple layers, performing stable self-location estimation processing on one side while simultaneously updating the map on the other, and autonomously searching for layers that experience large environmental changes.

[0022] 5 is a block diagram showing an example configuration of the mobile body 1. The mobile body 1 is configured as a mobile robot capable of autonomously moving in various environments, such as a transport robot that transports luggage in a warehouse or an inspection robot that patrols a construction site and collects images. For example, the mobile body 1 may be configured as a cleaning robot that comes into close contact with glass surfaces or walls of a building or house and cleans the surface on which it moves.

[0023] As shown in FIG. 5, the moving object 1 is composed of a sensor 10 , a driving unit 20 , a communication unit 30 , a storage unit 40 , and an information processing unit 100 .

[0024] The sensor 10 is configured to include, in addition to a distance measurement sensor, an RGB camera, a collision prevention sensor, a speed sensor, an acceleration sensor, etc. The sensor data acquired by the sensor 10 is supplied to the information processing unit 100.

[0025] The driving unit 20 is configured as a motor or the like that rotates the wheels of the mobile object 1. The driving unit 20 drives the mobile object 1 based on the control of the information processing unit 100, thereby allowing the mobile object 1 to move.

[0026] The communication unit 30 is a wireless communication module that performs wireless communication with an external device. The communication unit 30 supplies information received via wireless communication to the information processing unit 100, and transmits information supplied from the information processing unit 100 via wireless communication.

[0027] The storage unit 40 is configured by a volatile memory such as a dynamic random access memory (DRAM), etc. The storage unit 40 stores various data obtained by the calculations of the information processing unit 100.

[0028] The information processing unit 100 is configured with processors such as a CPU (Central Processing Unit) and a GPU (Graphics Processing Unit). The information processing unit 100 controls each unit of the mobile object 1. The configuration of the information processing unit 100 according to an embodiment of the present disclosure will be described below.

[0029] 3. First Embodiment (Configuration for Executing Multi-Layer Self-Location Estimation Processing) FIG. 6 is a block diagram showing an example of the functional configuration of the information processing unit 100 according to the first embodiment of the present disclosure.

[0030] The information processing unit 100 in FIG. 6 can execute the following processes by executing a program stored in a memory (not shown).

[0031] First, the information processing unit 100 generates first distance information and second distance information from sensor data output by a ranging sensor 111 included in the sensor 10. The ranging sensor 111 is configured with a 3D sensor capable of acquiring three-dimensional distance information in the environment, such as a 3D LiDAR or a 3D depth sensor. That is, the information processing unit 100 can extract first distance information of a first layer and second distance information of a second layer from the sensor data (three-dimensional distance information) output by a single ranging sensor 111.

[0032] Next, the information processing unit 100 performs a first self-location estimation process based on the first distance information and a second self-location estimation process based on the second distance information as a self-location estimation process for the moving object 1. That is, the information processing unit 100 performs a self-location estimation process in multiple layers. Hereinafter, the self-location estimation process in multiple layers will also be referred to as a multi-layer self-location estimation process. Note that, in the following description, the multiple layers will be described as layers that differ in the height direction relative to a horizontal plane, but they may also be layers that differ in the horizontal direction (left-right direction) or layers that differ in the diagonal direction.

[0033] Then, the information processing unit 100 outputs self-location information representing the self-location of the moving object 1 based on the scores of the processing results of the first self-location estimation processing and the second self-location estimation processing.

[0034] The layers into which the sensor data output by the ranging sensor 111 is divided are set based on environmental information about the environment in which the mobile object 1 moves. The environment in which the mobile object 1 moves may be an environment in which three-dimensional structures change from moment to moment, such as a parking lot, a factory, or a live performance stage, in addition to the warehouse described above. The environmental information may be, for example, height information input by the user via the UI presentation unit 112 depending on the variability of the environment in which the mobile object 1 moves, or height information predetermined for each environment in which the mobile object 1 moves. When height information is input by the user, the user can specify a layer (height) in which the environment changes little. In this way, the sensor data output by the ranging sensor 111 is divided into layers of heights corresponding to the environment in which the mobile object 1 moves.

[0035] The UI presentation unit 112 presents various UIs in addition to the UI (User Interface) for inputting the above-described environmental information. For example, the UI presentation unit 112 can display an environmental map of each layer used in the multi-layer self-location estimation process executed in the information processing unit 100.

[0036] The information processing unit 100 executes a program stored in a memory (not shown) to realize the functional blocks of a sensor data dividing unit 120, a multi-layer self-location estimating unit 130, and a self-location integrating unit 140.

[0037] The sensor data division unit 120 functions as a distance information generation unit that divides the sensor data output by the ranging sensor 111 into multiple layers and generates first distance information and second distance information from each of the divided sensor data. Specifically, the sensor data division unit 120 generates the first distance information and the second distance information by dividing the sensor data output by the ranging sensor 111 into layers in the height direction relative to the plane of movement of the moving object 1 based on environmental information (height information) input by the user via the UI presentation unit 112 or determined in advance.

[0038] The first distance information and the second distance information may each be two-dimensional point cloud information of the subject (environment) at a predetermined height relative to the plane of movement of the moving body 1, or three-dimensional point cloud information of the subject (environment) within a predetermined height range.

[0039] Here, point cloud information of the first layer and point cloud information of the second layer (first distance information and second distance information) are extracted from the sensor data output by the ranging sensor 111, but point cloud information of three or more layers can also be extracted.

[0040] The multi-layer self-location estimator 130 simultaneously performs self-location estimation processing in multiple layers in parallel. In the multi-layer self-location estimator 130, functional blocks that realize the self-location estimation processing in each layer are denoted by reference numerals with sub-numbers for each layer.

[0041] That is, the multi-layer self-location estimation unit 130 has a self-location estimation unit 131-1, a score determination unit 132-1, a map update unit 133-1, and an environment map storage unit 134-1 for the first layer, and also has a self-location estimation unit 131-2, a score determination unit 132-2, a map update unit 133-2, and an environment map storage unit 134-2 for the second layer. Hereinafter, when the layers are not to be distinguished from one another, the functional blocks for each layer will be described without the sub-numbers for each layer.

[0042] The self-position estimation unit 131 uses the environment map stored in the environment map storage unit 134 to perform a self-position estimation process based on the distance information of the layer, and supplies the processing result to the score determination unit 132. The environment map storage unit 134 stores an environment map of the layer that has been prepared in advance.

[0043] The score determination unit 132 calculates a self-location estimation score (hereinafter simply referred to as a score) that indicates the reliability of the processing result from the self-location estimation unit 131. The reliability of the processing result may be calculated based on the covariance of the distance information, or may be calculated based on the matching result of the distance information with the environment map. The score determination unit 132 supplies the processing result from the self-location estimation unit 131 together with the calculated score to the self-location integrating unit 140. The self-location integrating unit 140 functions as a self-location information output unit that outputs self-location information that indicates the self-location of the mobile object 1 based on the scores of the processing results of the self-location estimation processing of each layer from the score determination unit 132.

[0044] Furthermore, the score determination unit 132 determines whether the score of the processing result is smaller than a preset threshold value, and supplies the determination result to the map update unit 133. The map update unit 133 recreates or updates the environment map held in the environment map holding unit 134 in accordance with the determination result from the map update unit 133.

[0045] For example, as shown in FIG. 7, it is assumed that the multi-layer self-location estimating unit 130 is executing the multi-layer self-location estimation process for three layers.

[0046] The self-location estimation unit 131-1 uses a first layer environment map MP1 prepared in advance to perform self-location estimation processing based on distance information of the layer. The self-location estimation unit 131-2 uses a second layer environment map MP2 prepared in advance to perform self-location estimation processing based on distance information of the layer. The self-location estimation unit 131-3 uses a third layer environment map MP3 prepared in advance to perform self-location estimation processing based on distance information of the layer.

[0047] Then, the score determination unit 132 determines whether or not the scores of the results of the self-position estimation processes for each layer performed by the self-position estimation units 131-1, 131-2, and 131-3 are smaller than a preset threshold value.

[0048] 7, the score of the processing result for the first layer is determined to be smaller than the threshold, and the environment map MP1 for that layer is updated or recreated. On the other hand, the scores of the processing result for the second layer and the third layer are determined to be larger than the threshold, and are supplied to the self-location integrating unit 140.

[0049] In this way, with the multi-layer self-position estimation process, even if an environmental change occurs in one layer, the environmental map of the layer in which the environmental change occurred can be updated while the self-position estimation process is stably performed in other layers.

[0050] In the example of Figure 7, in the multi-layer self-location estimation process, the self-location estimation process of each layer is executed as a loosely coupled process, but it is also possible to execute the self-location estimation process of each layer as a tightly coupled process.

[0051] When the self-location estimation process for each layer is executed as a tightly coupled process, the positional relationship between the submaps that make up each environmental map may be used in creating the environmental map for each layer.

[0052] For example, as shown in FIG. 8, it is assumed that the multi-layer self-location estimating unit 130 creates an environment map while executing multi-layer self-location estimation processing for two layers.

[0053] The self-location estimation unit 131-1 creates submaps SM11 to SM14 for the first layer for each region in the environment, and executes a self-location estimation process based on distance information for that layer. The self-location estimation unit 131-2 creates submaps SM21 to SM24 for the second layer for each region in the environment, and executes a self-location estimation process based on distance information for that layer.

[0054] In the example of Figure 8, submaps SM11 to SM14 for the first layer are connected together based on their commonalities to form a closed loop. This creates an environment map MP1 for the entire environment for the first layer. However, submaps SM21 to SM24 for the second layer do not have enough commonalities to form a closed loop.

[0055] At this time, the positions of the corresponding submaps SM21 to SM24 are corrected based on the positional relationships between the submaps SM11 to SM14 that make up the environment map MP1. This creates a closed loop among the submaps SM21 to SM24, and creates an environment map MP2 of the entire environment for the second layer. As a result, consistency between the environment maps for each layer can be improved.

[0056] The multi-layer self-localization processing executed by the information processing unit 100 in Fig. 6 will be described with reference to the flowchart in Fig. 9. The processing in Fig. 9 is started, for example, when the moving body 1 as a mobile robot starts up in the environment.

[0057] In step S11, the sensor data division unit 120 divides the sensor data output by the ranging sensor 111 into multiple layers and supplies distance information (point cloud information) for each divided layer to the self-position estimation unit 131 of each layer.

[0058] In step S12, the self-position estimation unit 131 of each layer reads the environment map from the environment map storage unit 134 for each layer.

[0059] In step S13, the self-position estimation unit 131 of each layer uses the environment map read from the environment map holding unit 134 to perform self-position estimation processing based on the distance information (point cloud information) of the layer.

[0060] In step S14, the score determination unit 132 for each layer determines whether or not there is a score that is smaller than the threshold among the scores of the self-location estimation process results for each layer. If there is no score that is smaller than the threshold, that is, if the scores of all the process results are larger than the threshold, the process proceeds to step S15.

[0061] In step S15, the self-location integrating unit 140 extracts all the processing results from the score determining unit 132 of each layer.

[0062] In step S16, the self-location integrating unit 140 integrates the processing results of the self-location estimation processing of each layer based on the score. For example, the self-location integrating unit 140 integrates the processing results (coordinate information) of the self-location estimation processing of each layer according to the score to obtain the self-location information. In this case, the coordinate information may be integrated based on an extended Kalman filter using covariance, or may be integrated based on factor grabbing. Furthermore, the self-location integrating unit 140 may use the processing result (coordinate information) with the largest score as the self-location information.

[0063] Then, in step S17, the self-location integrating unit 140 outputs the self-location information.

[0064] On the other hand, if it is determined in step S14 that there is a score that is smaller than the threshold, the process proceeds to step S18, where the score determination unit 132 of each layer determines whether all scores are smaller than the threshold. If it is determined that all scores are not smaller than the threshold, the process proceeds to step S19.

[0065] In step S19, the map update unit 133 of the layer corresponding to the score smaller than the threshold updates or recreates the environment map stored in the environment map storage unit 134 of that layer.

[0066] In step S20, the self-location integrating unit 140 extracts the processing results of the scores greater than the threshold from the score determining unit 132 of the layer corresponding to the score greater than the threshold. Then, in step S16, the processing results are integrated based on the scores, and in step S17, the self-location information is output.

[0067] Now, if it is determined in step S18 that all scores are smaller than the threshold value, the process proceeds to step S21.

[0068] In step S21, the self-location integrating unit 140 determines the processing result (coordinate information) with the largest score among all scores smaller than the threshold as the self-location information.

[0069] In step S22, the map update unit 133 of the other layer updates or recreates the environment map held in the environment map holding unit 134 of the layer.

[0070] Then, in step S17, the processing result with the largest score among all scores smaller than the threshold is output as the self-location information. This allows the self-location estimation process to continue in the layer with the least environmental change even if there is an environmental change in all layers, thereby preventing unexpected movement stops of the moving object 1.

[0071] According to the above processing, the self-location estimation process is performed in multiple layers, which allows the self-location information to be output stably while avoiding layers with large environmental changes, thereby improving the robustness of the self-location estimation process.

[0072] 4. Second Embodiment (Configuration for Executing Layer Search Processing) FIG. 10 is a block diagram showing an example of the functional configuration of an information processing unit 100 according to a second embodiment of the present disclosure.

[0073] In the information processing unit 100 of Fig. 10, functional blocks similar to those of the information processing unit 100 of Fig. 6 are denoted by the same reference numerals, and descriptions thereof will basically be omitted. That is, the information processing unit 100 of Fig. 10 differs from the information processing unit 100 of Fig. 6 in that it newly includes a layer search self-position estimation unit 210 and a score comparison unit 220.

[0074] However, the sensor data division unit 120 in the information processing unit 100 in Fig. 10 divides the sensor data output by the ranging sensor 111 into multiple layers and generates third distance information in a layer different from the first distance information and the second distance information. That is, in addition to the point cloud information in the first layer and the point cloud information in the second layer, point cloud information in the third layer is extracted from the sensor data output by the ranging sensor 111. Hereinafter, the third layer will be referred to as a search layer. The search layer may be set according to the above-mentioned environmental information, or may be sequentially changed or randomly set according to the environment in which the mobile object 1 moves.

[0075] The layer search self-location estimation unit 210 uses the environment map stored in the environment map storage unit (not shown) to perform self-location estimation processing based on the third distance information (point cloud information of the search layer), and calculates a score for the processing result. The calculated score is supplied to the score comparison unit 220.

[0076] The score comparison unit 220 acquires the score of the processing result of the self-location estimation process for each layer in the multi-layer self-location estimation unit 130, and compares it with the score of the processing result of the self-location estimation process for the search layer from the score comparison unit 220. The score comparison result is supplied to the sensor data division unit 120.

[0077] The sensor data dividing unit 120 changes the layer into which the sensor data output by the ranging sensor 111 is divided, based on the score comparison result from the score comparing unit 220. In other words, the sensor data dividing unit 120 changes the layer into which the sensor data output by the ranging sensor 111 is divided, based on the score of the processing result of the self-position estimation processing based on the third distance information (point cloud information of the search layer).

[0078] That is, the self-position estimation process executed by the layer search self-position estimation unit 210 can be said to be a layer search self-position estimation process for searching for a layer having a specific environmental change (or a layer without an environmental change).

[0079] The layer search process executed by the information processing unit 100 in Fig. 10 will be described with reference to the flowchart in Fig. 11. The process in Fig. 11 is executed in parallel with the multi-layer self-location estimation process described with reference to the flowchart in Fig. 9.

[0080] In step S21, the self-position estimation unit 210 for layer search reads an environment map from an environment map holding unit (not shown), performs self-position estimation processing for layer search based on distance information (point cloud information) of the search layer, and calculates a score for the processing result.

[0081] In step S22, the score comparison unit 220 determines whether the score (reliability) of the processing result of the self-location estimation process for layer search is higher than the processing result of the self-location estimation process for each layer in the multi-layer self-location estimation unit 130. If it is determined that the score of the processing result of the self-location estimation process for layer search is not higher than the processing result of the self-location estimation process for each layer, the process returns to step S21, and the self-location estimation process for layer search is repeated.

[0082] On the other hand, if it is determined that the score of the processing result of the self-position estimation processing for layer search is higher than the processing result of the self-position estimation processing of each layer, proceed to step S23, and the sensor data division unit 120 changes the layer into which the sensor data output by the ranging sensor 111 is divided.

[0083] For example, if the score (reliability) of the processing result of the self-location estimation process for layer search is higher than the score (reliability) of the processing result of the self-location estimation process for the first layer, the first layer is changed to a search layer as the layer into which the sensor data is divided. Also, if the score of the processing result of the self-location estimation process for layer search is higher than the score of the processing result of the self-location estimation process for the second layer, the second layer is changed to a search layer as the layer into which the sensor data is divided. Furthermore, if the score of the processing result of the self-location estimation process for layer search is higher than the scores of the processing results of the self-location estimation processes for all layers, the layers into which the sensor data is divided are changed so that at least the search layer is included.

[0084] According to the above processing, in the multi-layer self-location estimation processing, a layer with large environmental changes can be autonomously searched for and changed to a layer with small environmental changes, thereby making it possible to further improve robustness against environmental changes.

[0085] 5. Third Embodiment (Configuration Including Multiple Distance Measuring Sensors) FIG. 12 is a block diagram showing an example of the functional configuration of an information processing unit 100 according to a third embodiment of the present disclosure.

[0086] In the information processing unit 100 in Fig. 12, functional blocks that are the same as those in the information processing unit 100 in Fig. 6 are denoted by the same reference numerals, and descriptions thereof will basically be omitted. That is, the information processing unit 100 in Fig. 12 differs from the information processing unit 100 in Fig. 6 in that it has sensor data acquisition units 310-1 and 310-2 instead of the sensor data division unit 120.

[0087] 12 generates first distance information and second distance information from sensor data output by distance measurement sensors 111-1 and 111-2 mounted at different height positions in the moving object 1. Each of the distance measurement sensors 111-1 and 111-2 may be configured as a 3D sensor capable of acquiring three-dimensional distance information in the environment, or may be configured as a 2D sensor capable of acquiring two-dimensional distance information in the environment, such as a 2D LiDAR or a 2D depth sensor.

[0088] That is, the sensor data acquisition unit 310-1 acquires sensor data output by the ranging sensor 111-1 and generates first distance information. When the ranging sensor 111-1 is configured as a 3D sensor, the sensor data acquisition unit 310-1 extracts first distance information of a first layer, which is set based on, for example, environmental information, from the sensor data (three-dimensional distance information) output by the ranging sensor 111-1, and supplies the first distance information to the multi-layer self-position estimation unit 130. When the ranging sensor 111-1 is configured as a 2D sensor, the sensor data acquisition unit 310-1 acquires sensor data (two-dimensional distance information) output by the ranging sensor 111-1 and supplies the sensor data to the multi-layer self-position estimation unit 130.

[0089] Similarly, the sensor data acquisition unit 310-2 acquires sensor data output by the ranging sensor 111-2 and generates second distance information. If the ranging sensor 111-2 is configured as a 3D sensor, the sensor data acquisition unit 310-2 extracts second distance information of a second layer, which is set based on, for example, environmental information, from the sensor data (three-dimensional distance information) output by the ranging sensor 111-2, and supplies the second distance information to the multi-layer self-position estimation unit 130. If the ranging sensor 111-2 is configured as a 2D sensor, the sensor data acquisition unit 310-2 acquires sensor data (two-dimensional distance information) output by the ranging sensor 111-2 and supplies the second distance information to the multi-layer self-position estimation unit 130.

[0090] Even in such a configuration, by executing the self-location estimation process in multiple layers, it is possible to avoid layers with large environmental changes and output self-location information stably, thereby improving the robustness of the self-location estimation process.

[0091] 6. Example of Computer Hardware Configuration The above-described series of processes can be executed by hardware or software. When the series of processes is executed by software, the program constituting the software is installed from a program recording medium into a computer incorporated in dedicated hardware, a general-purpose personal computer, or the like.

[0092] 13 is a block diagram showing an example of the hardware configuration of a computer that executes the above-described series of processes using a program. The information processing unit 100 that may be included in the mobile object 1 may be configured, for example, by a computer 400 having a configuration similar to that shown in FIG.

[0093] A CPU (Central Processing Unit) 401 , a ROM (Read Only Memory) 402 , and a RAM (Random Access Memory) 403 are interconnected by a bus 404 .

[0094] An input / output interface 405 is also connected to the bus 404. An input unit 406 including a keyboard, a mouse, etc., and an output unit 407 including a display, a speaker, etc. are connected to the input / output interface 405. Also connected to the input / output interface 405 are a storage unit 408 including a hard disk, a nonvolatile memory, etc., a communication unit 409 including a network interface, etc., and a drive 410 that drives removable media 411.

[0095] In the computer 400 configured as described above, the CPU 401 performs the above-described series of processes by, for example, loading a program stored in the storage unit 408 into the RAM 403 via the input / output interface 405 and the bus 404 and executing it.

[0096] The program executed by the CPU 401 is provided, for example, by being recorded on a removable medium 411 or via a wired or wireless transmission medium such as a local area network, the Internet, or digital broadcasting, and is installed in the storage unit 408 .

[0097] The program executed by computer 400 may be a program that processes in chronological order according to the order described in this specification, or may be a program that processes in parallel or at the required timing, such as when called.

[0098] In this specification, a system refers to a collection of multiple components (devices, modules (components), etc.), regardless of whether all of the components are contained in the same housing. Therefore, multiple devices housed in separate housings and connected via a network, and a single device housed in a single housing with multiple modules, are both systems.

[0099] The effects described in this specification are merely examples and are not limiting, and other effects may also be present.

[0100] The embodiments of the present disclosure are not limited to the above-described embodiments, and various modifications are possible within the scope of the gist of the present disclosure.

[0101] For example, the embodiment of the present disclosure can be configured as a cloud computing system in which a single function is shared and processed collaboratively by multiple devices via a network.

[0102] Furthermore, each step described in the above flowchart can be executed by one device, or can be shared and executed by a plurality of devices.

[0103] Furthermore, when one step includes multiple processes, the multiple processes included in that one step can be executed by one device or can be shared and executed by multiple devices.

[0104] The effects described in this specification are merely examples and are not limiting, and other effects may also be present.

[0105] Furthermore, the technology according to the present disclosure may have the following configurations: (1) An information processing method that generates first distance information and second distance information from sensor data output by a distance measurement sensor mounted on a moving object, performs a first self-location estimation process based on the first distance information and a second self-location estimation process based on the second distance information as self-location estimation processes for the moving object, and outputs self-location information representing the self-location of the moving object based on the results of the first self-location estimation process and the second self-location estimation process. (2) The information processing method described in (1), in which the first distance information and the second distance information are generated from each of the sensor data obtained by dividing the sensor data into a plurality of layers. (3) The information processing method described in (2), in which the first distance information and the second distance information are generated from each of the sensor data divided into the layers in a height direction relative to the plane of movement of the moving object. (4) The information processing method according to (3), wherein the first distance information and the second distance information are either two-dimensional point cloud information of the subject at a predetermined height relative to the plane of movement or three-dimensional point cloud information of the subject within a predetermined height range. (5) The information processing method according to (4), wherein the first distance information and the second distance information are extracted from the sensor data output by a single ranging sensor. (6) The information processing method according to (4), wherein the first distance information and the second distance information are generated from the sensor data output by ranging sensors mounted at different height positions. (7) The information processing method according to any one of (1) to (6), wherein the self-location information is output based on scores of the processing results of the first self-location estimation process and the second self-location estimation process, the scores representing reliability of the processing results. (8) The information processing method according to (7), wherein the reliability is calculated based on the covariance of distance information or a matching result. (9) The information processing method according to (7) or (8), wherein the processing result with the largest score is output as the self-location information. (10) The information processing method according to (7) or (8), wherein the processing results integrated according to the score are output as the self-location information.(11) The information processing method according to any one of (7) to (10), in which the first self-localization processing is performed using a first map as an environmental map, and the second self-localization processing is performed using a second map different from the first map, and the environmental map used in the self-localization processing, in which the score of the processing result is smaller than a threshold, is recreated or updated. (12) The information processing method according to (11), in which, if the scores of all the processing results are smaller than the threshold, the processing result with the largest score is output as the self-localization information. (13) The information processing method according to (11) or (12), in which positions between the sub-maps constituting one of the first map and the second map are corrected based on a positional relationship between the sub-maps constituting the other of the first map and the second map. (14) The information processing method according to any one of (2) to (13), in which the layer for dividing the sensor data is set based on environmental information related to an environment in which the mobile object moves. (15) The information processing method according to (14), wherein the environmental information is altitude information input by a user depending on the variability of the environment. (16) The information processing method according to (14), wherein the environmental information is altitude information determined in advance for each environment. (17) The information processing method according to any one of (2) to (16), wherein the layer into which the sensor data is divided is changed based on the processing result of a third self-location estimation process based on third distance information of the layer different from the first distance information and the second distance information. (18) The information processing method according to (17), wherein the layer into which the sensor data is divided is changed when the reliability of the processing result of the third self-location estimation process is higher than the reliability of the processing result of at least one of the first self-location estimation process and the second self-location estimation process.(19) An information processing device comprising: a distance information generating unit that generates first distance information and second distance information from sensor data output by a distance measuring sensor mounted on a moving body, a self-location estimating unit that executes, as a self-location estimation process of the moving body, a first self-location estimation process based on the first distance information and a second self-location estimation process based on the second distance information, and a self-location information output unit that outputs self-location information representing the self-location of the moving body based on processing results of the first self-location estimation process and the second self-location estimation process. (20) A program that causes a computer to execute a process of generating first distance information and second distance information from sensor data output by a distance measuring sensor mounted on a moving body,

[0106] REFERENCE SIGNS LIST 1 Mobile object, 10 Sensor, 20 Drive unit, 30 Communication unit, 40 Memory unit, 100 Information processing unit, 111 Distance measurement sensor, 112 UI presentation unit, 120 Sensor data division unit, 130 Multi-layer self-position estimation unit, 131 Self-position estimation unit, 132 Score determination unit, 133 Map update unit, 134 Environmental map storage unit, 140 Self-position integration unit, 210 Self-position estimation unit for layer search, 220 Score comparison unit

Claims

1. An information processing method comprising: generating first distance information and second distance information from sensor data output by a distance measurement sensor mounted on a moving body; executing a first self-location estimation process based on the first distance information and a second self-location estimation process based on the second distance information as self-location estimation processes for the moving body; and outputting self-location information representing the self-location of the moving body based on the processing results of the first self-location estimation process and the second self-location estimation process.

2. The information processing method according to claim 1, wherein the sensor data is divided into a plurality of layers, and the first distance information and the second distance information are generated from each of the sensor data.

3. The information processing method according to claim 2, wherein the first distance information and the second distance information are generated from the sensor data divided into layers in the height direction relative to the plane of movement of the moving object.

4. The information processing method according to claim 3, wherein the first distance information and the second distance information are either two-dimensional point cloud information of the subject at a predetermined height relative to the moving plane or three-dimensional point cloud information of the subject within a predetermined height range.

5. The information processing method according to claim 4, wherein the first distance information and the second distance information are extracted from the sensor data output by a single distance measuring sensor.

6. The information processing method according to claim 4, wherein the first distance information and the second distance information are generated from the sensor data output by the distance measuring sensors mounted at different height positions.

7. The information processing method according to claim 1, wherein the self-location information is output based on scores of the processing results of the first self-location estimation process and the second self-location estimation process, and the scores represent the reliability of the processing results.

8. The information processing method according to claim 7, wherein the reliability is calculated based on the covariance of distance information or a matching result.

9. The information processing method according to claim 7, wherein the processing result with the largest score is output as the self-location information.

10. The information processing method according to claim 7, wherein the processing results integrated according to the score are output as the self-location information.

11. The information processing method according to claim 7, wherein the first self-location estimation process is performed using a first map as an environmental map, and the second self-location estimation process is performed using a second map different from the first map, and the environmental map used in the self-location estimation process for which the score of the processing result is smaller than a threshold value is recreated or updated.

12. The information processing method according to claim 11, wherein, when the scores of all the processing results are smaller than the threshold value, the processing result with the largest score is output as the self-location information.

13. The information processing method according to claim 11, wherein the position between the sub-maps constituting one of the first map and the second map is corrected based on the positional relationship between the sub-maps constituting the other of the first map and the second map.

14. The information processing method according to claim 2, wherein the layers for dividing the sensor data are set based on environmental information relating to an environment in which the mobile object moves.

15. The information processing method according to claim 14, wherein the environmental information is altitude information input by the user according to the variability of the environment.

16. The information processing method according to claim 14, wherein the environmental information is altitude information determined in advance for each of the environments.

17. The information processing method described in claim 7, wherein the layer into which the sensor data is divided is changed based on the processing result of a third self-position estimation process based on third distance information of the layer different from the first distance information and the second distance information.

18. The information processing method according to claim 17, wherein the layer into which the sensor data is divided is changed if the reliability of the processing result of the third self-location estimation process is higher than the reliability of the processing result of at least one of the first self-location estimation process and the second self-location estimation process.

19. An information processing device comprising: a distance information generation unit that generates first distance information and second distance information from sensor data output by a distance measurement sensor mounted on a moving body; a self-location estimation unit that executes a first self-location estimation process based on the first distance information and a second self-location estimation process based on the second distance information as self-location estimation processes for the moving body; and a self-location information output unit that outputs self-location information representing the self-location of the moving body based on the processing results of the first self-location estimation process and the second self-location estimation process.

20. A program for causing a computer to execute the following process: generate first distance information and second distance information from sensor data output by a distance measuring sensor mounted on a moving object; execute a first self-location estimation process based on the first distance information and a second self-location estimation process based on the second distance information as self-location estimation processes for the moving object; and output self-location information representing the self-location of the moving object based on the results of the first self-location estimation process and the second self-location estimation process.

Images