Information processing method, information processing apparatus, and program

Abstract

The present disclosure relates to an information processing method, an information processing apparatus, and a program capable of improving robustness of self-position estimation processing. An information processing unit generates first distance information and second distance information from sensor data output by a distance measuring sensor mounted on a mobile body, executes first self-position estimation processing based on the first distance information and second self-position estimation processing based on the second distance information as self-position estimation processing of the mobile body, and outputs self-position information indicating a self-position of the mobile body based on respective processing results of the first self-position estimation processing and the second self-position estimation processing. The present disclosure can be applied to a mobile body that executes self-position estimation in an environment in which a three-dimensional structure changes drastically.

Description

Technical Field

[0001] This disclosure relates to information processing methods, information processing apparatuses and programs, and more particularly to information processing methods, information processing apparatuses and programs that enable the improvement of the robustness of self-position estimation processing. Background Technology

[0002] Typically, a mobile entity is known to move autonomously by estimating its own position based on the detection results of the orientation and distance of surrounding objects detected by a ranging sensor and the position information of objects in a pre-stored map of the mobile area.

[0003] For example, Patent Document 1 discloses a self-propelled robot that includes a distance sensor with a height variable. In a warehouse where shelves of different heights are arranged in each area, the height of the distance sensor is set to correspond to the height of the shelf panel in each area, and the robot estimates its own position by using the measurement results.

[0004] Citation List

[0005] Patent documents

[0006] Patent Document 1: Japanese Patent Application Publication No. 2017-102705 Summary of the Invention

[0007] The problem to be solved by the present invention

[0008] In the technology of Patent Document 1 for detecting shelf panels with a constant height, environmental changes greatly affect the position estimation process in environments where the 3D structure changes rapidly.

[0009] This disclosure is made in view of the circumstances and to enable improvements in the robustness of its own position estimation processing.

[0010] Solution to the problem

[0011] The information processing method according to this disclosure includes: generating first distance information and second distance information based on sensor data output by a ranging sensor mounted on a moving body; performing a first self-position estimation process based on the first distance information and performing a second self-position estimation process based on the second distance information as self-position estimation process for the moving body; and outputting self-position information indicating the self-position of the moving body based on the respective processing results of the first self-position estimation process and the second self-position estimation process.

[0012] The information processing apparatus according to this disclosure includes: a distance information generation unit that generates first distance information and second distance information based on sensor data output by a ranging sensor mounted on a moving body; a self-position estimation unit that performs a first self-position estimation process based on the first distance information and a second self-position estimation process based on the second distance information as a self-position estimation process for the moving body; and a self-position information output unit that outputs self-position information indicating the self-position of the moving body based on the respective processing results of the first self-position estimation process and the second self-position estimation process.

[0013] The program according to this disclosure is a program for causing a computer to perform the following processes: generating first distance information and second distance information based on sensor data output by a ranging sensor mounted on a moving body; performing a first self-position estimation process based on the first distance information and performing a second self-position estimation process based on the second distance information as a self-position estimation process for the moving body; and outputting self-position information indicating the self-position of the moving body based on the respective processing results of the first self-position estimation process and the second self-position estimation process.

[0014] In this disclosure, first distance information and second distance information are generated based on sensor data output by a ranging sensor mounted on a moving body. A first self-position estimation process based on the first distance information and a second self-position estimation process based on the second distance information are performed as self-position estimation processes for the moving body. Based on the processing results of the first self-position estimation process and the second self-position estimation process, self-position information indicating the self-position of the moving body is output. Attached Figure Description

[0015] Figure 1 This is a diagram illustrating an example of sensing in 3D LiDAR SLAM.

[0016] Figure 2 This is a diagram illustrating an example of sensing in 2D LiDAR SLAM.

[0017] Figure 3 This is a diagram illustrating a first example of a self-position estimation process based on the technology of this disclosure.

[0018] Figure 4 This is a diagram illustrating a second example of self-position estimation processing according to the technology of this disclosure.

[0019] Figure 5 This is a block diagram illustrating an example configuration of a moving body.

[0020] Figure 6 This is a block diagram illustrating an example of the functional configuration of an information processing unit.

[0021] Figure 7 This is a diagram illustrating the updating / recreation of the environment map based on scores.

[0022] Figure 8 This is a diagram illustrating the correction of positions between submaps.

[0023] Figure 9 This is a flowchart illustrating the multi-layer self-position estimation process.

[0024] Figure 10 This is a block diagram illustrating another example of the functional configuration of the information processing unit.

[0025] Figure 11 This is a flowchart illustrating the layer search process.

[0026] Figure 12 This is a block diagram illustrating yet another example of a functional configuration of the information processing unit.

[0027] Figure 13 This is a block diagram illustrating an example of a computer's hardware configuration. Detailed Implementation

[0028] In the following text, a mode for implementing the present disclosure (hereinafter referred to as an implementation mode) will be described. Note that the description is given in the following order.

[0029] 1. Problems with conventional techniques and an overview of the techniques according to this disclosure

[0030] 2. Configuration of moving units

[0031] 3. First Implementation (Configuration for Performing Multi-Layer Self-Position Estimation Processing)

[0032] 4. Second Implementation Method (Configuration of Execution Layer Search Processing)

[0033] 5. Third implementation method (including configuration of multiple ranging sensors)

[0034] 6. Example of computer hardware configuration

[0035] <1. Problems with conventional techniques and an overview of the techniques according to this disclosure>

[0036] (Conventional techniques and their problems)

[0037] LiDAR (Light Detection and Ranging) Simultaneous Localization and Mapping (SLAM) is known. It performs self-position estimation by creating a high-precision environmental map based on sensor data acquired from a laser sensor (distance sensor) called LiDAR mounted on a moving body (e.g., a self-propelled robot) and comparing and referencing it to a pre-prepared map. Representative methods of LiDAR SLAM include 2D LiDAR SLAM and 3D LiDAR SLAM. 2D LiDAR SLAM creates a two-dimensional map using 2D LiDAR and performs self-position recognition, while 3D LiDAR SLAM performs three-dimensional self-position estimation using all point cloud information obtained from 3D LiDAR.

[0038] Generally, 3D LiDAR SLAM is considered robust because it performs self-position estimation using numerous features and therefore exhibits a low rate of change relative to environmental variations. In contrast, in 2D LiDAR SLAM, the impact of environmental changes is much greater. Specifically, in 2D LiDAR SLAM, because sensing is performed only at a specific constant height, self-position estimation cannot be performed, or its accuracy is significantly reduced, when the environment at that height changes.

[0039] On the other hand, in environments that differ from typical living environments, such as warehouses where 3D structures change drastically, even 3D LiDAR SLAM can lead to a greater proportion of the impact of these changes.

[0040] For example, in Figure 1 In the warehouse shown, the 3D environment changes drastically in a short period of time compared to a typical living environment. In 3D LiDAR SLAM, point cloud information obtained by sensing the 3D structure (loaded goods, etc.) shown by the dashed line R1 in the figure is used, and therefore, the impact of environmental changes is amplified.

[0041] On the other hand, in 2D LiDAR SLAM, as long as a suitable height can be sensed, 2D LiDAR SLAM is not easily affected by environmental changes. However, because the sensor height is fixed for each moving object, the sensor height may not be suitable for every environment. For example, in... Figure 2 In the warehouse shown, when sensing is performed at the height of the pallet or other container loaded with goods, as indicated by the dashed line R2, there is an effect of environmental changes (such as the movement of the pallet itself or the loading of other goods around the pallet).

[0042] (An overview of the technology based on this disclosure)

[0043] In the techniques according to this disclosure, for example, point cloud information obtained by 3D LiDAR is segmented into layers in the height direction, enabling the execution of self-position estimation processing in multiple layers. Then, by avoiding layers with large environmental variations (where the processing results of self-position estimation are not used), a stable output of self-position information can be executed.

[0044] Specifically, such as Figure 3 As shown, self-position estimation processing is performed in two layers: 2D LiDAR SLAM using the 2D point cloud information of the object at height L11 and 2D LiDAR SLAM using the 2D point cloud information of the object at height L12. Furthermore, as... Figure 4 As shown, self-position estimation processing can be performed in two layers: 2D LiDAR SLAM using 2D point cloud information of the object at height L21 and 3D LiDAR SLAM using 3D point cloud information of the object in height range L22. Additionally, although not illustrated, self-position estimation processing, including 2D LiDAR SLAM or 3D LiDAR SLAM, can be performed in three or more layers.

[0045] Furthermore, in the technology according to this disclosure, robustness to environmental changes is improved by inter-layer feedback through self-position estimation processing in multiple layers, by performing stable self-position estimation processing on one layer while simultaneously performing map updates on another layer, and by autonomously searching for layers with large environmental changes.

[0046] <2. Configuration of moving objects>

[0047] Figure 5 This is a block diagram illustrating an example configuration of mobile body 1. Mobile body 1 is configured as a mobile robot capable of autonomous movement in various environments, such as a transport robot for transporting goods in a warehouse or similar location, and an inspection robot for conducting inspections and collecting images at a construction site. For example, mobile body 1 can be configured as a cleaning robot that comes into close contact with the glass surfaces, wall surfaces, etc., of buildings or residences and cleans the surfaces it travels on.

[0048] like Figure 5 As shown, the mobile body 1 includes a sensor 10, a drive unit 20, a communication unit 30, a storage unit 40, and an information processing unit 100.

[0049] In addition to the ranging sensor, sensor 10 also includes an RGB camera, a collision avoidance sensor, a speed sensor, and an acceleration sensor. The sensor data acquired by sensor 10 is supplied to information processing unit 100.

[0050] The drive unit 20 is configured, for example, as a motor that rotates the wheels of the moving body 1. By driving the drive unit 20 based on the control of the information processing unit 100, the moving body 1 can be moved.

[0051] The communication unit 30 is a wireless communication module that performs wireless communication with external devices. The communication unit 30 supplies information received via wireless communication to the information processing unit 100, and transmits information supplied from the information processing unit 100 via wireless communication.

[0052] Storage unit 40 is configured, for example, by using volatile memory such as dynamic random access memory (DRAM). Storage unit 40 stores various data obtained through calculations by information processing unit 100.

[0053] The information processing unit 100 is configured using a processor, such as a central processing unit (CPU) or a graphics processing unit (GPU). The information processing unit 100 controls each unit of the moving body 1. The configuration of the information processing unit 100 according to embodiments of the present disclosure will be described below.

[0054] <3. First Implementation Method (Configuration for Performing Multi-Layer Self-Position Estimation Processing)>

[0055] Figure 6 This is a block diagram illustrating an example of the functional configuration of an information processing unit 100 according to a first embodiment of the present disclosure.

[0056] Figure 6 The information processing unit 100 can perform the following processing by executing a program stored in a memory (not shown).

[0057] First, the information processing unit 100 generates first distance information and second distance information based on sensor data output by the ranging sensor 111 included in the sensor 10. The ranging sensor 111 is configured using a 3D sensor (e.g., a 3D LiDAR or a 3D depth sensor) capable of acquiring 3D distance information in the environment. That is, the information processing unit 100 can extract the first distance information (3D distance information) of the first layer and the second distance information of the second layer from the sensor data (3D distance information) output by a single ranging sensor 111.

[0058] Next, as part of the self-position estimation process for the moving body 1, the information processing unit 100 performs a first self-position estimation process based on the first distance information and a second self-position estimation process based on the second distance information. That is, the information processing unit 100 performs self-position estimation processing in multiple layers. In the following description, self-position estimation processing in multiple layers is also referred to as multi-layer self-position estimation processing. Note that in the following description, the multiple layers are assumed to be layers that are different from each other in the height direction relative to a horizontal plane, but the multiple layers may be layers that are different from each other in the lateral direction (left-right direction) or layers that are different from each other in the tilt direction.

[0059] Then, the information processing unit 100 outputs self-position information indicating the self-position of the moving body 1 based on the scores of the respective processing results of the first self-position estimation process and the second self-position estimation process.

[0060] The sensor data output by the ranging sensor 111 is segmented into layers based on environmental information related to the environment in which the mobile body 1 moves. The environment in which the mobile body 1 moves is a constantly changing 3D structure, such as a parking lot, factory, or concert stage, in addition to the warehouse mentioned above. The environmental information may be, for example, height information input by the user through the user interface (UI) presentation unit 112 based on the degree of change of the environment in which the mobile body 1 moves, or it may be height information predetermined for each environment in which the mobile body 1 moves. In the case of height information input by the user, the user can specify the layer (height) with less environmental change. In this way, the sensor data output by the ranging sensor 111 is segmented at the height corresponding to the environment in which the mobile body 1 moves to obtain each layer.

[0061] In addition to the user interface (UI) for inputting the aforementioned environmental information, the UI presentation unit 112 can also present various other UIs. For example, the UI presentation unit 112 can display the environmental map of each layer used in the multi-layer self-position estimation process performed in the information processing unit 100.

[0062] By executing a program stored in a memory (not shown), the information processing unit 100 implements functional blocks including a sensor data segmentation unit 120, a multi-layer self-position estimation unit 130, and a self-position integration unit 140.

[0063] The sensor data segmentation unit 120 functions as a distance information generation unit. This unit segments the sensor data output by the ranging sensor 111 into multiple layers and generates first and second distance information based on each segmented sensor data. Specifically, the sensor data segmentation unit 120 generates first and second distance information based on environmental information (height information) input by the user via the UI presentation unit 112 or pre-defined, by segmenting the sensor data output by the ranging sensor 111 into layers in the height direction relative to the travel surface of the moving body 1.

[0064] The first distance information and the second distance information can each be 2D point cloud information of an object (environment) at a predetermined height relative to the driving surface of the moving body 1, or they can be 3D point cloud information of an object (environment) within a predetermined height range.

[0065] Here, it is assumed that the point cloud information of the first layer and the point cloud information of the second layer (first distance information and second distance information) are extracted from the sensor data output by the ranging sensor 111, but it is also possible to extract point cloud information of three or more layers.

[0066] The multi-layer self-position estimation unit 130 performs self-position estimation processing in multiple layers simultaneously in parallel. In the multi-layer self-position estimation unit 130, as shown in the figure, the reference numerals of the functional blocks that implement self-position estimation processing in the corresponding layers are each assigned a branch number for each layer.

[0067] In other words, the multi-layer self-position estimation unit 130 includes a self-position estimation unit 131-1, a score determination unit 132-1, a map update unit 133-1, and an environment map preservation unit 134-1 for the first layer, and also includes a self-position estimation unit 131-2, a score determination unit 132-2, a map update unit 133-2, and an environment map preservation unit 134-2 for the second layer. In the following description, the functional blocks of each layer will be given without distinguishing between them, omitting the branch numbers of each layer.

[0068] The self-position estimation unit 131 performs self-position estimation processing based on layer distance information by using the environment map maintained in the environment map holding unit 134, and supplies the processing result to the score determination unit 132. The environment map holding unit 134 holds a pre-prepared environment map of the layer.

[0069] The score determination unit 132 calculates a self-position estimation score (hereinafter also simply referred to as the score), which indicates the reliability of the processing results from the self-position estimation unit 131. The reliability of the processing results can be calculated based on the covariance of the distance information, or it can be calculated based on the matching results between the distance information and the environment map. The score determination unit 132 supplies the processing results from the self-position estimation unit 131 and the calculated score to the self-position integration unit 140. The self-position integration unit 140 functions as a self-position information output unit, which outputs self-position information indicating the self-position of the moving body 1 based on the scores of the processing results of the self-position determination processing of each layer from the score determination unit 132 of each layer.

[0070] Furthermore, the score determination unit 132 determines whether the score of the processing result is less than a preset threshold and supplies the determination result to the map update unit 133. The map update unit 133 recreates or updates the environmental map maintained in the environmental map maintenance unit 134 based on the determination result from the map update unit 133.

[0071] For example, such as Figure 7 As shown, assume that the multi-layer self-position estimation unit 130 is performing multi-layer self-position estimation processing for three layers.

[0072] Self-position estimation unit 131-1 performs self-position estimation processing based on distance information from a pre-prepared first-layer environment map MP1. Self-position estimation unit 131-2 performs self-position estimation processing based on distance information from a pre-prepared second-layer environment map MP2. Self-position estimation unit 131-3 performs self-position estimation processing based on distance information from a pre-prepared third-layer environment map MP3.

[0073] Then, the score determination unit 132 determines whether the scores of the processing results of the self-position estimation processing of the corresponding layer by the self-position estimation units 131-1, 131-2, and 131-3 are less than a preset threshold.

[0074] exist Figure 7 In the example, the score of the processing result of the first layer is determined to be less than the threshold, and the environment map MP1 of that layer is updated or recreated. On the other hand, the scores of the processing results of the second layer and the third layer are determined to be greater than the threshold and are supplied to their own location integration unit 140.

[0075] As described above, based on the multi-layer self-position estimation process, even if environmental changes occur in one layer, the environmental map of the layer with environmental changes can be updated while the self-position estimation process is stably performed in another layer.

[0076] exist Figure 7 In the example, in the multi-layer self-position estimation process, it is assumed that the self-position estimation process of each layer is performed as a loosely coupled process, but it is also possible to perform the self-position estimation process of each layer as a tightly coupled process.

[0077] When the self-position estimation process of each layer is performed as a tightly coupled process, the positional relationships between the submaps that constitute the respective environment map can be used in the creation of the environment map of each layer.

[0078] For example, such as Figure 8 As shown, assume that the multi-layer self-position estimation unit 130 creates an environment map while performing multi-layer self-position estimation processing on two layers.

[0079] Self-position estimation unit 131-1 performs self-position estimation processing based on distance information of the first layer while creating sub-maps SM11 to SM14 for each region in the environment. Self-position estimation unit 131-2 performs self-position estimation processing based on distance information of the layer while creating sub-maps SM21 to SM24 for the second layer for each region in the environment.

[0080] exist Figure 8 In the example, a closed loop is formed by connecting the corresponding submaps SM11 to SM14 based on the common points between them in the first layer. This allows the creation of the environment map MP1 for the entire first layer. On the other hand, the submaps SM21 to SM24 in the second layer cannot form a closed loop because there are insufficient common points between them.

[0081] At this point, based on the positional relationships between submaps SM11 to SM14 that constitute environment map MP1, the positions between the corresponding submaps SM21 to SM24 are corrected. This creates a closed loop within submaps SM21 to SM24 and enables the creation of environment map MP2 for the entire environment in the second layer. Therefore, the consistency between environment maps in each layer can be increased.

[0082] Reference Figure 9 The flowchart will describe the process by Figure 6 The information processing unit 100 performs multi-layer self-position estimation processing. Figure 9 For example, the processing begins at a timed start when the mobile body 1, acting as a mobile robot, is activated in the environment.

[0083] In step S11, the sensor data segmentation unit 120 segments the sensor data output by the ranging sensor 111 into multiple layers, and supplies the distance information (point cloud information) of each segmented layer to the self-position estimation unit 131 of each layer.

[0084] In step S12, the self-position estimation unit 131 of each layer reads the environment map from the environment map holding unit 134 of each layer.

[0085] In step S13, each layer's self-position estimation unit 131 performs self-position estimation processing based on the distance information (point cloud information) of that layer by using the environment map read from the environment map holding unit 134.

[0086] In step S14, the score determination unit 132 of each layer determines whether there is a score less than a threshold in the score of the processing result of its own position estimation processing. If there is no score less than the threshold, that is, if all scores of the processing result are greater than the threshold, the processing proceeds to step S15.

[0087] In step S15, the self-position integration unit 140 extracts all processing results from the score determination unit 132 of each layer.

[0088] In step S16, the self-position integration unit 140 integrates the processing results of the self-position estimation processing of each layer based on scores. For example, the self-position integration unit 140 integrates the processing results (coordinate information) of the self-position estimation processing of the corresponding layer according to scores and uses the integrated result as its self-position information. In this case, the coordinate information can be integrated based on an extended Kalman filter using covariance, or it can be integrated based on a factor graph. Alternatively, the self-position integration unit 140 can use the processing result (coordinate information) with the highest score as its self-position information.

[0089] Then, in step S17, the self-position integration unit 140 outputs its own position information.

[0090] On the other hand, in step S14, if it is determined that there are scores less than the threshold, the process proceeds to step S18, and the score determination unit 132 of each layer determines whether all scores are less than the threshold. If it is determined that not all scores are less than the threshold, the process proceeds to step S19.

[0091] In step S19, the map update unit 133 of the layer corresponding to the score less than the threshold updates or recreates the environmental map held in the environmental map holding unit 134 of that layer.

[0092] In step S20, the self-position integration unit 140 extracts the processing results of scores greater than the threshold from the score determination unit 132 of each layer corresponding to scores greater than the threshold. Subsequently, in step S16, the processing results are integrated based on the scores, and in step S17, the self-position information is output.

[0093] Now, in step S18, if it is determined that all scores are less than the threshold, the process proceeds to step S21.

[0094] In step S21, the self-position integration unit 140 uses the processing result (coordinate information) of the largest score among all scores less than the threshold as its own position information.

[0095] In step S22, the map update unit 133 of other layers updates or recreates the environment map held in the environment map holding unit 134 of the layer.

[0096] Subsequently, in step S17, among all scores less than a threshold, the processing result with the highest score is output as its own position information. According to this arrangement, even if environmental changes occur in all layers, the self-position estimation process can continue in the layer with the least environmental change, thus avoiding unexpected stops such as the movement of the moving body 1.

[0097] Based on the above processing, since self-position estimation processing is performed in multiple layers, stable output of self-position information can be achieved while avoiding layers with large environmental changes, and thus the robustness of self-position estimation processing can be improved.

[0098] <4. Second Implementation Method (Configuration of Execution Layer Search Processing)>

[0099] Figure 10 This is a block diagram illustrating an example of the functional configuration of an information processing unit 100 according to a second embodiment of the present disclosure.

[0100] exist Figure 10 In the information processing unit 100, for the information processing unit 100, the information processing unit 100 is used to Figure 6 The information processing unit 100 includes functional blocks that are similar in function, which are represented by the same reference numerals, and the description of the functional block is substantially omitted. That is to say, Figure 10 Information processing unit 100 and Figure 6 The information processing unit 100 is different in that it newly includes a layer search self-position estimation unit 210 and a score comparison unit 220.

[0101] However, Figure 10The sensor data segmentation unit 120 in the information processing unit 100 segments the sensor data output by the ranging sensor 111 into multiple layers and generates third distance information for a layer different from the first and second distance information. That is, in addition to the point cloud information of the first and second layers, third-layer point cloud information is extracted from the sensor data output by the ranging sensor 111. Hereinafter, the third layer will be referred to as the search layer. The search layer can be set according to the aforementioned environmental information, or it can be changed sequentially or randomly according to the environment in which the moving body 1 moves.

[0102] The layer search self-position estimation unit 210 performs self-position estimation processing based on third distance information (point cloud information of the search layer) using the environment map maintained in the environment map preservation unit (not shown), and calculates a score of the processing result. The calculated score is supplied to the score comparison unit 220.

[0103] The score comparison unit 220 acquires the scores of the processing results of the self-position estimation processing of each layer in the multi-layer self-position estimation unit 130, and compares the scores with the scores of the processing results of the self-position estimation processing of the search layer from the score comparison unit 220. The score comparison results are supplied to the sensor data segmentation unit 120.

[0104] The sensor data segmentation unit 120 changes the layer to which the sensor data output by the ranging sensor 111 is segmented based on the score comparison result from the score comparison unit 220. In other words, the sensor data segmentation unit 120 changes the layer to which the sensor data output by the ranging sensor 111 is segmented based on the score of the processing result of the self-position estimation processing based on the third distance information (point cloud information of the search layer).

[0105] In other words, the self-position estimation process performed by the layer search self-position estimation unit 210 can be described as a layer search self-position estimation process for searching layers with specific environmental changes (or layers without environmental changes).

[0106] Reference Figure 11 The flowchart will describe the process by Figure 10 The information processing unit 100 performs layer search processing. Figure 11 Processing and reference Figure 9 The flowchart describes how multi-layered self-position estimation processes are performed in parallel.

[0107] In step S21, the layer search self-position estimation unit 210 reads the environment map from the environment map holding unit (not shown), performs layer search self-position estimation processing based on the distance information (point cloud information) of the search layer, and calculates the score of the processing result.

[0108] In step S22, the score comparison unit 220 determines whether the score (reliability) of the processing result of the layer search for self-position estimation is higher than the score of the processing result of the self-position estimation of each layer in the multi-layer self-position estimation unit 130. If it is determined that the score of the processing result of the layer search for self-position estimation is not higher than the score of the processing result of the self-position estimation of each layer, the process returns to step S21, and the layer search for self-position estimation is repeated.

[0109] On the other hand, if the score of the processing result of the layer search for its own position estimation is higher than the score of the processing result of the layer's own position estimation, the process proceeds to step S23, and the sensor data segmentation unit 120 changes the layer to which the sensor data output by the ranging sensor 111 is segmented.

[0110] For example, if the score (reliability) of the layer searching for its own position estimation processing is higher than the score (reliability) of the first layer's own position estimation processing, the first layer is changed to a search layer as the layer to which the sensor data is segmented. Furthermore, if the score of the layer searching for its own position estimation processing is higher than the score of the second layer's own position estimation processing, the second layer is changed to a search layer as the layer to which the sensor data is segmented. Moreover, if the score of the layer searching for its own position estimation processing is higher than the scores of the self-position estimation processing of all layers, the layers to which the sensor data is segmented are changed to include at least the search layer.

[0111] Based on the above processing, in the multi-layer self-position estimation process, it is possible to autonomously search for layers with large environmental changes and change them to layers with small environmental changes, thereby further improving the robustness to environmental changes.

[0112] <5. Third Implementation Method (Including Configuration of Multiple Distance Sensors)>

[0113] Figure 12 This is a block diagram illustrating an example of the functional configuration of an information processing unit 100 according to a third embodiment of the present disclosure.

[0114] exist Figure 12 In the information processing unit 100, for the information processing unit 100, the information processing unit 100 is used to Figure 6 The information processing unit 100 includes functional blocks that are similar in function, which are represented by the same reference numerals, and the description of the functional block is substantially omitted. That is to say, Figure 12 Information processing unit 100 and Figure 6 The difference of the information processing unit 100 is that it includes sensor data acquisition units 310-1 and 310-2, but does not include sensor data segmentation unit 120.

[0115] Figure 12 The information processing unit 100 generates first distance information and second distance information based on the sensor data output by the ranging sensors 111-1 and 111-2 installed at different height positions in the moving body 1. The ranging sensors 111-1 and 111-2 can each be configured by using a 3D sensor capable of acquiring 3D distance information in the environment, or they can each be configured by using a 2D sensor (e.g., a 2D LiDAR or a 2D depth sensor) capable of acquiring 2D distance information in the environment.

[0116] In other words, the sensor data acquisition unit 310-1 acquires the sensor data output by the ranging sensor 111-1 and generates first distance information. When the ranging sensor 111-1 is configured using a 3D sensor, the sensor data acquisition unit 310-1 extracts, for example, first distance information based on environmental information for a first layer from the sensor data (3D distance information) output by the ranging sensor 111-1 and supplies the first distance information to the multi-layer self-position estimation unit 130. When the ranging sensor 111-1 is configured using a 2D sensor, the sensor data acquisition unit 310-1 acquires the sensor data (2D distance information) output by the ranging sensor 111-1 and supplies the sensor data to the multi-layer self-position estimation unit 130.

[0117] Similarly, the sensor data acquisition unit 310-2 acquires the sensor data output by the ranging sensor 111-2 and generates second distance information. When the ranging sensor 111-2 is configured using a 3D sensor, the sensor data acquisition unit 310-2 extracts, for example, second distance information based on environmental information from the sensor data (3D distance information) output by the ranging sensor 111-2, and supplies the second distance information to the multi-layer self-position estimation unit 130. When the ranging sensor 111-2 is configured using a 2D sensor, the sensor data acquisition unit 310-2 acquires the sensor data (2D distance information) output by the ranging sensor 111-2 and supplies the sensor data to the multi-layer self-position estimation unit 130.

[0118] Furthermore, in such a configuration, since self-position estimation processing is performed in multiple layers, stable output of self-position information can be achieved while avoiding layers with large environmental changes, and thus the robustness of self-position estimation processing can be improved.

[0119] <6. Example of computer hardware configuration>

[0120] The series of processes described above can be executed by hardware or by software. In the case where the series of processes are executed by software, the program constituting the software is installed from a program recording medium onto a computer integrated into dedicated hardware, or onto a general-purpose personal computer, etc.

[0121] Figure 13 This is a block diagram illustrating an example of the hardware configuration of a computer that performs the above series of processes via a program. An information processing unit 100, which may be included in the mobile body 1, may, for example, be composed of a... Figure 13 The configuration shown is similar to that of computer 400.

[0122] The central processing unit (CPU) 401, read-only memory (ROM) 402 and random access memory (RAM) 403 are interconnected via bus 404.

[0123] Furthermore, the input / output interface 405 is connected to the bus 404. Input units 406, including a keyboard and mouse, and output units 407, including a display and speakers, are connected to the input / output interface 405. Additionally, a storage unit 408, a communication unit 409, and a driver 410 are connected to the input / output interface 405. The storage unit 408 includes a hard disk, non-volatile memory, etc., the communication unit 409 includes a network interface, etc., and the driver 410 drives the removable medium 411.

[0124] In the computer 400 configured as described above, for example, the CPU 401 loads the program stored in the storage unit 408 into the RAM 403 via the input / output interface 405 and the bus 404 and executes the program to perform the series of processes described above.

[0125] The program executed by the CPU 401 is provided, for example, recorded on removable medium 411 or via wired or wireless transmission medium such as a local area network, the Internet or digital broadcasting, and is installed in storage unit 408.

[0126] The program executed by the computer 400 may be a program that is processed sequentially in the order described in this specification, or it may be a program that is processed in parallel or at necessary time intervals (e.g., at the time of execution call).

[0127] Note that in this specification, a system refers to a collection of multiple components (devices, modules (parts), etc.), regardless of whether all components are housed in the same housing. Therefore, both multiple devices housed in separate housings and connected via a network, and a device in which multiple modules are housed in a single housing, are systems.

[0128] The effects described in this specification are illustrative only and not limiting, and other effects may also exist.

[0129] The implementation of this disclosure is not limited to the above-described implementation, and various modifications may be made without departing from the spirit of this disclosure.

[0130] For example, implementations of this disclosure may employ a cloud computing configuration, in which a function is shared and collaboratively processed by multiple devices via a network.

[0131] Furthermore, each step described in the flowchart above can be performed by a single device, or by multiple devices in a shared manner.

[0132] Furthermore, in cases where a step includes multiple processes, the multiple processes included in a step can be performed by a single device, or can be performed by multiple devices in a shared manner.

[0133] The effects described in this specification are illustrative only and not limiting, and other effects may also exist.

[0134] Furthermore, the technology based on this disclosure can be configured as follows. (1)

[0136] An information processing method, comprising:

[0137] First distance information and second distance information are generated based on sensor data output by a ranging sensor installed on the moving body;

[0138] Perform a first self-position estimation process based on a first distance information and a second self-position estimation process based on a second distance information, as a self-position estimation process for the moving body; and

[0139] Based on the processing results of the first and second self-position estimation processes, the self-position information indicating the self-position of the moving body is output. (2)

[0141] According to the information processing method in (1), where

[0142] First distance information and second distance information are generated based on the individual sensor data obtained by segmenting sensor data into multiple layers. (3)

[0144] According to the information processing method in (2), where

[0145] First distance information and second distance information are generated based on individual sensor data obtained by segmenting sensor data into layers in the height direction relative to the driving surface of the moving body. (4)

[0147] According to the information processing method in (3), where

[0148] The first distance information and the second distance information are respectively two-dimensional point cloud information of an object at a predetermined height relative to the driving surface or three-dimensional point cloud information of an object within a predetermined height range. (5)

[0150] According to the information processing method in (4),

[0151] First distance information and second distance information are extracted from sensor data output by a single ranging sensor. (6)

[0153] According to the information processing method in (4),

[0154] First distance information and second distance information are generated based on sensor data output from distance measuring sensors installed at different heights. (7)

[0156] According to any one of the information processing methods in (1) to (6), where

[0157] The self-position information is output based on the scores of the first and second self-position estimation processes, respectively.

[0158] The score indicates the reliability of the processing results. (8)

[0160] According to the information processing method in (7), where

[0161] Reliability is calculated based on the covariance of distance information or the matching results. (9)

[0163] According to the information processing method in (7) or (8), where

[0164] The processing result with the highest score is output as its own position information. (10)

[0166] According to the information processing method in (7) or (8), where

[0167] The processing result based on the score integration will be output as its own location information. (11)

[0169] According to any one of the information processing methods in (7) to (10), where

[0170] A first self-position estimation process is performed using a first map as the environment map, and a second self-position estimation process is performed using a second map different from the first map.

[0171] Recreate or update the environment map for the following self-location estimation process where the score of the processing result is less than a threshold. (12)

[0173] According to the information processing method in (11), where

[0174] If the scores of all processed results are less than the threshold, the processed result with the highest score will be output as its own location information. (13)

[0176] According to the information processing method of (11) or (12), where,

[0177] Based on the positional relationships within the submaps that constitute one of the first and second maps, the positions between the other submaps that constitute the first and second maps are corrected. (14)

[0179] According to any one of the information processing methods in (2) to (13), where

[0180] Layers for segmenting sensor data are defined based on environmental information related to the environment in which the moving body moves. (15)

[0182] According to the information processing method in (14), where

[0183] Environmental information is height information input by the user based on the degree to which the environment is prone to change. (16)

[0185] According to the information processing method in (14), where

[0186] Environmental information is altitude information predetermined for each environment. (17)

[0188] According to any one of the information processing methods in (2) to (16), where

[0189] Based on the processing result of the third self-position estimation process, the layer used to segment the sensor data is changed. The third self-position estimation process is based on third distance information from a layer that is different from the first distance information and the second distance information. (18)

[0191] According to the information processing method in (17), where

[0192] If the reliability of the processing result of the third self-position estimation process is higher than the reliability of the processing result of at least one of the first self-position estimation process or the second self-position estimation process, the layer used to segment the sensor data is changed. (19)

[0194] An information processing apparatus, comprising:

[0195] The distance information generation unit generates first distance information and second distance information based on sensor data output by a ranging sensor mounted on the moving body.

[0196] A self-position estimation unit performs a first self-position estimation process based on a first distance information and a second self-position estimation process based on a second distance information, as a self-position estimation process for a moving body; and

[0197] The self-position information output unit outputs self-position information indicating the self-position of the moving body based on the respective processing results of the first self-position estimation process and the second self-position estimation process. (20)

[0199] A program that causes a computer to perform the following processes:

[0200] First distance information and second distance information are generated based on sensor data output by a ranging sensor installed on the moving body;

[0201] Perform a first self-position estimation process based on a first distance information and a second self-position estimation process based on a second distance information, as a self-position estimation process for the moving body; and

[0202] Based on the processing results of the first and second self-position estimation processes, the self-position information indicating the self-position of the moving body is output.

[0203] List of reference numerals

[0204] 1. Moving body; 10. Sensors; 20. Drive unit; 30. Communication unit; 40. Storage unit; 100. Information processing unit; 111. Ranging sensor; 112. UI rendering unit; 120. Sensor data segmentation unit; 130. Multi-layer self-position estimation unit; 131. Self-position estimation unit; 132. Score determination unit; 133. Map update unit; 134. Environmental map preservation unit; 140. Self-position integration unit; 210. Layer search self-position estimation unit; 220. Score comparison unit.

Claims

1. An information processing method, comprising: First distance information and second distance information are generated based on sensor data output by a ranging sensor installed on the moving body; Perform a first self-position estimation process based on the first distance information and a second self-position estimation process based on the second distance information as the self-position estimation process of the moving body; as well as Based on the processing results of the first self-position estimation process and the second self-position estimation process, self-position information indicating the self-position of the moving body is output.

2. The information processing method according to claim 1, wherein, The first distance information and the second distance information are generated based on the individual sensor data obtained by segmenting the sensor data into multiple layers.

3. The information processing method according to claim 2, wherein, The first distance information and the second distance information are generated based on the individual sensor data obtained by segmenting the sensor data into layers in the height direction relative to the driving surface of the moving body.

4. The information processing method according to claim 3, wherein, The first distance information and the second distance information are respectively two-dimensional point cloud information of an object at a predetermined height relative to the driving surface or three-dimensional point cloud information of the object within a predetermined height range.

5. The information processing method according to claim 4, wherein, The first distance information and the second distance information are extracted from the sensor data output by the single ranging sensor.

6. The information processing method according to claim 4, wherein, The first distance information and the second distance information are generated based on sensor data output by the ranging sensors installed at different heights.

7. The information processing method according to claim 1, wherein, The self-position information is output based on the scores of the respective processing results of the first and second self-position estimation processes, and... The score indicates the reliability of the processing result.

8. The information processing method according to claim 7, wherein, The reliability is calculated based on the covariance of distance information or the matching result.

9. The information processing method according to claim 7, wherein, The processing result with the highest score is output as its own position information.

10. The information processing method according to claim 7, wherein, The processing result, which is integrated based on the score, will be output as the user's own location information.

11. The information processing method according to claim 7, wherein, The first self-position estimation process is performed by using a first map as the environment map, and the second self-position estimation process is performed by using a second map different from the first map. Recreate or update the environment map for the following self-location estimation process, in which the score of the processing result is less than a threshold.

12. The information processing method according to claim 11, wherein, If the scores of all the processing results are less than the threshold, the processing result with the highest score is output as the self-position information.

13. The information processing method according to claim 11, wherein, Based on the positional relationship between the sub-maps that constitute one of the first map and the second map, the position between the sub-maps that constitute the other of the first map and the second map is corrected.

14. The information processing method according to claim 2, wherein, Layers for segmenting the sensor data are set based on environmental information related to the environment in which the mobile body moves.

15. The information processing method according to claim 14, wherein, The environmental information is height information input by the user based on the degree to which the environment is prone to change.

16. The information processing method according to claim 14, wherein, The environmental information is pre-determined altitude information for each environment.

17. The information processing method according to claim 7, wherein, Based on the processing result of the third self-position estimation process, the layer used to segment the sensor data is changed. The third self-position estimation process is based on third distance information of the layer that is different from the first distance information and the second distance information.

18. The information processing method according to claim 17, wherein, If the reliability of the processing result of the third self-position estimation process is higher than the reliability of the processing result of at least one of the first self-position estimation process or the second self-position estimation process, the layer used to segment the sensor data is changed.

19. An information processing apparatus, comprising: A distance information generation unit generates first distance information and second distance information based on sensor data output by a ranging sensor installed on a moving body. A self-position estimation unit performs a first self-position estimation process based on the first distance information and a second self-position estimation process based on the second distance information as the self-position estimation process of the moving body. as well as The self-position information output unit outputs self-position information indicating the self-position of the moving body based on the respective processing results of the first self-position estimation process and the second self-position estimation process.

20. A program that causes a computer to perform the following processes: First distance information and second distance information are generated based on sensor data output by a ranging sensor installed on the moving body; Perform a first self-position estimation process based on the first distance information and a second self-position estimation process based on the second distance information as the self-position estimation process of the moving body; as well as Based on the processing results of the first self-position estimation process and the second self-position estimation process, self-position information indicating the self-position of the moving body is output.

Images