Moving body route analysis method, moving body route analysis device, and program

Abstract

Provided are a device and a method for generating a route via which a moving body can safely reach a target position without coming into contact with an obstacle. The present invention calculates error in the estimated localization position of a moving body, obtained by including prescribed error in the estimated localization position calculated through a localization position estimation process. Furthermore, the present invention: calculates the estimated self-localization position error reflecting a region where the environment is difficult to recognize, in which the rate of increase in the estimated self-localization position error occurring in a region where the environment is difficult to recognize is set higher than the rate of increase in the estimated self-localization position error in regions other than the region where the environment is difficult to recognize; analyzes whether there is any overlap between the calculated estimated self-localization position error reflecting a region where the environment is difficult to recognize and an obstacle; and determines that a route where there is no overlap is a route over which the moving body can move without coming into contact with an obstacle.

Description

Mobile object route analysis method, mobile object route analysis device, and program

[0001] The present disclosure relates to a mobile object path analysis method, a mobile object path analysis device, and a program, and more particularly to a mobile object path analysis method, a mobile object path analysis device, and a program that analyze a path that enables a mobile object such as a robot to safely reach a destination even if the path includes an area where self-position estimation is difficult.

[0002] In recent years, the use of mobile objects such as robots and drones has become widespread. There are two ways to control mobile objects such as robots and drones: one is controlled by a human operating a controller, and the other is controlled autonomously by the mobile object itself without using an external controller.

[0003] An autonomously controlled mobile body sequentially executes a self-position estimation process along its travel route. That is, the autonomously controlled mobile body sequentially executes a self-position estimation process to estimate the current position of the mobile body, such as a robot, on a map, and controls itself to move while checking its current position so as not to deviate from a predetermined route.

[0004] An example of prior art that discloses a driving control configuration that utilizes such self-position estimation processing is Patent Document 1 (JP 2021-196487 A). This document discloses a configuration that generates a map that defines a robot's possible travel route using 3D Building Information Modeling (BIM) data and the robot's self-position estimation processing. Note that 3D BIM data is 3D map data that includes the internal structure of a building and attribute information of the structure.

[0005] Patent Document 1 discloses a configuration in which a new environmental map is generated from BIM data that defines the routes that the robot can travel, depending on the robot's characteristics such as its size and range of motion, and the robot is moved according to the routes set on the generated environmental map.

[0006] One method of self-position estimation is, for example, SLAM (Simultaneous Localization and Mapping) processing. SLAM processing analyzes images captured by a camera attached to an autonomous robot, analyzes the movement of the robot itself from the movement of a subject included in the captured images, and analyzes the direction and distance of movement of the robot to estimate its current self-position.

[0007] However, this SLAM processing involves analyzing the movement of objects within multiple image frames captured by a camera, and then analyzing the amount and direction of movement of the vehicle's relative position based on the results of this analysis, which has the problem of gradually accumulating errors.

[0008] In particular, if the area in which the robot travels is surrounded by flat walls, corridors, or other features with few features, the images captured by the camera will remain almost unchanged even if the robot's position changes as it travels. In such cases, it becomes difficult to estimate the robot's position from the images, and the robot may not be able to travel along the planned route.

[0009] Japanese Patent Application Laid-Open No. 2021-196487

[0010] The present disclosure has been made in consideration of, for example, the above-mentioned problems, and aims to provide a mobile object path analysis method, a mobile object path analysis device, and a program for analyzing a moving path that enables a mobile object such as a robot to safely reach its destination even if the path includes an area where self-position estimation is difficult.

[0011] A first aspect of the present disclosure resides in a mobile body path analysis method in which a data processing unit of a mobile body path analysis device calculates an estimated self-position error that includes a predetermined range of error in the estimated self-position calculated in the mobile body's self-position estimation process, analyzes whether the calculated area of ​​estimated self-position error overlaps with an obstacle, and determines that a route without overlap is a route along which the mobile body can travel without coming into contact with or colliding with an obstacle.

[0012] Furthermore, a second aspect of the present disclosure resides in a mobile body path analysis device having a data processing unit that calculates an estimated self-position error that includes a predetermined range of error in the estimated self-position calculated in the mobile body's self-position estimation process, analyzes whether the calculated area of ​​estimated self-position error overlaps with an obstacle, and determines that a route without overlap is a route that the mobile body can travel without coming into contact with or colliding with an obstacle.

[0013] Furthermore, a third aspect of the present disclosure is a program that causes a moving body path analysis device to execute a moving body path analysis process, the program causing a data processing unit of the moving body path analysis device to execute the following processes: a process of calculating an estimated self-position error that includes a predetermined range of error in the estimated self-position calculated in the moving body's self-position estimation process; a process of analyzing whether or not the calculated area of ​​estimated self-position error overlaps with an obstacle; and a process of determining that a route without overlap is a route that the moving body can travel on without coming into contact with or colliding with an obstacle.

[0014] The program of the present disclosure is, for example, a program that can be provided in a computer-readable format via a storage medium or a communication medium to an information processing device or a computer system capable of executing various program codes. By providing such a program in a computer-readable format, processing according to the program is realized on the information processing device or the computer system.

[0015] Further objects, features, and advantages of the present disclosure will become apparent from the following detailed description of the embodiments of the present disclosure and the accompanying drawings. Note that in this specification, a system refers to a logical collective configuration of multiple devices, and is not limited to devices that are located within the same housing.

[0016] According to one embodiment of the present disclosure, an apparatus and method for generating a path that allows a mobile body to safely reach a target position without coming into contact with an obstacle are realized. Specifically, for example, an estimated self-position error is calculated by adding a predetermined error to the estimated self-position calculated by the mobile body's self-position estimation process. Furthermore, an estimated self-position error that reflects a difficult-to-recognize area is calculated, in which the increase rate of the estimated self-position error that occurs in an environment difficult to recognize area is set higher than the increase rate of the estimated self-position error in areas other than the environment difficult to recognize area. The presence or absence of overlap between the calculated estimated self-position error that reflects a difficult-to-recognize area and an obstacle is analyzed, and a route without overlap is determined to be a route that the mobile body can travel without coming into contact with an obstacle. This configuration realizes an apparatus and method for generating a path that allows a mobile body to safely reach a target position without coming into contact with an obstacle. Note that the effects described in this specification are merely examples and are not limited thereto, and additional effects may also be present.

[0017] 1 is a diagram illustrating an overview of the processing of the present disclosure. FIG. 1 is a diagram illustrating an overview of the processing of the present disclosure. FIG. 2 is a diagram illustrating an example configuration of a robot that is an example of a moving body path analysis device of the present disclosure. FIG. 3 is a diagram illustrating an example configuration of a moving body path analysis device of the present disclosure. FIG. 4 is a diagram illustrating an example configuration of a moving body path analysis device of the present disclosure. FIG. 5 is a diagram illustrating an overall sequence executed by a data processing unit of a moving body path analysis device. FIG. 6 is a diagram illustrating a flowchart illustrating a processing sequence executed by a data processing unit of a moving body path analysis device of the present disclosure. FIG. 7 is a diagram illustrating definitions of "areas difficult to recognize the environment" and "landmarks" according to sensors used by a moving body such as a robot. FIG. 8 is a diagram illustrating an example of a "recognition-convenient heat map" that makes it possible to distinguish "areas difficult to recognize the environment" and "landmarks" according to the type of sensor (camera, LiDAR, GPS, or GNSS) used by a moving body. FIG. 9 is a diagram illustrating an example of a "recognition-convenient heat map" that makes it possible to distinguish "areas difficult to recognize the environment" and "landmarks" according to the type of sensor (camera, LiDAR, GPS, or GNSS) used by a moving body. 1 is a diagram illustrating an example of a "recognition-convenient heat map" that makes it possible to identify "difficult-to-recognize areas" and "landmarks" according to the type of sensor (camera, LiDAR, GPS, or GNSS) used by a mobile body. FIG. 2 is a diagram illustrating two examples of integration processing of "difficult-to-recognize areas" that can be performed when integrating "recognition-convenient heat maps" corresponding to different sensors. FIG. 3 is a diagram illustrating an example of probability distribution data indicating the self-position of a mobile body (robot) estimated in a self-position estimation processing by SLAM. FIG. 4 is a diagram illustrating an example of probability distribution data indicating the self-position of a mobile body (robot) estimated in a self-position estimation processing. FIG. 5 is a diagram illustrating an example of probability distribution data indicating the self-position of a mobile body (robot) estimated in a self-position estimation processing. FIG. 6 is a diagram illustrating a specific example of differences in the rate of increase of the "estimated self-position error 123" according to the traveling environment of a robot. FIG. 7 is a diagram illustrating an example (enlarged example) of changes in the self-position estimation error when a robot travels a traveling route that includes an "environment recognition-convenient area". FIG. 8 is a diagram illustrating a specific example of calculation processing of the "difficult-to-recognize area-reflecting estimated self-position error". ｄｉｓｔ, Σ ｔｉｍｅ ) of a plurality of sensors. ｄｉｓｔ , Σ ｔｉｍｅ ) of the plurality of sensors. ｄｉｓｔ , Σ ｔｉｍｅ ) is a diagram illustrating a result of an integration process. A diagram illustrating a specific example of an optimal route generation process executed by a data processing unit of a mobile object route analysis device of the present disclosure. A diagram illustrating a flowchart illustrating a detailed sequence of an optimal route generation process executed by a data processing unit of a mobile object route analysis device of the present disclosure. A diagram illustrating a flowchart illustrating a detailed sequence of an optimal route generation process executed by a data processing unit of a mobile object route analysis device of the present disclosure. A diagram illustrating a specific example of an analysis process using an "estimated self-position error reflecting a difficult-to-recognize area". A diagram illustrating a specific example of an analysis process using an "estimated self-position error reflecting a difficult-to-recognize area". A diagram illustrating a specific example of an analysis process using an "estimated self-position error reflecting a difficult-to-recognize area". A diagram illustrating a specific example of a generation process of a new route in which a landmark is set as a via point (waypoint). A diagram illustrating an example of an analysis process of an "estimated self-position error" or a "estimated self-position error reflecting a difficult-to-recognize area" when a robot moves along a new route. A diagram illustrating an example of an addition process of a follow route. A diagram illustrating an example of display data displayed on a display unit. A diagram illustrating an example of display data displayed on a display unit. A diagram illustrating an example of a hardware configuration of a mobile object route analysis device of the present disclosure.

[0018] The details of the moving object route analysis method, moving object route analysis device, and program of the present disclosure will be described below with reference to the drawings. The description will be made in accordance with the following items: 1. Overview of the processing of the present disclosure 2. Example configuration of the moving object route analysis device of the present disclosure 3. Overall sequence of moving object control processing executed by the moving object route analysis device of the present disclosure 4. 4. Details of each process executed by the data processing unit of the mobile object route analysis device according to the present disclosure 4-(1) Recognition heat map generation process (step S11 in FIG. 6, step S102 in FIG. 7) 4-(2) Self-position estimation process (step S12 in FIG. 6, step S104 in FIG. 7) 4-(3) Difficult-to-recognize area reflected estimated self-position error calculation process (step S13 in FIG. 6, step S105 in FIG. 7) 4-(4) Optimal route generation process (step S14 in FIG. 6, step S106 in FIG. 7) 4-(5) Display information generation process (step S16 in FIG. 6, step S108 in FIG. 7) 5. Hardware configuration example of the mobile object route analysis device according to the present disclosure 6. Summary of the configuration of the present disclosure

[0019] [1. Overview of the Processing of the Present Disclosure] First, an overview of the processing of the present disclosure will be described.

[0020] An overview of the processing of the present disclosure will be described with reference to Figure 1 and subsequent figures. In the following embodiments, a mobile robot will be described as an example of a mobile object. However, the mobile object of the present disclosure is not limited to a mobile robot, and may include various mobile objects such as a flying mobile object such as a drone, and an autonomously driven vehicle that travels autonomously.

[0021] As explained above, an autonomously controlled moving body sequentially executes a self-position estimation process along its moving route. That is, a moving body such as a robot sequentially executes a self-position estimation process to estimate the robot's current position on a map, confirms its current position, and moves while controlling itself so as not to deviate from a predetermined planned route.

[0022] As described above, one method of self-position estimation is, for example, SLAM (Simultaneous Localization and Mapping). SLAM is a process that analyzes images captured by a camera attached to, for example, an autonomous mobile robot, analyzes the movement of the robot itself from the movement of the subject included in the captured images, and analyzes the direction and distance of movement of the robot to estimate its current self-position.

[0023] However, this SLAM processing involves analyzing the movement of objects within multiple image frames captured by a camera, and then analyzing the amount and direction of movement of the vehicle's relative position based on the results of this analysis, which has the problem of gradually accumulating errors.

[0024] In particular, if the area in which the robot travels is surrounded by flat walls, corridors, etc. with few features, the images captured by the camera will remain almost unchanged even if the robot's position changes as it travels. In such cases, it becomes difficult to estimate the robot's position from the images, and it may become impossible for the robot to travel along the planned route.

[0025] 1 is a diagram showing an example of a robot 100 that moves autonomously under autonomous control traveling from a start point (S) to a target position (G). The robot 100 travels by referring to a map, with the shortest route 20 from the start point (S) to the target position (G) as the planned route.

[0026] The robot 100 runs along the shortest route 20 while executing the self-position estimation process using the above-mentioned SLAM processing. That is, the robot 100 analyzes the images captured by the camera attached to the robot 100, analyzes the movement of the robot 100 itself from the movement of the subject included in the captured images, analyzes the direction and distance of movement of the robot, estimates its current self-position, and runs along the planned shortest route 20 so that its self-position does not deviate.

[0027] However, in the example shown in Figure 1, the shortest route 20 includes an environment recognition difficult area 21. The environment recognition difficult area 21 is, for example, a space surrounded by flat walls, floors, ceilings, etc. without any pattern, and is an area where the surrounding scenery changes little even when the robot 100 moves. In other words, it is an area where the image captured by the camera changes little even when the robot 100 moves and changes its position. In such a case, it becomes difficult to estimate the robot's own position from the image, and there is a possibility that the robot will not be able to move along the planned shortest route 20.

[0028] As a result, the robot 100 may move along a path 22 indicated by a dotted line in FIG. 1, and may collide with a wall.

[0029] The process of the present disclosure solves this problem by enabling a mobile body such as a robot to reach a destination even if the route includes an area where self-position estimation is difficult.

[0030] An overview of the processing of the present disclosure will be described with reference to Fig. 2. Similar to Fig. 1, Fig. 2 is a diagram showing an example in which the robot 100, which moves under autonomous control, travels from a start point (S) to a target position (G).

[0031] The robot 100 travels from a start point (S) to a target position (G). The robot 100 travels while executing the self-position estimation process using the SLAM process described above. That is, the robot 100 analyzes images captured by a camera attached to the robot 100, analyzes the movement of the robot 100 itself from the movement of the subject included in the captured images, and analyzes the direction and distance of movement of the robot to estimate its current self-position.

[0032] The shortest route 20 shown in Fig. 2 is the shortest route from the start point (S) to the target position (G), similar to the shortest route 20 described with reference to Fig. 1. However, as shown in Fig. 2, the shortest route 20 includes an environment recognition difficult area 21.

[0033] In the process of the present disclosure, before the robot 100 starts running, an analysis process is performed on the path from the start point (S) to the target position (G).

[0034] First, a map that can be referenced by the robot 100 is input. The input map is a map on which attribute information such as location information corresponding to each object and area on the map, pattern, texture, and structure information, etc., is recorded. For example, a 3D BIM (Building Information Modeling) map can be used. The 3D BIM map is 3D map data that includes, for example, the internal structure of a building and attribute information of a structure.

[0035] The map to be used is not limited to the 3D BIM map, but can be either a 3D map or a 2D map. However, it is necessary that the map contains attribute information such as location information corresponding to each of the above objects and areas.

[0036] In the process disclosed herein, before the robot 100 starts running, an analysis process is performed on a map in which various attribute information is recorded for the route from the start point (S) to the target position (G). Specifically, areas in the map that may be the movement route for the robot 100 to reach the target position (G) are analyzed, and a new map, i.e., a "recognition-friendly heat map," is generated that makes it possible to identify "areas that are difficult to recognize the environment" within the analyzed area and "landmarks," which are objects or areas for which position information can be acquired.

[0037] Furthermore, the robot 100 performs an analysis process on various route candidates that can reach the target position (G) based on the input map and the newly generated "recognition-convenience heat map." Specifically, for each route candidate, the robot 100 performs an analysis process on the estimated self-position error and the possibility of collision with an obstacle when the robot 100 travels and performs a self-position estimation process. The estimated self-position error is data that defines an area that includes a predetermined range of error in the estimated self-position calculated in the self-position estimation process performed by the robot 100.

[0038] Based on this analysis, the shortest route from among the route candidates that can safely reach the target position (G) without colliding with any obstacles is determined as the "optimal route." Specifically, for example, an "optimal route 30" as shown in Fig. 2 is determined. The optimal route 30 shown in Fig. 2 is a route that uses landmarks 31 as waypoints.

[0039] As described above, the landmarks 31 are objects or areas for which position information can be acquired. The position information of the landmarks 31 can be acquired from a map referenced by the robot 100 or a "recognition-convenience heat map."

[0040] The position information of the landmarks 31 is used to correct errors that occur in the self-position estimation process executed by the robot 100 while it is running. That is, the robot 100 reaches the position of the landmark 31 from which position information can be acquired, acquires accurate position information from the landmark 31, and corrects errors in the estimated position in the self-position estimation process of the robot 100.

[0041] The position information of the landmark 31 is information recorded in a map referenced by the robot 100 or in a "recognition-convenience heat map" as attribute information corresponding to the landmark 31. The position information of the landmark 31 may be any position information that can be used to correct an error in the estimated position in the self-position estimation process performed by the robot 100, and may be, for example, absolute position information such as latitude and longitude information, or position information indicating a coordinate position on a local map used by the robot.

[0042] The robot 100 determines the shortest route from among route candidates that can safely reach the target position (G) without colliding with obstacles, for example, the "optimal route 30" that uses the landmark 31 shown in Fig. 2 as a waypoint. The robot 100 then travels along the determined "optimal route 30" shown in Fig. 2.

[0043] As shown in FIG. 2, the robot 100 travels through an environment recognition difficult area 21 for a certain distance from the start position (S), but since this travel distance is short, the error in the estimated position in the self-position estimation process does not become excessive, and the robot 100 can reach the landmark 31.

[0044] Since the landmarks 31 are points having position information, the robot 100 inputs this position information and performs a correction process to eliminate position errors that occur in the self-position estimation process. This correction process calculates an error-free self-position.

[0045] After performing a correction process for the estimated self-position using the position information of the landmark 31, the robot 100 starts traveling from the landmark 31 toward the target position (G). That is, the robot 100 starts traveling along the latter half of the "optimal route 30". Note that although the latter half of the "optimal route 30" also includes the difficult-to-recognize environment region 21, the traveling section of this difficult-to-recognize environment region 21 is also short, so the error in the estimated position in the self-position estimation process does not become excessive, and the robot 100 can reach the target position (G).

[0046] In this way, in the processing of the present disclosure, a new map, i.e., a "recognition-friendly heat map," is first generated that makes it possible to identify "areas where environmental recognition is difficult" and "landmarks," which are objects or areas for which location information can be obtained.

[0047] Furthermore, based on the input map and the "recognition-friendly heat map," an analysis process is performed on various route candidates that can reach the target position (G), and the shortest route from among the route candidates that can safely reach the target position (G) without colliding with obstacles is determined as the "optimal route." For example, an "optimal route" is determined using landmarks 31 as waypoints, as shown in FIG. 2, and the robot 100 is caused to travel along the determined "optimal route." By performing such processing, a mobile object such as a robot can be made to safely reach the target position (G) even if an area that is difficult to recognize the environment exists on the route.

[0048] 2. Configuration Example of a Moving Object Path Analysis Device According to the Present Disclosure Next, a configuration example of a moving object path analysis device according to the present disclosure will be described.

[0049] 3 is a block diagram showing an example of the configuration of a robot 100, which is an example of a moving object path analysis device according to the present disclosure. The robot 100 is, for example, an autonomous robot.

[0050] As shown in FIG. 3, the robot 100 includes a sensor 101, a driving unit 102, a data processing unit (processor) 103, a storage unit (memory) 104, a display unit (monitor) 105, and an input unit (UI) 106.

[0051] The sensor 101 is a sensor having at least one of, for example, a camera, LiDAR, GPS (GNSS), etc. The robot 100 can have various configurations, such as a configuration having one of the sensors of a camera, LiDAR, and GPS (GNSS), or a configuration having a plurality of different sensors.

[0052] LiDAR (Light Detection and Ranging) is a sensor that outputs laser light and measures the distance to an object and the three-dimensional shape of the object by analyzing the light reflected from the object. GPS (GNSS) is a satellite positioning system. Both GPS and GNSS are satellite positioning systems. In the following description, GPS and GNSS will be collectively referred to as GPS.

[0053] The robot 100 performs the aforementioned self-position estimation process, such as SLAM (Simultaneous Localization and Mapping), using, for example, detection information from the sensor 101. SLAM is classified into camera-based SLAM (Visual SLAM), which is performed using images captured by a camera, and laser-based SLAM (LiDAR SLAM), which is performed using information detected by a sensor such as LiDAR.

[0054] A robot 100 equipped with a camera as the sensor 101 can perform camera-assisted SLAM (visual SLAM). A robot 100 equipped with a LiDAR as the sensor 101 can perform laser-assisted SLAM (LiDAR SLAM).

[0055] The driving unit 102 executes driving processing for a moving body such as a robot, for example, by controlling the wheels and legs of the robot 100 to make the robot 100 move.

[0056] The data processing unit (processor) 103 has, for example, a processor and executes various data processing operations. Specifically, it executes various operations such as generating a travel route for the robot 100, estimating the robot's position using SLAM processing or the like, analyzing the possibility of collision with an obstacle, generating display information including route information and the like, and generating drive control information for the robot 100. Details and specific examples of the operations executed by the data processing unit (processor) 103 will be described later.

[0057] The storage unit (memory) 104 is composed of recording media such as RAM, ROM, flash memory, etc., and stores a map, which is map information of the space in which the robot 100 travels. It is also used to store sensor detection information such as images captured by sensors such as cameras, metadata corresponding to the images, and other path information generated by the data processing unit 103. It is also used as a work area for data processing by the data processing unit (processor) 104.

[0058] The display unit (monitor) 105 is used for displaying various display information generated by the data processing unit 103, such as a map including route information generated by the data processing unit 103.

[0059] The input unit (UI) 106 is an input unit that constitutes a user interface (UI) that can be operated by a user such as an operator. For example, it is used for inputting the start position and target position of the robot 100, inputting movement start control information for the robot 100, etc.

[0060] The configuration of the robot 100 shown in Fig. 3 is one example of the configuration of the moving object path analysis device of the present disclosure. The moving object path analysis device of the present disclosure is not limited to the configuration of the robot 100 shown in Fig. 3, and can also be configured as a system including, for example, the configuration shown in Fig. 4, that is, the robot 100 and an information processing device 110 that can communicate with the robot 100.

[0061] 4 includes a sensor 101, a driving unit 102, a data processing unit (processor) 103, a storage unit (memory) 104, and a communication unit 107. An information processing device 110 capable of communicating with the robot 100 includes a display unit (monitor) 111, an input unit (UI) 112, and a communication unit 113.

[0062] The sensor 101, drive unit 102, data processing unit (processor) 103, and storage unit (memory) 104 of the robot 100 have the same configuration as those described with reference to Fig. 3. The communication unit 1107 communicates with the information processing device 110 via a communication network.

[0063] The information processing device 110 is, for example, a device such as a PC or a smartphone. The display unit (monitor) 111 displays information received from the robot 100 via the communication unit 113. For example, display information such as a map including route information generated by the data processing unit 103 of the robot 100 is displayed.

[0064] The input unit (UI) 112 is an input unit that constitutes a user interface (UI) that can be operated by a user such as an operator. For example, it is used to input the start position and target position of the robot 100, and to input movement start control information for the robot 100. This input information is transmitted to the robot 100 via the communication unit 113. The communication unit 113 communicates with the robot 100 via a communication network.

[0065] Furthermore, the moving object path analysis device of the present disclosure can also be configured as an information processing device 120 capable of communicating with the robot 100, as shown in Fig. 5. The information processing device 120 capable of communicating with the robot 100 shown in Fig. 5 has a data processing unit 115, and the data processing unit 115 of the information processing device 120 controls the robot 100.

[0066] The robot 100 shown in Fig. 5 has the same configuration as that described with reference to Fig. 4. The information processing device 120 capable of communicating with the robot 100 shown in Fig. 5 has a data processing unit 115, a storage unit (memory) 116, a display unit (monitor) 111, an input unit (UI) 112, and a communication unit 113. The display unit (monitor) 111, the input unit (UI) 112, and the communication unit 113 of the information processing device 120 execute the same processes as those described with reference to Fig. 4.

[0067] The data processing unit 115 of the information processing device 120 has, for example, a processor and executes various data processing operations. Specifically, the data processing unit 115 executes the same processing operations as those executed by the data processing unit 103 of the robot 100 described above with reference to FIG. 3, or a part of the processing operations.

[0068] Specifically, the processing unit 104 executes various processes such as generating a travel route for the robot 100, estimating the robot's position using SLAM processing or the like, analyzing the possibility of collision with an obstacle, generating display information including route information and the like, and generating drive control information for the robot 100. Note that these processes may be shared and executed by the data processing unit 103 of the robot 100.

[0069] 3. Overall Sequence of Mobile Object Control Processing Executed by the Mobile Object Path Analysis Device of the Present Disclosure Next, the overall sequence of mobile object control processing executed by the mobile object path analysis device of the present disclosure will be described.

[0070] Fig. 6 is a diagram illustrating the overall sequence executed by the data processing unit of the moving object path analysis device. The data processing unit of the moving object path analysis device corresponds to, for example, the data processing unit 103 of the robot 100 shown in Fig. 3 or the data processing unit 115 of the information processing device 120 shown in Fig. 5. Either data processing unit can execute similar processing. Below, the processing executed by the data processing unit 103 of the robot 100 in Fig. 3 will be described as a representative example.

[0071] 6, the data processing unit 103 of the robot 100 inputs a map from the memory unit 104, inputs sensor detection data from the sensor 101, and inputs a target position (G) from the input unit (UI) 106. Furthermore, as shown on the right side of Fig. 6, the data processing unit 103 finally generates and outputs drive control information 127 to the drive unit 102, and further generates and outputs display information 128 to the display unit 105.

[0072] The data processing unit 103 executes the processes of steps S11 to S16 shown in Fig. 6. These processes are executed under the control of the processor of the data processing unit 103 in accordance with a program stored in the storage unit 104, for example.

[0073] First, an outline of the series of steps S11 to S16 shown in Fig. 6 will be explained. Details of each step will be explained in order later.

[0074] (Step S11) First, in step S11, the data processing unit 103 generates the recognition-convenient heat map 121 using the map input from the storage unit 104.

[0075] Various maps, including three-dimensional maps such as the BIM map described above, two-dimensional maps, etc., can be used as maps input to the data processing unit 103. In the example shown in Fig. 6, the map is input from the storage unit 104, but it may also be configured to acquire the map from an external device, such as a map providing server, via a communication unit, for example.

[0076] The map used by the data processing unit 103 to generate the recognition heat map 121 is a map to which various attribute information is added for each object and area, such as a building or structure, within the map. The attribute information includes, for example, position information for each object or area, as well as pattern information (texture information) for the floors and walls of the building.

[0077] In step S11, the data processing unit 103 uses the input map to generate the recognition-convenient heat map 121. As described above, the recognition-convenient heat map 121 is a map that makes it possible to identify "areas where environmental recognition is difficult" and "landmarks," which are objects or areas for which position information can be acquired.

[0078] In addition, the "areas where it is difficult to recognize the environment" and "landmarks" may be different areas or objects depending on the type of sensor, for example, camera, LiDAR, GPS, etc., and the recognition-friendly heat map 121 is a map that can identify the "areas where it is difficult to recognize the environment" and "landmarks" according to the type of sensor.

[0079] For example, the "difficult-to-recognize areas" and "landmarks" when using camera-based SLAM (Visual SLAM), the "difficult-to-recognize areas" and "landmarks" when using laser-based SLAM (LiDAR SLAM), and the "difficult-to-recognize areas" and "landmarks" when using GPS may be different.

[0080] In step S11, the data processing unit 103 generates a recognition-convenient heat map 121 that makes it possible to identify "difficult-to-recognize areas" and "landmarks" according to the type of sensor, using the map input from the storage unit 104. Specifically, for example, the recognition-convenient heat map 121 is generated in which "difficult-to-recognize areas" and "landmarks" according to each sensor are set in different colors.

[0081] (Step S12) In step S12, the data processing unit 103 receives the sensor detection data and executes a self-position estimation process. The data processing unit 103 executes the self-position estimation process using the sensor detection data at the current position of the robot 100.

[0082] The sensor detection data is detection data from at least one of a camera, LiDAR, and GPS, or combined data from detection data from multiple sensors.

[0083] For example, when the data processing unit 103 receives a camera-captured image from the sensor 101, the data processing unit 103 executes a camera-assisted SLAM (Visual SLAM) using the camera-captured image in step S12 to perform self-position estimation processing.

[0084] In addition, when the data processing unit 103 inputs LiDAR detection information (object distance information) from the sensor 101, it executes laser-based SLAM (LiDAR SLAM) using the LiDAR detection information (object distance information) to perform self-position estimation processing.

[0085] Furthermore, when the data processing unit 103 receives GPS position information (satellite positioning information) that is GPS detection information from the sensor 101, it performs a self-position estimation process using the GPS position information.

[0086] In addition, when the data processing unit 103 receives input of multiple different sensor detection information from the sensor 101, it executes self-location estimation processing in parallel using each sensor detection information, and calculates one self-location estimation result from the multiple self-location estimation results by a pre-defined integration processing.

[0087] 6, the data processing unit 103 performs the self-location estimation process in step S12 to calculate an estimated self-location error 123 in addition to the estimated self-location 122. The estimated self-location error 123 is data that defines an area including a predetermined range of error in the estimated self-location calculated in the self-location estimation process performed by the robot 100.

[0088] For example, in a self-position estimation process using SLAM, the direction and amount of movement of various objects around a moving body (robot) are analyzed based on camera images and laser analysis data, and the movement of the moving body itself is analyzed, and the direction and distance of movement of the moving body are analyzed to estimate the current self-position.

[0089] However, it is not always possible to reliably analyze the movement direction and amount of movement of various objects around a moving body (robot) based on camera images and laser analysis data. In particular, when the area in which the moving body (robot) travels is an area surrounded by flat walls, corridors, etc. with few features, it is difficult to accurately analyze the movement direction and amount of movement of various objects around the moving body (robot).

[0090] As a result, the error, which is the difference between the self-position estimated by the self-position estimation process and the actual position, gradually increases as the mobile body travels. Therefore, the "estimated self-position error 123," which is data defining an area including a predetermined range of error in the estimated self-position calculated in the self-position estimation process, also gradually increases as the mobile body travels.

[0091] When the data processing unit 103 performs the self-location estimation process in step S12, it also calculates an "estimated self-location error 123," which is data that defines an area that includes a predetermined range of error in the estimated self-location calculated in the self-location estimation process, in addition to the estimated self-location 122. Details and specific examples of this process will be described later.

[0092] (Step S13) Next, in step S13, the data processing unit 103 executes a "process for calculating an estimated self-position error reflecting a difficult-to-recognize area."

[0093] As mentioned above, the "estimated self-position error," which is the difference between the estimated self-position and the actual self-position, gradually increases according to the distance traveled by the robot 100, and the rate of increase in the error becomes particularly large when traveling in an area where environmental recognition is difficult.

[0094] In step S13, the "recognition-convenient heat map 121" generated in the previous step S11, i.e., the "recognition-convenient heat map 121" that makes it possible to identify the "difficult-to-recognize environmental areas," is used to execute an "estimated self-position error calculation process that reflects the difficult-to-recognize environmental areas" that takes the difficult-to-recognize environmental areas into account, thereby generating an "estimated self-position error 124 that reflects the difficult-to-recognize environmental areas."

[0095] For example, the "estimated self-position error 123" is data in which the error increases almost uniformly according to the travel distance of the robot 100, whereas the "difficult-to-recognize area reflected estimated self-position error 124" is data in which the rate of increase in error changes not only according to the travel distance of the robot 100 but also according to the travel environment of the robot 100. In other words, the data is set so that the rate of increase in error when the robot 100 travels in an "area difficult to recognize the environment" is larger than when the robot 100 travels in an area other than the "difficult-to-recognize area."

[0096] Specific examples of the "estimated self-position error 123" and the "difficult-to-recognize area reflected estimated self-position error 124" and details of the calculation process of these errors will be described later.

[0097] (Step S14) Next, in step S14, the data processing unit 103 executes a process of planning a movement path for the robot 100.

[0098] In step S14, the data processing unit 103 executes a process of planning the movement path of the robot 100 using the target position (G) input via the input unit 106, the "recognition-friendly heat map 121" generated in step S11, and the "estimated self-position error 124 reflecting the difficult-to-recognize area" generated in step S13.

[0099] Specifically, a path is generated that enables the robot 100 to safely reach the target position (G) without colliding with obstacles such as walls on the path. For example, if an "environment recognition-difficult area" exists on the shortest path from the current position of the robot 100 to the target position (G), and the "recognition-difficult area-reflecting estimated self-position error 124" becomes large, it is predicted that the robot 100 will collide with an obstacle such as a wall on the path.

[0100] When such a prediction is made, a route other than the shortest route from the current position of the robot 100 to the target position (G) is planned. For example, a new optimal route is generated using landmarks for which position information set in the "recognition-convenience heat map 121" is available as waypoints. The process of generating this optimal route will be described in detail later.

[0101] As shown in FIG. 6, in step S14, the data processing unit 103 executes a process for planning a movement route of the robot 100 to generate an "optimal route 125" and further generates "route supplemental information 126."

[0102] The "route supplemental information 126" is information used for display processing, and is configured, for example, with information indicating areas where there is a possibility of collision, etc. Specific examples will be described later.

[0103] (Step S15) Next, in step S15, the data processing unit 103 executes a drive control information generation process.

[0104] That is, the "drive control information 127" for moving the robot 100 according to the "optimal route 125" generated in step S14 is generated and output to the drive unit 102, causing the robot 100 to execute the movement process.

[0105] "Drive control information 127" is generated to move the robot 100 along the "optimal route 125" generated in step S14, for example, a new route with landmarks as waypoints whose identifiable position information can be obtained from the "recognition-friendly heat map 121", and is output to the drive unit 102 to execute the movement process of the robot 100.

[0106] (Step S16) Next, in step S16, the data processing unit 103 executes a display information generation process.

[0107] In step S16, the data processing unit 103 receives the following pieces of information as input, and generates display information 128 to be output to the display unit 105 based on the input information: (a) a map stored in the storage unit 104; (b) a target position (G) input by the user via the input unit 106; (c) a "recognition-friendly heat map 121" generated in step S11; (d) an "estimated self-position 122" generated in step S12; (e) an "estimated self-position error 124 reflecting a difficult-to-recognize area" generated in step S13; (f) an "optimal route 125" generated in step S14; and (g) "route supplementary information 126" generated in step S14.

[0108] In step S16, the data processing unit 103 uses the above information (a) to (g) to generate display information 128 to be output to the display unit 105. Specific examples of the display information 128 will be described later.

[0109] The series of processes executed by the data processing unit of the moving object route analysis device of the present disclosure has been described above.

[0110] Next, a processing sequence executed by the data processing unit of the mobile object route analysis device of the present disclosure will be described with reference to the flowchart shown in Fig. 7. Note that the processing according to the flow shown in Fig. 7 and subsequent figures is executed under the control of the data processing unit, which is composed of a CPU or the like having a program execution function, in accordance with a program stored in the storage unit of the mobile object route analysis device. The processing of each step of the flow shown in Fig. 7 will be described in order below.

[0111] (Step S101) First, in step S101, the data processing unit 103 of the robot 100 inputs a map. The input map is either a three-dimensional map such as the BIM map described above, a two-dimensional map, or the like. The input map is input from the storage unit 104 or from an external source.

[0112] (Step S102) Next, in step S102, the data processing unit 103 generates a recognition heat map using the input map.

[0113] This process corresponds to the process of step S11 described with reference to Fig. 6. As described with reference to Fig. 6, the recognition-convenient heat map 121 is a map that makes it possible to identify "areas where environmental recognition is difficult" and "landmarks," which are objects or areas for which position information can be acquired.

[0114] As mentioned above, the "areas where it is difficult to recognize the environment" and "landmarks" may be different areas or objects depending on the type of sensor, such as a camera, LiDAR, GPS, etc., and the recognition-friendly heat map 121 is a map that makes it possible to identify the "areas where it is difficult to recognize the environment" and "landmarks" according to the type of sensor.

[0115] In step S102, the data processing unit 103 uses the input map to generate a recognition-convenient heat map 121 that makes it possible to identify "difficult-to-recognize areas" and "landmarks" according to the type of sensor. Specifically, for example, the recognition-convenient heat map 121 is generated in which the "difficult-to-recognize areas" and "landmarks" according to each sensor are set in different colors.

[0116] (Step S103) Next, in step S103, the data processing unit 103 inputs a target position (G) from the input unit 106. The target position (G) is input by a user such as an operator, for example.

[0117] (Step S104) Next, in step S104, the data processing unit 103 receives the sensor detection data and executes a self-position estimation process. The data processing unit 103 executes the self-position estimation process using the sensor detection data at the current position of the robot 100.

[0118] This process corresponds to the process of step S12 described with reference to Fig. 6. As described with reference to Fig. 6, the sensor detection data is detection data from at least one sensor selected from a camera, LiDAR, and GPS, or combined data of detection data from multiple sensors.

[0119] Based on the sensor detection values ​​input from the sensor 101, the data processing unit 103 performs, for example, one of the following self-position estimation processes: self-position estimation process using camera-based SLAM (Visual SLAM); self-position estimation process using laser-based SLAM (LiDAR SLAM); or self-position estimation process using GPS position information.

[0120] (Step S105) Next, in step S105, the data processing unit 103 executes a calculation process of the "difficult-to-recognize area reflected estimated self-position error" taking into account the difficult-to-recognize area.

[0121] This process corresponds to the process of step S13 described with reference to Fig. 6. As described with reference to Fig. 6, the estimated self-location error is data that includes a predetermined range of error area in the estimated self-location calculated in the self-location estimation process, and gradually increases as the mobile object travels. Furthermore, the rate of increase in the error increases when the mobile object travels in an "environment recognition difficult area."

[0122] In step S105, a distinction is made between "areas where it is difficult to recognize the environment" and "areas other than areas where it is difficult to recognize the environment," and a calculation process is performed to calculate an "estimated self-position error reflecting areas where it is difficult to recognize" by changing the rate of increase in the "self-position error" when traveling through each area.

[0123] Details and specific examples of the calculation process for this "estimated self-position error reflecting difficult-to-recognize area" will be explained later.

[0124] (Step S106) Next, in step S106, the data processing unit 103 executes a process of planning an optimal path for the robot 100.

[0125] This process corresponds to the process of step S14 described with reference to Fig. 6. In step S106, the data processing unit 103 executes a process of generating an optimal path for the robot 100 using the target position (G) input via the input unit 106, the "recognition-friendly heat map 121" generated in step S102, and the "difficult-to-recognize area-reflecting estimated self-position error 124" generated in step S105.

[0126] Specifically, a path is generated that allows the robot 100 to safely reach the target position (G) without colliding with obstacles such as walls along the path. As described above, for example, the optimal path is a new path that uses landmarks, whose position information can be acquired and that are set in the "recognition-convenience heat map 121," as waypoints.

[0127] (Step S107) Next, in step S107, the data processing unit 103 generates drive control information for moving along the optimal route and executes drive processing for the robot.

[0128] This process corresponds to the process of step S15 described with reference to Fig. 6. That is, "drive control information 127" for moving the robot 100 according to the optimal route generated in step S106, i.e., the "optimal route 125" shown in Fig. 6, is generated and output to the drive unit 102, causing the robot 100 to execute the movement process.

[0129] (Step S108) Next, in step S108, the data processing unit 103 generates display information including the optimum route and the position of the robot 100 and outputs it to the display unit 105.

[0130] This process corresponds to the process of step S16 described with reference to Fig. 6. That is, the display information 128 shown in Fig. 6 is generated and output to the display unit 105.

[0131] (Step S109) Next, in step S109, the data processing unit 103 executes a process for correcting the optimum route as necessary.

[0132] This process is a path correction process executed by a user such as an operator who monitors the movement of the robot 100, and is executed as needed, but can be omitted if not required.

[0133] (Step S110) Next, in step S110, the data processing unit 103 determines whether the robot 100 has reached the target position (G) input in step S103. If the robot 100 has not reached the target position (G), the process returns to step S104 and repeats the processes from step S104 onwards.

[0134] If it is determined that the target position (G) input in step S103 has been reached, the process ends.

[0135] The processing from step S104 onwards is repeated until the robot 100 reaches the target position (G), but the interval between these processing steps can be set in various ways, such as a predetermined time interval, for example, repeating every 10 seconds, or a predetermined distance traveled, for example, repeating every 20 meters.

[0136] The main processes executed by the moving body path analysis device of the present disclosure can be summarized as follows: (a) A process of generating a "recognition-convenient heat map" that makes it possible to identify "difficult-to-recognize areas" and "landmarks," which are objects and areas for which position information can be acquired; (b) A process of using the "recognition-convenient heat map" to distinguish between "difficult-to-recognize areas" and "areas other than difficult-to-recognize areas" and to generate an "estimated self-position error reflecting difficult-to-recognize areas" that changes the rate of increase in "self-position error" when traveling through each area; (c) A process of using the "estimated self-position error reflecting difficult-to-recognize areas" to generate an "optimal route" that prevents the moving body (robot) from colliding with obstacles; (d) A process of moving the moving body according to the generated "optimal route"; (e) A process of generating display information including the generated "recognition-convenient heat map" and "optimal route," etc., and outputting it to a display unit;

[0137] By performing these processes, the moving object route analysis device of the present disclosure can allow the moving object to safely reach the target position (G). Furthermore, it can provide a user such as an operator with information including areas on the optimal route of the moving object that are difficult to recognize the environment, and the optimal route.

[0138] [4. Details of Each Process Executed by the Data Processing Unit of the Moving Object Route Analysis Device of the Present Disclosure] Next, details of each process executed by the data processing unit of the moving object route analysis device of the present disclosure will be described.

[0139] The details of each of the following processes will be explained in order: (1) Recognition heat map generation process (step S11 in FIG. 6, step S102 in FIG. 7) (2) Self-position estimation process (step S12 in FIG. 6, step S104 in FIG. 7) (3) Calculation process of estimated self-position error reflecting difficult-to-recognize area (step S13 in FIG. 6, step S105 in FIG. 7) (4) Optimal route generation process (step S14 in FIG. 6, step S106 in FIG. 7) (5) Display information generation process (step S16 in FIG. 6, step S108 in FIG. 7)

[0140] (4-(1) Recognition-Convenient Heat Map Generation Process (Step S11 in FIG. 6, Step S102 in FIG. 7)) First, the recognition-convenient heat map generation process will be described in detail.

[0141] The process of generating the recognition-convenient heat map is executed in step S11 of Fig. 6 described above or in step S102 of the flowchart shown in Fig. 7. For example, the data processing unit 103 of the robot 100 shown in Fig. 3 generates the recognition-convenient heat map 121 using a map input from the storage unit 104.

[0142] The map input to the data processing unit 103 can be a 3D map such as the BIM map mentioned above, a 2D map, or various other maps. However, this map has various attribute information added to each object and area such as a building or structure in the map. The attribute information includes, for example, position information for each object or area, and information such as pattern information (texture information) for the floors and walls of the building.

[0143] The data processing unit 103 uses the input map to generate a recognition-friendly heat map 121. As described above, the recognition-friendly heat map 121 is a map that makes it possible to identify "areas where environmental recognition is difficult" and "landmarks," which are objects or areas for which position information can be acquired.

[0144] As described above, the "difficult-to-recognize environmental areas" and "landmarks" may be different areas or objects depending on the type of sensor, such as a camera, LiDAR, or GPS, and the recognition-friendly heat map 121 is a map that makes it possible to identify the "difficult-to-recognize environmental areas" and "landmarks" corresponding to the type of sensor. Specifically, the recognition-friendly heat map 121 is generated by setting the "difficult-to-recognize environmental areas" and "landmarks" corresponding to each sensor in different colors, for example.

[0145] The landmark position information may be any position information that can be used to correct errors in the estimated position in the self-position estimation process performed by a mobile body such as a robot, and may be, for example, absolute position information such as latitude and longitude information, or position information indicating a coordinate position on a local map used by the robot.

[0146] As described above, a recognition-friendly heat map is a map that makes it possible to identify "areas that are difficult to recognize the environment" and "landmarks," which are objects or areas for which location information can be acquired. In the example described above with reference to FIG. 1, an "area that is difficult to recognize the environment" is, for example, a space surrounded by flat, patternless walls, floors, ceilings, etc., and is an area in which the surrounding scenery changes very little even when moving from one location to another. Also, a "landmark" is an object or area for which location information can be acquired.

[0147] However, the definitions of these "areas where it is difficult to recognize the environment" and "landmarks" differ depending on the sensors attached to a mobile body such as a robot. With reference to Figure 8, the definitions of "areas where it is difficult to recognize the environment" and "landmarks" according to the sensors used by a mobile body such as a robot will be described.

[0148] 8 is a diagram illustrating the following definitions of "difficult-to-recognize environmental areas" and "landmarks" that can be identified from the recognition heat map: (a) Common definition (definition common to each sensor) (b1) Camera-compatible definition for performing self-location estimation processing using camera-based SLAM (Visual SLAM) (b2) LiDAR-compatible definition for performing self-location estimation processing using laser-based SLAM (LiDAR SLAM) (b3) GPS-compatible definition for performing self-location estimation processing using a satellite positioning system

[0149] First, we will explain the "area where it is difficult to recognize the environment." (a) The common definition of the "area where it is difficult to recognize the environment" (a definition common to all sensors) is as follows: The "area where it is difficult to recognize the environment" is an area where it is difficult to estimate the self-position with high accuracy.

[0150] However, the areas where it is difficult to estimate one's own position with high accuracy differ depending on the sensor used. (b1) to (b3) in the table of Fig. 8 show "areas where it is difficult to recognize the environment" depending on the sensor used. The "areas where it is difficult to recognize the environment" depending on the sensor used will be explained below with reference to the table.

[0151] (b1) The definition of a camera-compatible "difficult-to-recognize environment area" that performs self-location estimation processing using camera-based SLAM (Visual SLAM) will be described. When a camera is used as a sensor, the self-location estimation processing is performed by analyzing the image captured by the camera. Therefore, an area that is difficult to distinguish from other image areas in the image captured by the camera becomes an "difficult-to-recognize environment area."

[0152] Specifically, as shown in Figure 8, the following areas are "difficult-to-recognize environments": (Example 1) Area with little texture (Example 2) Dark area (Example 3) Area with only repeated patterns (Example 4) Area with many moving objects

[0153] (b2) The definition of a "difficult-to-recognize environment area" for LiDAR, which performs self-location estimation processing using laser-assisted SLAM (LiDAR SLAM), is explained below. When LiDAR is used as a sensor, distance information obtained as laser detection data is analyzed to perform self-location estimation processing. Therefore, an area that is difficult to distinguish from other areas based on distance information obtained as laser detection data is called an "difficult-to-recognize environment area."

[0154] Specifically, as shown in Figure 8, the following areas are "difficult-to-recognize areas": (Example) Areas surrounded by flat corridors, floors, and ceilings

[0155] (b3) A definition of a GPS-compatible "environment recognition difficult area" for performing self-location estimation processing using a GPS satellite positioning system will be described. When using GPS or GNSS as a sensor, the self-location estimation processing is performed by analyzing data received from a satellite. Therefore, an area where it is difficult to receive data from a satellite is an "environment recognition difficult area."

[0156] Specifically, as shown in FIG. 8, the following areas are "difficult-to-recognize areas": (Example 1) Indoors (Example 2) Underground mall

[0157] Next, a description will be given of a "landmark." (a) The common definition of a "landmark" (a definition common to all sensors) is as follows: A "landmark" is an object or area for which position information can be acquired.

[0158] As mentioned above, the location information may be any location information that can be used to correct errors in the estimated location in the self-location estimation process performed by a mobile body such as a robot, and may be, for example, absolute location information such as latitude and longitude information, or location information that indicates a coordinate position on a local map used by the robot.

[0159] "Landmarks" also become different objects or areas depending on the sensor used. (b1) to (b3) in the table of Fig. 8 show explanations of "landmarks" according to the sensor used. Below, "landmarks" according to the sensor used will be explained with reference to the table.

[0160] (b1) The definition of a camera-compatible "landmark" that executes self-location estimation processing using camera-assisted SLAM (Visual SLAM) will be described.

[0161] When using a camera as a sensor, a process is executed to detect objects or areas that can be used as "landmarks" from the image captured by the camera. Therefore, objects or areas that can be distinguished from other image areas in the image captured by the camera and for which location information can be obtained become "landmarks."

[0162] When using a camera as a sensor, the location information is obtained from a map referenced by a mobile object such as a robot, or from a "recognition-specific heat map." Objects or areas whose location information is recorded as attribute information on the map referenced by the mobile object or on the "recognition-specific heat map" become "landmarks."

[0163] That is, as shown in Figure 8, objects and areas that satisfy the following two conditions (Condition 1) and (Condition 2) become "landmarks" when using a camera as a sensor: (Condition 1) Objects and areas for which location information is registered as attribute information (Condition 2) Objects and areas that can be identified from images captured by the camera

[0164] Specific examples include objects or areas that have characteristics that can be identified from camera images, and objects or areas that have at least image characteristics, such as buildings, structures, objects, patterns, image markers, etc., that can become "landmarks."

[0165] (b2) The definition of a LiDAR-compatible "landmark" that performs self-localization processing using laser-assisted SLAM (LiDAR SLAM) is explained below. When LiDAR is used as a sensor, the self-localization processing is performed by analyzing distance information of surrounding objects and areas obtained as laser detection data. Therefore, a "landmark" is an object or area that can be distinguished from other areas based on distance information obtained as laser detection data and from which position information can be obtained.

[0166] Even when LiDAR is used as a sensor, location information is obtained from a map referenced by a mobile object such as a robot, or from a "recognition-specific heat map." Objects or areas whose location information is recorded as attribute information on the map referenced by the mobile object or the "recognition-specific heat map" become "landmarks."

[0167] That is, as shown in Fig. 8, an object or area that satisfies the following two conditions (Condition 1) and (Condition 2) becomes a "landmark" when using LiDAR as a sensor: (Condition 1) An object or area whose position information is registered as attribute information; (Condition 2) An object or area that can be identified from laser analysis information (three-dimensional structure data).

[0168] Specifically, objects or areas that can be "landmarks" include buildings, structures, objects, statues, etc., which are objects with three-dimensional features that can be identified from laser analysis information (three-dimensional structural data). However, planar image markers, etc., are not included because they cannot be identified from laser analysis information (three-dimensional structural data).

[0169] (b3) A definition of a GPS-compatible "landmark" for performing self-location estimation processing using a GPS satellite positioning system will be described. When using GPS or GNSS as a sensor, the self-location estimation processing is performed by analyzing data received from a satellite. Therefore, an area where data from a satellite can be received becomes a "landmark."

[0170] Specifically, as shown in Fig. 8, the following areas are "landmarks" when using GPS or GNSS as sensors: (Example 1) Outdoor area (Example 2) Indoor area near a window or door

[0171] 9 to 11 show examples of "recognition-convenient heat maps" that enable identification of "areas difficult to recognize the environment" and "landmarks" according to the type of sensor (camera, LiDAR, GPS, or GNSS) used by the moving object. Figures 9 to 11 show the following three examples of "recognition-convenient heat maps."

[0172] (1) A "recognition-friendly heat map" when implementing camera-based SLAM (Visual SLAM) using a camera as the sensor. (2) A "recognition-friendly heat map" when implementing laser-based SLAM (LiDAR SLAM) using LiDAR as the sensor. (3) A "recognition-friendly heat map" when using GPS as the sensor. All of these are maps of floors in the same building (school), with an entrance in the lower right corner and multiple classrooms.

[0173] In the example of "(1) 'Recognition-friendly heat map' when executing camera-assisted SLAM (Visual SLAM) using a camera as a sensor" shown in Figure 9, classrooms with the same shape are set as "difficult-to-recognize areas." The elevator section and entrance hall are also set as "landmarks." The elevator section and entrance hall are objects whose position information is recorded in the "recognition-friendly heat map," and it is possible to obtain position information from these objects.

[0174] The hallway between the two classrooms on the left side is not set as an "area where environmental recognition is difficult." This is because the walls of the hallway have a non-repeated pattern, and it is an area where highly accurate self-location estimation processing can be performed by position identification based on camera images.

[0175] In the example of "(2) 'Recognition-Convenient Heat Map' when Laser-Based SLAM (LiDAR SLAM) Using LiDAR as a Sensor" shown in Figure 10, classrooms with the same shape are set as "Difficult-to-Recognize Environment Areas." The elevator and entrance hall are also set as "Landmarks." The elevator and entrance hall are objects whose position information is recorded in the "Recognition-Convenient Heat Map," and it is possible to obtain position information from these objects.

[0176] Unlike the example (1) above where cameras are used, the hallway between the two classrooms on the left is set as a "difficult-to-recognize area." This is because even if the hallway walls have a non-repeating pattern, the walls are flat, making it nearly impossible to identify the location using LiDAR detection information (distance information), and therefore making it difficult to perform highly accurate self-location estimation processing using laser-based SLAM (LiDAR SLAM).

[0177] In the example of "(3) 'Heat map for recognition' when GPS is used as a sensor" shown in Fig. 11, all areas except for areas near windows and doors are set as "areas difficult to recognize the environment." Furthermore, areas near windows and doors are set as "landmarks."

[0178] Since data transmitted from satellites such as GPS and GNSS can only be received in areas near windows and doors, only these areas are set as areas where location information can be acquired, i.e., "landmarks." Areas other than those near windows and doors cannot receive data transmitted from satellites such as GPS and GNSS, and location information cannot be acquired, so all of these areas are set as "areas where environment recognition is difficult."

[0179] Note that the three types of "recognition-convenient heat maps" described with reference to Figures 9 to 11 are "recognition-convenient heat maps" referenced by mobile bodies that use either a camera, LiDAR, or GPS as a sensor.

[0180] The "recognition heat map" referenced by a mobile body that uses multiple different sensors, such as a camera and LiDAR, or a camera and GPS, or a camera, LiDAR, and GPS, is a map that integrates (synthesizes) the three types of "recognition heat maps" shown in Figures 9 to 11.

[0181] Regarding "landmarks" in "recognition-specific heat maps" corresponding to different sensors, it is possible to set all of the "landmarks" in the "recognition-specific heat maps" corresponding to each sensor before integration in the "recognition-specific heat map" after integration.

[0182] For example, a mobile object having a camera and a GPS as sensors will generate and use a single "recognition-specific heat map" that combines the "recognition-specific heat map" for the camera (1) in Figure 9 and the "recognition-specific heat map" for the GPS (3) in Figure 11. When combining these "recognition-specific heat maps," all "landmarks" are recorded, including the "landmarks" on the "recognition-specific heat map" for the camera (1) in Figure 9 and the "landmarks" on the "recognition-specific heat map" for the GPS (3) in Figure 11.

[0183] In other words, for "landmarks" that indicate objects or areas for which location information can be obtained, all can be used as "landmarks" as long as location information can be obtained from any one sensor, so when integrating "recognition-related heat maps," all "landmarks" on the "recognition-related heat maps" corresponding to each sensor before integration are recorded.

[0184] On the other hand, with regard to the "difficult-to-recognize areas," it is possible to selectively use one of several different processes to determine how the "difficult-to-recognize areas" in the "recognition-friendly heat maps" corresponding to each sensor before integration are reflected in the "recognition-friendly heat map" after integration.

[0185] Two examples of integration processing of "difficult-to-recognize regions" that can be performed when integrating "recognition-friendly heat maps" corresponding to different sensors will be described with reference to FIG.

[0186] 12 shows a diagram for explaining the following two examples of integrating "difficult-to-recognize regions." (a) Example 1 of integrating difficult-to-recognize regions corresponding to different sensors (example of processing in which priority is given to shortening the movement path) (b) Example 2 of integrating difficult-to-recognize regions corresponding to different sensors (example of processing in which priority is given to minimizing positional deviation) Either of these two examples of processing can be selected by user specification (input).

[0187] 12(a) shows an example 1 of integration processing of difficult-to-recognize regions corresponding to two different sensors, which is an example of processing prioritizing the shortest travel path. The integration processing example shown in FIG. 12(a) is an example in which only the overlapping areas of the difficult-to-recognize regions in the "recognition-friendly heat maps" corresponding to two different sensors are reflected in the integrated "recognition-friendly heat map."

[0188] This processing example shows how to minimize the "difficult-to-recognize areas" in the integrated "recognition-friendly heat map." Minimizing the "difficult-to-recognize areas" in this way allows a mobile object, such as a robot, to move over a wider area. In other words, when setting a travel route that avoids the "difficult-to-recognize areas," this has the advantage of increasing the degree of freedom in setting the route.

[0189] 12B shows an example of integration processing for two different sensor-compatible difficult-to-recognize regions, in which minimization of positional deviation is prioritized. The integration processing example shown in FIG. 12B is an example in which all of the difficult-to-recognize regions in each of the two sensor-compatible "recognition-friendly heat maps" are reflected in the integrated "recognition-friendly heat map."

[0190] This processing example maximizes the "difficult-to-recognize area" in the integrated "heat map for recognition." By maximizing the "difficult-to-recognize area" in this way, a mobile object such as a robot moves while avoiding areas that are "difficult-to-recognize areas" for at least one of the sensors in use. Setting a route that avoids "difficult-to-recognize areas" using the "heat map for recognition" generated by this integration process has the advantage of allowing the setting of a route consisting only of areas where self-location estimation processing can be performed reliably with high accuracy.

[0191] The process of generating the "recognition-convenient heat map" for each sensor described with reference to Figures 9 to 11 and the "recognition-convenient heat map" obtained by combining the "recognition-convenient heat maps" for multiple sensors is executed in step S11 of Figure 6 described above and step S102 of the flowchart shown in Figure 7. For example, the data processing unit 103 of the robot 100 shown in Figure 3 generates the recognition-convenient heat map 121 using a map input from the memory unit 104.

[0192] As described above, the map input to the data processing unit 103 can be a 3D map such as the BIM map described above, a 2D map, or various other maps. However, these are maps to which various attribute information is added for each object and area, such as a building or structure, within the map. Attribute information includes, for example, position information for each object or area, and information such as pattern information (texture information) for the floors and walls of a building.

[0193] Specific examples of map-related attribute information required to generate a camera-compatible "recognition-friendly heat map" include the following: Location information linked to an object or area Pattern information (texture information) of building floors and walls (e.g., wallpaper patterns, rug patterns, furniture locations, wall materials, etc.) Brightness information (e.g., lighting locations and lighting times) Information on the presence or absence of repeating patterns (e.g., wallpaper patterns, rug patterns, etc.) Information on the frequency of appearance of moving objects (e.g., times of day with heavy foot traffic, times when construction workers are present, etc.)

[0194] Specific examples of map-related attribute information required to generate a LiDAR-compatible "recognition heat map" include the following: Location information associated with an object or area Presence or absence of structural features for each object or area (e.g., presence or absence of three-dimensional structural features such as walls, corridors, etc.)

[0195] Specific examples of map-related attribute information required to generate a GPS-enabled "recognition-friendly heat map" include the following: Information on whether or not a region can receive GPS satellite radio waves (e.g., whether or not the region is near an indoor window or entrance)

[0196] In this way, the mobile object path analysis device disclosed herein, for example, the data processing unit 103 of the robot 100 shown in FIG. 3, acquires a map on which various attribute information is recorded from the memory unit 104, etc., in step S11 of FIG. 6 and step S102 of FIG. 7, and generates a "recognition-convenience heat map."

[0197] The "recognition-friendly heat map" generated by the data processing unit 103 of the robot 100 is a map that distinguishes at least two elements: "difficult-to-recognize areas," which are areas where highly accurate self-location estimation is difficult, and "landmarks," which are objects or areas for which location information can be acquired. A mobile body such as a robot refers to this "recognition-friendly heat map" to determine the optimal route to a target position (G).

[0198] (4-(2) Self-Location Estimation Process (Step S12 in FIG. 6, Step S104 in FIG. 7)) Next, the self-location estimation process will be described in detail. The self-location estimation process is the process executed in step S12 in FIG. 6 described above and step S104 in the flowchart shown in FIG. 7.

[0199] 3 executes self-location estimation processing using sensor detection data at the current position of the robot 100. The sensor detection data is detection data from at least one sensor selected from a camera, LiDAR, and GPS, or a combination of detection data from multiple sensors.

[0200] For example, when the data processing unit 103 receives a camera-captured image from the sensor 101, the data processing unit 103 performs self-location estimation processing by executing camera-assisted SLAM (Visual SLAM) using the camera-captured image. Furthermore, when the data processing unit 103 receives LiDAR detection information (object distance information) from the sensor 101, the data processing unit 103 performs self-location estimation processing by executing laser-assisted SLAM (LiDAR SLAM) using the LiDAR detection information (object distance information). Furthermore, when the data processing unit 103 receives GPS position information (satellite positioning information), which is GPS detection information, from the sensor 101, the data processing unit 103 performs self-location estimation processing using the GPS position information.

[0201] In addition, when the data processing unit 103 receives input of multiple different sensor detection information from the sensor 101, it executes self-location estimation processing in parallel using each sensor detection information, and calculates one self-location estimation result from the multiple self-location estimation results by a pre-defined integration processing.

[0202] For example, when the robot 100 shown in FIG. 3 uses a GPS as the sensor 101 to receive position information from a GPS satellite and perform self-position estimation processing, the absolute position can be calculated based on the latitude and longitude information received from the satellite.

[0203] 3 uses a camera as the sensor 101, it performs self-location estimation processing by executing camera-assisted SLAM (Visual SLAM) using an image captured by the camera. Also, when the robot 100 shown in FIG. 3 uses LiDAR as the sensor 101, it performs self-location estimation processing by executing laser-assisted SLAM (LiDAR SLAM) using LiDAR detection information (object distance information).

[0204] The self-position of a moving body (robot) estimated in a self-position estimation process using SLAM using these camera images or LiDAR detection information (object distance information) is expressed as probability distribution data.

[0205] For example, as shown in Figure 13, assume that robot 100 moves from point A to point B, and at point B, performs self-position estimation processing using SLAM using camera images or LiDAR detection information (object distance information).

[0206] The result of the self-location estimation process using SLAM is calculated as an estimated self-location probability distribution curve as shown in the graph on the right side of the figure. The estimated self-location probability distribution curve is a curve that shows the possibility of the robot 100 being located as a probability value. Note that although the estimated self-location probability distribution curve shown in FIG. 13 is shown as one-dimensional data in the x direction, it is actually a multidimensional normal distribution curve.

[0207] The center P of the estimated self-location probability distribution curve shown in FIG. 13 is the position where the robot 100 is most likely to be located, and corresponds to the mean value of the multidimensional normal distribution.

[0208] The estimated self-location probability distribution curve shown in Fig. 13 is shown as one-dimensional data in the x direction, and an example of this expanded onto the xy plane on which the robot 100 moves is shown in Fig. 14. The upper part of Fig. 14 shows one-dimensional data in the x direction of the estimated self-location probability distribution curve, which is made up of the normal distribution curve described with reference to Fig. 13.

[0209] The ellipse shown in the lower part of Fig. 14 is a circle (ellipse) obtained by expanding the region of the mean value ± standard deviation (σ) of the upper estimated self-location probability distribution curve onto the xy plane where the robot 100 moves. The probability that the robot 100 exists in the region of the mean value ± standard deviation (σ) of the upper estimated self-location probability distribution curve is approximately 68% because the estimated self-location probability distribution curve is a normal distribution curve. Therefore, the circular region shown in the lower part of Fig. 14 is the region where the probability that the robot 100 exists is approximately 68% on the xy plane where the robot 100 moves.

[0210] When a mobile body such as a robot 100 moving on an xy plane performs a self-location estimation process by executing a camera-based SLAM (Visual SLAM) or a laser-based SLAM (LiDAR SLAM), a process is performed to calculate, for example, a circular area shown at the bottom of Figure 14 as an area with a high probability of being the mobile body's self-location.

[0211] The "estimated self-location 122" generated during the self-location estimation process in step S12 shown in FIG. 6 is the center position P of the circular area shown in FIG. 14 or the position P of the average value of the estimated self-location probability distribution curve in the upper part. The "estimated self-location error 123" also generated during the self-location estimation process in step S12 corresponds to the average value ± standard deviation (σ) of the estimated self-location probability distribution curve in the upper part of FIG. 14, and the circular area shown in the lower part of FIG. 14 corresponds to the "area of ​​the estimated self-location error" in the xy plane on which the robot 100 moves. The example in which the "estimated self-location error 123" generated during the self-location estimation process in step S12 shown in FIG. 6 is the average value ± standard deviation (σ) of the estimated self-location probability distribution curve in the upper part of FIG. 14 is just an example. The "estimated self-location error 123" generated during the self-location estimation process is not limited to this example, and may be calculated using the above formula, for example: average value ± nσ of the estimated self-location probability distribution, where σ is the standard deviation and n is an arbitrary positive number. However, the positive number n is predetermined.

[0212] (4-(3) Calculation Process of Estimated Self-Location Error Reflecting Recognition Difficult-to-Recognize Region (Step S13 in FIG. 6, Step S105 in FIG. 7)) Next, the calculation process of the estimated self-location error reflecting the recognition difficult region will be described in detail.

[0213] The process of calculating the estimated self-position error reflecting the difficult-to-recognize area is the process executed in step S13 of FIG. 6 described above or step S105 of the flowchart shown in FIG.

[0214] For example, in step S13 of Fig. 6, the data processing unit 103 of the robot 100 shown in Fig. 3 inputs the "recognition-convenient heat map 121" generated in step S11 of Fig. 6, i.e., the "recognition-convenient heat map 121" that makes it possible to identify "difficult-to-recognize environmental regions," and the estimated self-position error 123 calculated in the self-position estimation process in step S12, and executes a "difficult-to-recognize region-reflecting estimated self-position error calculation process." As a result of this process, a "difficult-to-recognize region-reflecting estimated self-position error 124" shown in Fig. 6 is generated.

[0215] Specific examples of the "estimated self-location 122" and "estimated self-location error 123" generated during the self-location estimation process in step S12 shown in Fig. 6 will be described with reference to Fig. 15. As previously described with reference to Fig. 14, Fig. 15 shows, at the top, one-dimensional data in the x direction of an estimated self-location probability distribution curve consisting of a normal distribution curve generated by the self-location estimation process. At the bottom, a circular region obtained by expanding the region of the mean value ± standard deviation (σ) of the estimated self-location probability distribution curve onto the xy plane on which the robot 100 moves is shown.

[0216] The "estimated self-location 122" generated during the self-location estimation process in step S12 shown in Fig. 6 is the center position P of the circular region where the average value P of the estimated self-location probability distribution curve at the top is located, or the average value P of the estimated self-location probability distribution curve at the top, as shown in Fig. 15. The "estimated self-location error 123" corresponds to the region of the average value ± standard deviation (σ) of the estimated self-location probability distribution curve at the top of Fig. 15, or the circular region shown at the bottom of Fig. 15.

[0217] In this way, the "estimated self-position error 123" is data that indicates a region that includes a predetermined range of error in the estimated self-position calculated in the self-position estimation process (for example, a region of the mean value ± standard deviation (σ) of the estimated self-position probability distribution curve).

[0218] The "estimated self-location 122" is the location where the robot 100 is most likely to actually be located, but the actual location of the robot 100 is not necessarily the "estimated self-location 122." In other words, the actual location of the robot 100 is likely to be in a nearby area centered on the "estimated self-location 122." The "estimated self-location error 123" indicates an area where the robot 100 is likely to be located, and is data that defines a predetermined error range of the "estimated self-location 122" (for example, a range of the mean value ± standard deviation (σ) of the estimated self-location probability distribution curve). However, as described above, the "estimated self-location error 123" is not limited to this example, and may be calculated using, for example, the mean value ± nσ of the estimated self-location probability distribution, where σ is the standard deviation and n is an arbitrary positive number.

[0219] 15 is the "estimated self-location error 123" on the xy plane on which the robot 100 moves, and defines the area where the robot 100 is likely to be located, i.e., the error range of the "estimated self-location 122." As described above, the estimated self-location probability distribution curve is a normal distribution curve, and the "estimated self-location error 123," which is the circular area shown in the bottom of Fig. 15, is an area that indicates the area on the xy plane on which the robot 100 moves, with a probability of the robot 100 being present of approximately 68%.

[0220] This "estimated self-position error 123" increases in accordance with the travel distance of the robot 100. That is, the error in the estimated self-position calculated by the SLAM process, which is executed continuously or intermittently in parallel with the travel process of the robot 100, gradually increases.

[0221] However, the rate of increase in the estimated self-location error differs depending on the driving environment of the robot 100. When the robot 100 is driving in a driving environment where highly accurate self-location estimation is possible, the rate of increase in the estimated self-location error is low. However, when the robot 100 is driving in a driving environment where highly accurate self-location estimation is difficult, the rate of increase in the estimated self-location error is high. A driving environment where highly accurate self-location estimation is difficult is an "area where environment recognition is difficult."

[0222] 16, a specific example of the difference in the rate of increase of the "estimated self-position error 123" depending on the traveling environment of the robot 100 will be described. FIG. 16 shows the following two examples: (a) An example of an estimated self-position error when traveling in a normal area (= an area where it is not difficult to recognize the environment) (b) An example of an estimated self-position error when traveling in an area where it is difficult to recognize the environment

[0223] 16(a) and 16(b), each robot 100 executes a self-location estimation process. During this self-location estimation process, the circular area shown in the figure, i.e., the "estimated self-location error 123" where the robot's existence probability is approximately 68% is calculated.

[0224] 16, the "estimated self-location error 123b" when traveling in an environment where it is difficult to recognize (b) shown in the lower part is larger than the "estimated self-location error 123a" when traveling in a normal area (= an area where it is not difficult to recognize the environment) shown in the upper part (a). In this way, when the robot 100 travels in a traveling environment where it is difficult to estimate its own location with high accuracy, i.e., in an "area where it is difficult to recognize the environment," the rate of increase in the estimated self-location error becomes high.

[0225] In the calculation process of the estimated self-position error reflecting the difficult-to-recognize area, which is executed in step S13 shown in FIG. 6 and step S105 in FIG. 7, the "estimated self-position error reflecting the difficult-to-recognize area 124" is calculated taking into account whether or not the running environment of the robot 100 includes an "area difficult to recognize the environment."

[0226] That is, in the calculation process of the estimated self-position error reflecting the difficult-to-recognize area, which is performed in step S13 shown in FIG. 6 and step S105 in FIG. 7, the "estimated self-position error 124 reflecting the difficult-to-recognize area" is calculated with an increase rate that is changed depending on not only the travel distance of the robot 100 but also the travel environment.

[0227] 17 is a diagram showing an example (enlarged example) of the change in the self-localization estimation error when the robot 100 travels along a movement route that includes an "environment recognition difficult area." The robot 100 moves from point A to point B. An "environment recognition difficult area" exists between points A and B. The section from P1 to P2 shown in the figure is the "environment recognition difficult area."

[0228] The ellipse shown on the travel path of the robot 100 in Figure 17 represents the "self-localization estimation error," which expands with the travel distance. The expansion rate of this "self-localization estimation error" increases when passing through the "difficult-to-recognize environment area" from P1 to P2.

[0229] The "estimated self-location error 124 reflecting the difficult-to-recognize area" finally calculated at point B is the circular area indicated by the solid line. On the other hand, the circular area indicated by the dotted line at point B is the estimated self-location error that does not take into account the "difficult-to-recognize area." In other words, it is the "estimated self-location error" when it is assumed that there is no "difficult-to-recognize area" on the route from A to B, and this "estimated self-location error" corresponds to the "estimated self-location error 123" calculated in the self-location estimation process in step S12 shown in FIG. 6.

[0230] In the “process for calculating estimated self-position error reflecting difficult-to-recognize area” in step S13 shown in FIG. 6, the “estimated self-position error 123” calculated in the self-position estimation process in step S12 is input, and further, an “area difficult to recognize in the environment” is detected from the “recognition-friendly heat map 121” generated in step S11, and an “estimated self-position error 124 reflecting difficult-to-recognize area” corresponding to the circular area indicated by the solid line at point B in FIG. 16 is calculated.

[0231] A specific example of the calculation process of the "estimated self-location error 124 reflecting the difficult-to-recognize area" will be described with reference to Fig. 18. The circular area indicated by the dotted line around the robot 100 shown in the upper left of Fig. 18 is the estimated self-location error that does not take into account the "difficult-to-recognize area." That is, it is the "estimated self-location error 123" calculated in the self-location estimation process of step S12 shown in Fig. 6. This "estimated self-location error 123" is the value of the "estimated self-location error 123" calculated in the self-location estimation process of step S12 shown in Fig. 6, and this calculated value is used as the variance-covariance value (=Σ ｏｒｉｇｉｎ )

[0232] Note that the variance-covariance value (=Σ ｏｒｉｇｉｎ ) is a value that defines the dotted line area around the robot 100 in the upper left of Fig. 18, and corresponds to the area of ​​the estimated self-location error when it is assumed that the robot 100 does not pass through the "difficult-to-recognize area." For example, in the example of Fig. 17, this corresponds to the estimated self-location error 123b at point B.

[0233] As previously explained with reference to Figures 13 to 15, the region defined by the "estimated self-location error 123" corresponds to the region of the mean value ± standard deviation (σ) of the estimated self-location probability distribution curve. This region increases sequentially according to the movement distance and movement time of the robot 100. However, as described above, the "estimated self-location error 123" may be calculated using the above calculation formula: mean value ± nσ of the estimated self-location probability distribution, where σ is the standard deviation and n is an arbitrary positive number.

[0234] 18 is an estimated self-position error that takes into account the "difficult-to-recognize environment region," and is the "difficult-to-recognize region-reflecting estimated self-position error 124" calculated in step S13 shown in FIG. 6. This "difficult-to-recognize region-reflecting estimated self-position error 124" is a calculated value in the difficult-to-recognize region-reflecting estimated self-position error calculation process in step S13 shown in FIG. 6, and this calculated value is used as a variance-covariance value (=Σ ａｃｔｕａｌ )

[0235] Furthermore, the correction error R, which corresponds to the difference between the "estimated self-position error 124 reflected in the difficult-to-recognize area" shown by the solid line around the robot 100 in the upper left of Figure 18 and the "estimated self-position error 123" shown by the dotted line, is the "increased error R caused by traveling through the difficult-to-recognize area."

[0236] This correction error R (increased error R due to passing through the difficult-to-recognize area) is calculated according to the calculation formula (Formula 1) shown in the lower left of Fig. 18. That is, R = d x Σ ｄｉｓｔ +t ×Σ ｔｉｍｅ ... (Equation 1) It is calculated according to the above equation (Equation 1).

[0237] where d = distance traveled within the recognition-difficult area t = time spent within the recognition-difficult area Σ ｄｉｓｔ = Variance-covariance per unit distance traveled within the recognition-difficult area Σ ｔｉｍｅ = variance-covariance per unit stay time in the difficult-to-recognize area, and Σ ｄｉｓｔ and Σ ｔｉｍｅ = is a user specified parameter.

[0238] The circular area indicated by the solid line around the robot 100 shown in the upper left of FIG. 18, that is, the "difficult-to-recognize area reflected estimated self-position error 124" (=Σ ａｃｔｕａｌ ) is the “estimated self-position error 123” (=Σ ｏｒｉｇｉｎ ) and a correction error R (an increased error R due to traveling through a recognition-difficult area). That is, the "recognition-difficult area reflected estimated self-position error 124" (=Σ ａｃｔｕａｌ ) can be calculated according to the following formula: Σ ａｃｔｕａｌ =Σ ｏｒｉｇｉｎ +R =Σ ｏｒｉｇｉｎ +d×Σ ｄｉｓｔ +t ×Σ ｔｉｍｅ ... (Equation 2) According to the above equation (Equation 2), "the estimated self-position error 124 reflecting the recognition-difficult area" (=Σ ａｃｔｕａｌ ) can be calculated.

[0239] In step S13 of FIG. 6 and step S105 of the flowchart shown in FIG. 7, the "difficult-to-recognize area reflected estimated self-position error 124" is calculated by the above-described process.

[0240] The “estimated self-position error 124 reflecting the recognition-difficult area” (=Σ ａｃｔｕａｌ ) and the "estimated self-position error 123" (=Σ ｏｒｉｇｉｎ The "correction error R (increased error R due to travelling through the difficult-to-recognize area)" corresponding to the difference between the ｄｉｓｔ +t ×Σ ｔｉｍｅ The calculation is performed according to the above formula, and the following parameters are used to calculate the correction error R: ｄｉｓｔ = Variance-covariance per unit distance traveled within the recognition-difficult area Σ ｔｉｍｅ = Variance-covariance per unit stay time in the difficult-to-recognize area These parameters take different values ​​depending on the type of sensor used by the robot 100.

[0241] Therefore, when the robot 100 moves using a plurality of different types of sensors, the formula for calculating the "increased error R due to traveling in a recognition-difficult area" is: R=d×Σｄｉｓｔ +t ×Σ ｔｉｍｅ The two parameters (Σ ｄｉｓｔ , Σ ｔｉｍｅ ) is calculated in advance by setting the parameters of each of the multiple sensors to be used (Σ ｄｉｓｔ , Σ ｔｉｍｅ ) is integrated, it is necessary to use the integrated parameters.

[0242] The parameters of each of the multiple sensors (Σ ｄｉｓｔ , Σ ｔｉｍｅ An example of the integration process will be described with reference to FIG. 19 and subsequent figures.

[0243] As described above with reference to FIG. 12, in the integration process of "recognition-friendly heat maps" corresponding to different sensors, there are two processing modes for integrating "difficult-to-recognize regions."

[0244] 19, like FIG. 12 described above, illustrates an example of the integration process of the following two "difficult-to-recognize regions." (a) Example 1 of integration process of difficult-to-recognize regions corresponding to different sensors (an example of processing in which the shortest possible movement path is prioritized) (b) Example 2 of integration process of difficult-to-recognize regions corresponding to different sensors (an example of processing in which the minimization of positional deviation is prioritized)

[0245] 19(a) is an example in which only the overlapping areas of the difficult-to-recognize areas in the "recognition-convenient heat maps" corresponding to two different sensors are reflected in the integrated "recognition-convenient heat map." In this integration processing example 1, as shown on the right side of FIG. 19(a), the robot 100 passes through a short "difficult-to-recognize area" consisting only of the overlapping areas of the difficult-to-recognize areas in the "recognition-convenient heat maps" corresponding to two different sensors.

[0246] On the other hand, integration process example 2 shown in Figure 19(b) is an example in which all of the difficult-to-recognize regions in each of the "recognition-convenient heat maps" corresponding to two different sensors are reflected in the integrated "recognition-convenient heat map." In this integration process example 2, as shown on the right side of Figure 19(b), the robot 100 passes through a long "difficult-to-recognize region" consisting of all of the difficult-to-recognize regions in each of the "recognition-convenient heat maps" corresponding to two different sensors.

[0247] That is, in the integration processing examples 1 and 2, the travel distance d and the stay time t in the "difficult-to-recognize area" have different values.

[0248] In both of the integration processing examples 1 and 2 shown in FIG. 19, the parameters (Σ ｄｉｓｔ , Σ ｔｉｍｅ However, as described above, in integration processing examples 1 and 2, the travel distance d and stay time t of the "difficult-to-recognize area" are different values, and therefore the calculated results are different.

[0249] Referring to FIG. 20, the parameters (Σ ｄｉｓｔ , Σ ｔｉｍｅ The details of the calculation of the integration process of the i-th sensor-compatible recognition-difficult area before integration are described below. ｄｉｓｔｉ Variance-covariance of the difficult-to-recognize area per unit distance traveled after integration: Σ ｄｉｓｔＵｎｉｔｅｄ Let's say.

[0250] According to the following formula, the variance-covariance per unit distance traveled for each sensor-compatible recognition-difficult area before integration: Σ ｄｉｓｔｉ The sum of these values ​​is used to calculate the variance-covariance per unit distance traveled in the unrecognizable area after integration: Σ ｄｉｓｔＵｎｉｔｅｄ It is calculated as Σ ｄｉｓｔＵｎｉｔｅｄ = SUM (Σ ｄｉｓｔｉ ) where sum() is a function that represents summation, and SUM(Σ ｄｉｓｔｉ ) is Σ for i = 1 to N ｄｉｓｔｉ This is a function that calculates the sum of

[0251] Variance-covariance per unit time spent in the difficult-to-recognize area after integration: Σ ｔｉｍｅＵｎｉｔｅｄ is calculated in the same way. That is, the variance-covariance per unit stay time of each sensor-compatible recognition-difficult area before integration is calculated according to the following formula: Σ ｔｉｍｅｉ The sum of these values ​​is used to calculate the variance-covariance per unit time spent in the difficult-to-recognize area after integration: Σ ｔｉｍｅＵｎｉｔｅｄ It is calculated as Σ ｔｉｍｅＵｎｉｔｅｄ = SUM (Σ ｔｉｍｅｉ ) where sum() is a function that represents summation, and SUM(Σ ｔｉｍｅｉ) is Σ for i = 1 to N ｔｉｍｅｉ This is a function that calculates the sum of

[0252] By this calculation process, the parameters of each of the multiple sensors (Σ ｄｉｓｔ , Σ ｔｉｍｅ The integration result is then used to calculate the correction error R (the increased error R due to travelling through the difficult-to-recognize area) as explained above, i.e., R = d × Σ ｄｉｓｔ +t ×Σ ｔｉｍｅ ...(Equation 1) Apply to the above equation (Equation 1).

[0253] Applying the integrated result to the calculation formula for the correction error R (the increased error R due to passing through the difficult-to-recognize area) results in the following formula (Formula 3): R = d × Σ ｄｉｓｔＵｎｉｔｅｄ +t ×Σ ｔｉｍｅＵｎｉｔｅｄ (Equation 3) Using the above equation (Equation 3), it is possible to calculate the correction error R (increased error R resulting from traveling through a recognition-difficult area) when multiple sensors are used.

[0254] However, as described above, the travel distance d and the stay time t of the "difficult-to-recognize area" are different between integration processing examples 1 and 2, and therefore the calculated results are different. ｄｉｓｔ , Σ ｔｉｍｅ ) shows an example of the application area of ​​the integration processing results.

[0255] In "(a) Integration Process Example 1 of Difficult-to-Recognize Areas Corresponding to Different Sensors (Processing Example with Priority on Shortening of Travel Path)" in Figure 21, only the overlapping area of ​​the difficult-to-recognize areas in the "Heat Map for Convenience of Recognition" corresponding to two different sensors becomes the difficult-to-recognize area of ​​the integrated "Heat Map for Convenience of Recognition." In this case, the correction error R (increased error R due to traveling through the difficult-to-recognize area) when passing through a short "Difficult-to-Recognize area" consisting only of the overlapping area of ​​the difficult-to-recognize areas corresponding to each sensor is calculated using the above formula (Formula 3).

[0256] On the other hand, in "(b) Example 2 of Integration Processing of Difficult-to-Recognize Areas Corresponding to Different Sensors (Example of Processing Prioritizing Minimization of Positional Deviation)" in Figure 21, all of the difficult-to-recognize areas in each of the "heat maps for recognition" corresponding to two different sensors are integrated to form the difficult-to-recognize area of ​​the "heat map for recognition." In this case, the correction error R (increased error R due to traveling through the difficult-to-recognize area) when passing through a long "difficult-to-recognize area" consisting of all of the difficult-to-recognize areas corresponding to each sensor is calculated using the above formula (Formula 3).

[0257] 21(a) is an example in which the "difficult-to-recognize region" after integration is minimized, and by minimizing the "difficult-to-recognize region," a mobile body such as a robot can move over a wider area. In other words, when setting a movement route that avoids the "difficult-to-recognize region," there is an advantage in that the degree of freedom in setting the route is increased.

[0258] On the other hand, the example shown in Figure 21 (b) is an example in which the "difficult-to-recognize areas" in the integrated "recognition-friendly heat map" are maximized. By setting a route that avoids the "difficult-to-recognize areas" as much as possible, there is an advantage in that a route can be set that consists only of areas where the self-location estimation process can be performed reliably with high accuracy.

[0259] (4-(4) Optimal Route Generation Process (Step S14 in FIG. 6, Step S106 in FIG. 7)) Next, the optimal route generation process will be described in detail.

[0260] The optimum route generation process is executed in step S14 of FIG. 6 and step S106 of the flowchart shown in FIG.

[0261] For example, in step S14 of FIG. 6, the data processing unit 103 of the robot 100 shown in FIG. 3 executes a process of planning the movement path of the robot 100 using the target position (G) input via the input unit 106, the “recognition-friendly heat map 121” generated in step S11, and the “estimated self-position error 124 reflecting the difficult-to-recognize area” generated in step S13.

[0262] Specifically, the optimal path is generated as a path that allows the robot 100 to safely reach the target position (G) without colliding with obstacles such as walls along the path. For example, if an "environment recognition-difficult area" exists on the shortest path from the current position of the robot 100 to the target position (G), and the "recognition-difficult area-reflecting estimated self-position error 124" becomes large, it is predicted that the robot 100 will collide with an obstacle such as a wall along the path.

[0263] When such a prediction is made, a route other than the shortest route from the current position of the robot 100 to the target position (G) is planned. For example, a new optimal route is generated with landmarks, whose position information is available and set in the "recognition-convenience heat map 121," as waypoints.

[0264] A specific example of the optimum path generation process will be described with reference to Fig. 22. Fig. 22 is a diagram showing an example of the process of generating an optimum path 190 for the robot 100 to reach the target position (G) 150.

[0265] First, a target position (G) 150 is set. The target position (G) 150 is set by a user input. For example, the data processing unit 103 of the robot 100 displays a recognition-specific map or a map used to generate the recognition-specific map on the display unit 105, and the user sets the target position (G) 150 at an arbitrary position on the displayed map.

[0266] Next, the shortest path 160 from the current position of the robot 100 to the target position (G) 150 is set. However, in the example shown in Fig. 22, the shortest path 160 includes multiple difficult-to-recognize environments 171, 172. These difficult-to-recognize environments 171, 172 are areas recorded in the recognition-convenient heat map.

[0267] Next, the data processing unit of the moving object path analysis device disclosed herein analyzes whether the robot 100 will come into contact with or collide with an obstacle such as a wall when moving toward the target position (G) using the shortest path 160. This analysis process is performed by setting the "estimated self-location error reflecting the difficult-to-recognize region" described above with reference to FIG. 17 and other figures on the shortest path 160. As described above with reference to FIG. 17 and other figures, the "estimated self-location error reflecting the difficult-to-recognize region" is represented as a circular region that continues on the path of movement of the robot 100, and gradually expands as the robot 100 moves.

[0268] The data processing unit of the mobile object path analysis device disclosed herein analyzes whether or not an obstacle overlaps with a circular area indicating an "estimated self-position error reflecting a difficult-to-recognize area" set on the shortest path 160. That is, if the circular area is set to contact or go beyond a wall or the like, it is determined that the robot 100 will contact or collide with an obstacle such as a wall if it moves along this shortest path 160 toward the target position (G). If such a determination is made, the shortest path 160 shown in FIG. 22 is no longer used as a movement path, and another path is considered.

[0269] During this route planning process, the data processing unit of the mobile object route analysis device of the present disclosure considers a route that has landmarks recorded in the recognition heat map set as waypoints. For example, as shown in Figure 22, the data processing unit analyzes whether a route that has landmark a, 181, and landmark b, 182 set as waypoints is a safe route.

[0270] The data processing unit of the mobile object path analysis device of the present disclosure sets a circular area indicating the "estimated self-position error reflecting the difficult-to-recognize area" on a path set with landmark a 181 and landmark b 182 as waypoints, and analyzes whether the set circular area overlaps with an obstacle. In other words, it determines whether the set circular area is set to come into contact with or go beyond a wall or the like on the path.

[0271] If it is determined that the set circular area does not overlap any obstacles on the route and does not come into contact with or go beyond a wall or the like, it is determined that the robot 100 can safely reach the target position 150 by moving along this route, and this route is set as the optimal route 190. The data processing unit of the mobile object route analysis device of the present disclosure performs, for example, such processing to determine the optimal route 190.

[0272] Next, a detailed sequence of the optimal route generation process executed by the data processing unit of the mobile object route analysis device of the present disclosure will be described with reference to the flowcharts shown in Figures 23 and 24. The process of each step of the flowcharts shown in Figures 23 and 24 will be described in order below.

[0273] (Step S201) First, in step S201, the data processing unit of the moving object route analysis device of the present disclosure executes a process of defining an obstacle.

[0274] Specifically, the decision is made as to which of the above definitions (a) or (b) should be used: (a) Only real obstacles such as walls and pillars are considered to be obstacles; or (b) In addition to real obstacles such as walls and pillars, "difficult-to-recognize areas" recorded in the recognition heat map are also considered to be obstacles.

[0275] If the definition is "(a) only real obstacles such as walls and pillars are considered to be obstacles," then the "difficult-to-recognize areas" recorded in the heat map for recognition are interpreted as areas in which the robot 100 can move, and path planning is performed accordingly. On the other hand, if the definition is "(b) in addition to real obstacles such as walls and pillars, the "difficult-to-recognize areas" recorded in the heat map for recognition are also considered to be obstacles," then the "difficult-to-recognize areas" recorded in the heat map for recognition are interpreted as areas in which the robot 100 cannot move, and path planning is performed accordingly.

[0276] (Step S202) Next, in step S202, the data processing unit of the moving object route analysis device inputs the setting information of the target position and generates a route that can reach the target position. As described above, the target position is input by, for example, the user.

[0277] In step S202, the data processing unit of the moving object route analysis device first generates the shortest route that avoids the obstacles defined in step S201 and that can reach the target position.

[0278] (Step S203) Next, in step S203, the data processing unit of the moving object path analysis device analyzes the "estimated self-position error" or the "estimated self-position error reflecting the difficult-to-recognize area" when the robot 100 moves along the path generated in step S202.

[0279] Specifically, based on the "estimated self-position error" or the "estimated self-position error reflecting the difficult-to-recognize area" when the robot 100 moves, it is determined whether the robot 100 will come into contact with or collide with an obstacle.

[0280] The "estimated self-location error" to be analyzed here is either the "estimated self-location error 123" generated in the self-location estimation process executed in step S12 shown in FIG. 6, or the "estimated self-location error 124 reflected in the difficult-to-recognize area" generated in the estimated self-location error calculation process reflected in the difficult-to-recognize area executed in step S13 shown in FIG. 6.

[0281] In step S201, if it is defined that "(a) only real obstacles such as walls and pillars are considered as obstacles", there is a possibility that the route generated in step S202 will include a "difficult-to-recognize area", so analysis processing is performed using the "estimated self-position error 124 reflecting difficult-to-recognize area" generated in step S13 shown in Figure 6.

[0282] On the other hand, if it is defined in step S201 that "(b) in addition to real obstacles such as walls and pillars, the 'difficult-to-recognize areas' recorded in the recognition heat map are also considered to be obstacles," the route generated in step S202 does not include the 'difficult-to-recognize areas', and therefore analysis processing is performed using the 'estimated self-position error 123' generated in step S12 shown in Figure 6.

[0283] These "estimated self-position error 124 reflecting difficult-to-recognize area" and "estimated self-position error 123" are data with different magnification rates, such as the "estimated self-position error 124 reflecting difficult-to-recognize area" and "estimated self-position error 123b" previously described with reference to Figure 17.

[0284] A specific example of analysis processing using the "difficult-to-recognize area reflected estimated self-position error 124" will be described with reference to FIGS.

[0285] 25 shows an example of a recognition-convenience map 200 displayed on the display unit for the user to set the target position (G) in step S202. The recognition-convenience map 200 shows the starting position (S) of the robot 100, as well as an environmental recognition-difficult area 201 and landmarks 202.

[0286] The user sets a target position (G) at any position on this recognition-convenience map 200. Fig. 26 shows the processing of step S202, i.e., a state in which the shortest route 210 that can reach the target position (G) 203 set by the user is set. The shortest route 210 includes an environment recognition difficulty area 201.

[0287] In step S203, the data processing unit of the moving object route analysis device executes an analysis process of the "estimated self-position error 124 reflected in the difficult-to-recognize area" on the shortest route 210 that includes the difficult-to-recognize area 201.

[0288] A specific example of the analysis process is shown in Fig. 27. The series of circular areas on the shortest route 210 shown in Fig. 27 is the "estimated self-location error 124 reflecting the difficult-to-recognize area" predicted to occur when the robot 100 moves on the shortest route 210. This "estimated self-location error 124 reflecting the difficult-to-recognize area" is calculated according to the calculation formula previously described with reference to Fig. 18.

[0289] In this way, in step S203, the data processing unit of the mobile object path analysis device generates, for example, a series of circular areas on the shortest path 210 shown in Figure 27 as transition data of the ``estimated self-position error'' or the ``estimated self-position error reflecting the difficult-to-recognize area'' when the robot 100 moves along the path generated in step S202.

[0290] (Step S204) Next, in step S204, the data processing unit of the moving object path analysis device determines the possibility of a collision between the robot 100 and an obstacle based on the "estimated self-position error" or the "estimated self-position error reflecting the difficult-to-recognize area."

[0291] In step S204, the data processing unit of the mobile object path analysis device determines whether the "estimated self-position error 124 reflecting difficult-to-recognize areas" shown in FIG. 27, i.e., whether the row of circular areas on the shortest path 210 shown in FIG. 27 overlaps with an obstacle, i.e., whether the area is large enough to contact or exceed the wall that is the obstacle.

[0292] If the row of circular areas on the shortest path 210 shown in FIG. 27 is in contact with or exceeds the size of an obstacle wall, it means that if the robot 100 moves along the shortest path 210 shown in FIG. 27, there is a possibility that it will come into contact with or collide with the obstacle wall.

[0293] In the example shown in Fig. 27, a circular area corresponding to the "estimated self-position error 124 reflecting the difficult-to-recognize area" near the right end of the difficult-to-recognize area 201 is in contact with or exceeds the wall, which is an obstacle. In other words, there is a "possible point 221 of contact or collision with an obstacle." This means that if the robot 100 moves along the shortest path 210 shown in Fig. 27, there is a possibility that it will come into contact with or collide with the wall, which is an obstacle.

[0294] (Step S205) Next, in step S205, the data processing unit of the moving object path analysis device determines whether or not the analyzed path is a path that is free from the possibility of the robot 100 coming into contact with or colliding with an obstacle.

[0295] As shown in Figure 27, when the ``estimated self-position error 124 reflecting the difficult-to-recognize area'', i.e., when the circular area is in contact with or exceeds the size of the wall that is an obstacle, it is determined that there is a possibility of contact or collision with the obstacle.

[0296] If the determination in step S205 is Yes, that is, if it is determined that the analyzed route is a route that is unlikely to cause the robot 100 to come into contact with or collide with an obstacle, the process proceeds to step S221. On the other hand, if the determination in step S205 is No, that is, if it is determined that the analyzed route is a route that is likely to cause the robot 100 to come into contact with or collide with an obstacle, the process proceeds to step S206.

[0297] (Step S206) If the determination in step S205 is No, that is, if it is determined that the analyzed route is one in which the robot 100 may come into contact with or collide with an obstacle during its movement, the process proceeds to step S206.

[0298] In this case, the data processing unit of the mobile object route analysis device determines in step S206 whether the number of route generation times has reached a predetermined upper limit. If it is determined that the number of route generation times has reached the predetermined upper limit, the process proceeds to step S231. On the other hand, if the number of route generation times has not reached the predetermined upper limit, the process proceeds to step S207.

[0299] (Step S207) If the determination in step S206 is No, that is, if the number of times the route has been generated has not reached the predetermined upper limit, the process proceeds to step S207.

[0300] In this case, in step S207, the data processing unit of the moving object route analysis device searches for landmarks that can be set as waypoints.

[0301] The data processing unit of the mobile object path analysis device refers to the recognition heat map to search for landmarks that can be set as waypoints on the path from the robot's 100 start position (S) to its target position (G).

[0302] When there are multiple landmark candidates that can be set as waypoints, the landmark that allows the route length from the start position (S) to the target position (G) to be set to be short is selected preferentially.

[0303] (Step S208) In step S208, the data processing unit of the moving object route analysis device determines whether or not a landmark that can be set as a waypoint has been detected in step S207.

[0304] If a landmark that can be set as a waypoint cannot be detected, the process proceeds to step S231. On the other hand, if a landmark that can be set as a waypoint can be detected, the process proceeds to step S209.

[0305] (Step S209) In step S208, if a landmark that can be set as a waypoint on the path from the start position (S) of the robot 100 to the target position (G) is detected, the process of step S209 is executed.

[0306] In this case, the data processing unit of the moving object route analysis device generates a new route in which the landmarks are set as waypoints in step S209.

[0307] A specific example of a process for generating a new route in which a landmark is set as a via point (waypoint) will be described with reference to Fig. 28. As shown in Fig. 28, a new route 230 is generated in which a landmark 202 is set as a via point (waypoint) on the route from the start position (S) of the robot 100 to the target position (G).

[0308] Next, the process from step S203 onward is executed for this new path. That is, in step S203, the "estimated self-position error" or the "estimated self-position error reflecting the difficult-to-recognize area" is analyzed when the robot 100 moves along the new path generated in step S209.

[0309] Furthermore, in step S204, the possibility of collision between the robot 100 and an obstacle is determined based on the "estimated self-position error" or the "estimated self-position error reflecting the difficult-to-recognize area."

[0310] A specific example of this processing will be described with reference to Fig. 29. The data processing unit of the mobile object route analysis device sets an "estimated self-location error 124 reflecting difficult-to-recognize areas," i.e., a string of circular areas shown in Fig. 29, on a new route 230 that has landmarks 202 set as waypoints, as shown in Fig. 29. This "estimated self-location error 124 reflecting difficult-to-recognize areas" is calculated according to the calculation formula previously described with reference to Fig. 18.

[0311] The data processing unit of the mobile object route analysis device determines whether the "estimated self-position error 124 reflecting the difficult-to-recognize area" on the new route 230, i.e., the circular area shown in Figure 29, is an area of ​​a size that touches or exceeds the wall, which is an obstacle.

[0312] 29, the circular area is not large enough to contact or exceed the size of an obstacle such as a wall. This means that when the robot 100 moves along the new path 230 that has the landmark 202 set as a waypoint, it can reach the target position (G) 203 without contacting or colliding with an obstacle such as a wall.

[0313] That is, the determination in step S205 for the new route 230 is Yes, and the process can proceed to step S221. The process from step S221 onwards will be described with reference to the flowchart on the left side of FIG.

[0314] (Step S221) In step S221, the data processing unit of the moving object route analysis device selects the shortest route as the optimum route from route candidates that do not come into contact with or collide with obstacles.

[0315] In the above example, there is only one route candidate, the new route 130 shown in Fig. 29, so this new route 130 is selected as the optimum route. As shown in Fig. 29, the new route 230 with the landmark 202 set as a waypoint is set as the optimum route 240.

[0316] (Step S222) In step S222, if there is a landmark on the selected optimal route, the data processing unit of the mobile object route analysis device executes a route correction process (follow route addition process) to enable acquisition of location information at the landmark on the route.

[0317] This is a process in which robot 100 acquires location information associated with a landmark at the landmark's location and adds a follow route, which is a driving route for correcting errors in the estimated self-position that occur in the self-position estimation process executed by robot 100, i.e., a route for the self-position correction process.

[0318] An example of the process of adding a follow path will be described with reference to Fig. 30 . For example, as shown in Fig. 30 , a follow path 241 that circles around the landmark 202 is added. By traveling along the follow path 241 that circles the landmark 202 in this way, the robot 100 acquires position information associated with the landmark 202 and performs a process of correcting an error in the estimated self-position that occurs in the self-position estimation process of the robot 100. Note that this follow path 241 generation process is executed by the mobile object path analysis device of the present disclosure, for example, a data processing unit of the robot 100. Alternatively, the follow path may be set by inputting it by the user via a display unit (UI).

[0319] (Steps S223 to S224) Next, in steps S223 to S224, the data processing unit of the moving object route analysis device generates display data for output to the display unit. For example, it generates an image including an optimal route, route candidates, and an environment recognition difficult area, and outputs it to the display unit.

[0320] The display image generation process and output process will be described in detail in the next section.

[0321] Next, we will explain the processing of steps S231 to S233, which is the flow on the right side of Fig. 24. The processing of steps S231 to S233 is executed when a Yes determination is made in step S206 of the flow shown in Fig. 23, and when a No determination is made in step S208.

[0322] In the flow shown in Figure 23, a "Yes" determination is made in step S206 when it is determined that the number of route generation attempts has reached a predetermined upper limit. Furthermore, a "No" determination is made in step S208 when a landmark that can be set as a waypoint has not been detected. In these cases, the processes of steps S231 to S233 are executed.

[0323] The following describes the processing of steps S231 to S233 in the flow on the right side of Figure 24. (Step S231) If it is determined in step S206 that the number of times a route has been generated has reached a predetermined upper limit, or if no landmark that can be set as a waypoint has been detected in step S208, the processing of step S231 is executed.

[0324] In either of these cases, in step S231, the data processing unit of the moving object route analysis device generates an error message to the display unit to the effect that a safe route that does not come into contact with or collide with an obstacle cannot be detected.

[0325] (Steps S232 to S233) Next, in steps S232 to S233, the data processing unit of the moving object route analysis device generates display data for output to the display unit. For example, it generates an image including an optimal route, route candidates, and an environment recognition difficult area, and outputs it to the display unit.

[0326] The display image generation process and output process will be described in detail in the next section.

[0327] As described above, when generating a path to a target position (G), the movement control device of the robot 100 or the like according to the present disclosure analyzes various paths based on the "estimated self-position error" or the "estimated self-position error reflecting the difficult-to-recognize area" to determine whether the robot 100 will come into contact with or collide with an obstacle, and executes a process of selecting, as the optimal path, a path that will not cause the robot 100 to come into contact with or collide with an obstacle. This process enables the robot 100 to move safely to the target position (G) without coming into contact with or colliding with an obstacle.

[0328] Furthermore, when generating the optimal route, a route is generated using landmarks whose location information can be obtained as waypoints, thereby eliminating errors that occur and accumulate in the robot's self-position estimation process. This enables the robot to move safely to the target position (G) without accumulating excessive errors.

[0329] (4-(5) Display Information Generation Process (Step S16 in FIG. 6, Step S108 in FIG. 7)) Next, the display information generation process will be described in detail.

[0330] The display information generation process is executed in step S16 in FIG. 6 and step S108 in the flowchart shown in FIG.

[0331] The data processing unit 103 of the mobile object path analysis device of the present disclosure, for example, the robot 100 shown in FIG. 3 , receives the following information and generates display information 128 to be output to the display unit 105 based on the received information: (a) a map stored in the storage unit 104; (b) a target position (G) input by the user via the input unit 106; (c) a "recognition-friendly heat map 121" generated in step S11; (d) an "estimated self-position 122" generated in step S12; (e) an "estimated self-position error 124 reflecting difficult-to-recognize areas" generated in step S13; (f) an "optimal path 125" generated in step S14; and (g) "route supplementary information 126" generated in step S14.

[0332] In step S16, the data processing unit 103 generates display information 128 to be output to the display unit 105 using the above information (a) to (g).

[0333] The display information generated by the moving object path analysis device of the present disclosure, for example, the data processing unit 103 of the robot 100 shown in FIG. 3, includes various types of display information.

[0334] Specifically, information combining the above data (a) to (g) is displayed on the display unit 105 at various times.

[0335] An example of data displayed on the display unit 105 will be described with reference to Fig. 31 and subsequent figures. The display data shown in Fig. 31 is an example of display data that displays the robot 100, an environment recognition difficulty area 201, landmarks 202, a target position (G) 203, and a shortest route 210 on a recognition-convenient heat map 200.

[0336] By looking at this display data, the user can check the positional relationship between the robot 100 and the target position (G) 203, the setting status of the shortest route 210, and the difficult-to-recognize environment area 201 and the position of the landmarks 202.

[0337] The display data shown in Figure 32 further displays a row of circular areas indicating the ``estimated self-position error 124 reflecting areas difficult to recognize'' on the shortest path 210 connecting the robot 100 and the target position (G) 203, and outputs a warning message saying ``Collision risk'' in areas where there is a high possibility of contact with an obstacle such as a wall.

[0338] By looking at this display data, the user can recognize that if the robot 100 travels along the shortest path 210 to the target position (G) 203, there is an area where there is a high possibility of contact with an obstacle such as a wall.

[0339] 33 is an example of display data that further displays an optimal route 240 connecting the robot 100 and the target position (G) 203. The optimal route 240 is a route that passes through landmarks 202, and a series of circular areas indicating the "estimated self-position error 124 reflecting difficult-to-recognize areas" is displayed on this optimal route 240. In this example, the series of circular areas does not come into contact with walls, which are obstacles, and the user can understand that the robot 100 can safely reach the target position (G) 202 by traveling along the optimal route 240.

[0340] The user can view the data displayed on the display unit and input various control information, such as route correction, stopping the robot 100, and starting the robot 100, and the data processing unit of the robot 100 controls the robot 100 in accordance with the user input.

[0341] 5. Example of Hardware Configuration of a Moving Object Path Analysis Device According to the Present Disclosure Next, an example of a hardware configuration of a moving object path analysis device according to the present disclosure will be described with reference to FIG.

[0342] Each element of the hardware configuration shown in Fig. 34 will be described below. A CPU (Central Processing Unit) 501 functions as a data processing unit that executes various processes according to programs stored in a ROM (Read Only Memory) 502 or a storage unit 508. For example, it executes processes according to the sequences described in the above-mentioned embodiments. A RAM (Random Access Memory) 503 stores programs and data executed by the CPU 501. The CPU 501, ROM 502, and RAM 503 are interconnected by a bus 504.

[0343] The CPU 501 is connected to an input / output interface 505 via a bus 504, and the input / output interface 505 is connected to an input unit 506 consisting of various sensors, a camera, a switch, a keyboard, a mouse, a microphone, etc., and an output unit 507 consisting of a display, a speaker, etc.

[0344] The storage unit 508 connected to the input / output interface 505 is composed of, for example, a USB memory, an SD card, a hard disk, etc., and stores various data and programs executed by the CPU 501. The communication unit 509 functions as a transmission / reception unit for data communication via a network such as the Internet or a local area network, and communicates with external devices.

[0345] A drive 510 connected to the input / output interface 505 drives a removable medium 511 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory such as a memory card, and executes recording or reading of data.

[0346] [6. Summary of the Configuration of the Present Disclosure] The embodiments of the present disclosure have been described above in detail with reference to specific examples. However, it is obvious that those skilled in the art can modify or substitute the embodiments without departing from the gist of the present disclosure. In other words, the present invention has been disclosed in the form of examples and should not be interpreted as being limited. To determine the gist of the present disclosure, the claims should be taken into consideration.

[0347] The technology disclosed in this specification can be configured as follows: (1) A mobile object path analysis method in which a data processing unit of a mobile object path analysis device calculates an estimated self-position error that includes a predetermined range of error in the estimated self-position calculated in the self-position estimation process of the mobile object, analyzes whether or not the calculated estimated self-position error area overlaps with an obstacle, and determines that a path without overlap is a path that the mobile object can travel without coming into contact with or colliding with an obstacle.

[0348] (2) The mobile body path analysis method according to (1), wherein the data processing unit extracts a probability distribution area of ​​a predetermined area from an estimated self-position probability distribution generated in a self-position estimation process of the mobile body, and calculates the estimated self-position error.

[0349] (3) The mobile body path analysis method according to (1) or (2), wherein the data processing unit calculates the estimated self-location error by extracting an area defined by the mean value of the estimated self-location probability distribution ±nσ, where σ is the standard deviation and n is a positive number, from the estimated self-location probability distribution generated in the mobile body's self-location estimation process.

[0350] (4) The mobile body path analysis method according to any one of (1) to (3), wherein the data processing unit calculates the estimated self-location error by setting the area of ​​the estimated self-location error to gradually expand in accordance with the movement of the mobile body.

[0351] (5) The data processing unit sets the area of ​​the estimated self-position error on the moving path to the target position of the moving body, analyzes whether or not the areas of the estimated self-position error set on the moving path overlap with obstacles, and determines that a path without overlap is a path along which the moving body can move without coming into contact with or colliding with obstacles. (4) The method for analyzing a moving body path

[0352] (6) The data processing unit calculates an estimated self-location error that reflects a difficult-to-recognize area, in which the rate of increase in the estimated self-location error that occurs in an area where it is difficult to recognize the environment and where high-precision self-location estimation processing is difficult is set higher than the rate of increase in the estimated self-location error in a normal area other than the difficult-to-recognize area; analyzes whether the area of ​​the calculated estimated self-location error that reflects the difficult-to-recognize area overlaps with an obstacle; and determines that a route without overlap is a route that the moving body can travel without coming into contact with or colliding with an obstacle. This is a method of analyzing a moving body path described in any of (1) to (5).

[0353] (7) The data processing unit performs a calculation process to add a predetermined correction error R to the estimated self-position error in a normal area other than an environment recognition difficult area where high-precision self-position estimation processing is difficult, thereby calculating the estimated self-position error reflected in the recognition difficult area. (6) The mobile body path analysis method described in (6).

[0354] (8) The method for analyzing a moving body path according to (6) or (7), wherein the data processing unit calculates an estimated self-position error reflecting a difficult-to-recognize area, in which the area of ​​the estimated self-position error reflecting a difficult-to-recognize area is set to gradually expand in accordance with the movement of the moving body in the difficult-to-recognize environment area.

[0355] (9) A method for analyzing a path of a moving body according to any one of (6) to (8), wherein the difficult-to-recognize area is an area that differs depending on the type of sensor used by the moving body for environment recognition, and the data processing unit calculates the estimated self-position error reflecting the difficult-to-recognize area based on the difficult-to-recognize area that is determined depending on the type of sensor used by the moving body for environment recognition.

[0356] (10) The mobile body path analysis method described in any one of (6) to (9), wherein the data processing unit determines a path that does not overlap with an area of ​​the estimated self-position error that reflects the difficult-to-recognize area as the optimal path for the mobile body, and generates drive control information for moving the mobile body along the optimal path.

[0357] (11) The mobile object path analysis method according to any one of (6) to (10), wherein the data processing unit calculates the estimated self-position error reflecting the difficult-to-recognize area by referring to a recognition-related heat map that can identify the difficult-to-recognize area.

[0358] (12) A mobile object path analysis method according to any one of (6) to (11), wherein the data processing unit generates a recognition-related heat map capable of identifying the difficult-to-recognize environment area using a map having attribute information set therein that can determine whether or not high-precision self-position estimation processing is difficult.

[0359] (13) The method for analyzing a path of a moving body according to (12), wherein the difficult-to-recognize environment area is an area that differs depending on the type of sensor used by the moving body for environment recognition, and the data processing unit generates a recognition-convenience heat map that makes it possible to identify the difficult-to-recognize environment area corresponding to each type of sensor used by the moving body for environment recognition.

[0360] (14) The mobile body path analysis method according to any one of (6) to (13), wherein the data processing unit generates a path as the moving path of the mobile body, with landmarks, which are objects or areas for which location information can be acquired, set as intermediate points.

[0361] (15) The method for analyzing a moving body path described in (14), wherein the data processing unit analyzes whether or not the area of ​​the estimated self-position error reflecting the difficult-to-recognize area overlaps with an obstacle for a route that has the landmark set as a waypoint, and if it is confirmed that the route does not overlap, determines that the route that has the landmark set as a waypoint is a route that the moving body can travel on without coming into contact with or colliding with an obstacle.

[0362] (16) The mobile object path analysis method according to any one of (6) to (15), wherein the data processing unit generates a recognition-friendly heat map that makes it possible to identify the difficult-to-recognize environment area and landmarks, which are objects or areas for which location information can be acquired.

[0363] (17) The mobile object path analysis method according to (16), wherein the data processing unit generates the recognition heat map recording landmarks, which are objects or areas for which position information can be acquired, using a map in which position information is set as attribute information for each object or area.

[0364] (18) The mobile object path analysis method according to any one of (6) to (17), wherein the data processing unit generates display data that makes it possible to identify at least one of the difficult-to-recognize area and the area of ​​the estimated self-position error reflecting the difficult-to-recognize area, and whether or not the overlap with an obstacle is present.

[0365] (19) A mobile body path analysis device having a data processing unit that calculates an estimated self-position error that includes a predetermined range of error in the estimated self-position calculated in the mobile body's self-position estimation process, analyzes whether the calculated estimated self-position error area overlaps with an obstacle, and determines that a route without overlap is a route that the mobile body can travel without coming into contact with or colliding with an obstacle.

[0366] (20) A program for causing a moving body path analysis device to execute a moving body path analysis process, the program causing a data processing unit of the moving body path analysis device to execute the following processes: a process for calculating an estimated self-position error that includes a predetermined range of error in the estimated self-position calculated in the moving body's self-position estimation process; a process for analyzing whether or not the calculated area of ​​estimated self-position error overlaps with an obstacle; and a process for determining that a route without overlap is a route that the moving body can travel on without coming into contact with or colliding with an obstacle.

[0367] Furthermore, the series of processes described in this specification can be executed by hardware, software, or a combination of both. When executing processes by software, a program recording the processing sequence can be installed and executed in the memory of a computer incorporated in dedicated hardware, or the program can be installed and executed on a general-purpose computer capable of executing various processes. For example, the program can be pre-recorded on a recording medium. In addition to installing the program from the recording medium to the computer, the program can also be received via a network such as a LAN (Local Area Network) or the Internet and installed on a recording medium such as an internal hard disk.

[0368] The various processes described in this specification may not only be executed in chronological order as described, but may also be executed in parallel or individually depending on the processing capabilities of the devices executing the processes or as needed. Furthermore, in this specification, a system refers to a logical collective configuration of multiple devices, and is not limited to devices that are all located in the same housing.

[0369] As described above, according to one embodiment of the present disclosure, an apparatus and method for generating a path that allows a moving body to safely reach a target position without coming into contact with an obstacle are realized. Specifically, for example, an estimated self-position error is calculated by adding a predetermined error to the estimated self-position calculated by the moving body's self-position estimation process. Furthermore, an estimated self-position error that reflects a difficult-to-recognize area is calculated by setting the increase rate of the estimated self-position error that occurs in an environment recognition difficult area higher than the increase rate of the estimated self-position error in areas other than the environment recognition difficult area. The apparatus analyzes whether the calculated estimated self-position error reflecting a difficult-to-recognize area overlaps with an obstacle, and determines that a route without overlap is a route that the moving body can travel without coming into contact with an obstacle. This configuration realizes an apparatus and method for generating a path that allows a moving body to safely reach a target position without coming into contact with an obstacle.

[0370] 20 Shortest route 21 Difficult-to-recognize environmental area 22 Travel route 30 Optimal route 31 Landmark 100 Robot (mobile body) 101 Sensor 102 Driving unit 103 Data processing unit (processor) 104 Storage unit (memory) 105 Display unit (monitor) 106 Input unit (UI) 107 Communication unit 110 Information processing device 111 Display unit (monitor) 112 Input unit (UI) 113 Communication unit 115 Data processing unit 116 Storage unit (memory) 120 Information processing device 121 Recognition-related heat map 122 Estimated self-position 123 Estimated self-position error 124 Estimated self-position error reflecting difficult-to-recognize area 125 Optimal route 126 Route supplementary information 127 Driving control information 150 Target position (G) 160 Shortest route 171, 172 Difficult-to-recognize environmental area 181, 182 Landmark 190 Optimal route 200 Recognition-friendly heat map 201 Difficult-to-recognize environmental area 202 Landmark 210 Shortest route 230 New route 240 Optimal route 240 Follow route 501 CPU 502 ROM 503 RAM 504 Bus 505 Input / output interface 506 Input unit 507 Output unit 508 Storage unit 509 Communication unit 510 Drive 511 Removable media

Claims

1. A mobile object path analysis method in which a data processing unit of a mobile object path analysis device calculates an estimated self-position error that includes a specified range of error in the estimated self-position calculated in the mobile object's self-position estimation process, analyzes whether the calculated estimated self-position error area overlaps with an obstacle, and determines that a route without overlap is a route that the mobile object can travel without coming into contact with or colliding with an obstacle.

2. A mobile body path analysis method as described in claim 1, wherein the data processing unit extracts a probability distribution area of ​​a predetermined region from the estimated self-position probability distribution generated in the mobile body's self-position estimation process, and calculates the estimated self-position error.

3. The mobile body path analysis method according to claim 1, wherein the data processing unit calculates the estimated self-position error by extracting an area defined by the mean value of the estimated self-position probability distribution ±nσ, where σ is the standard deviation and n is a positive number, from the estimated self-position probability distribution generated in the mobile body's self-position estimation process.

4. A method for analyzing a moving body path according to claim 1, wherein the data processing unit calculates an estimated self-position error by setting the area of ​​the estimated self-position error to gradually expand in accordance with the movement of the moving body.

5. The mobile body path analysis method described in claim 4, wherein the data processing unit sets the area of ​​the estimated self-position error on the moving path to the target position of the mobile body, analyzes whether or not the areas of the estimated self-position error set on the moving path overlap with obstacles, and determines that a path without overlap is a path along which the mobile body can move without coming into contact with or colliding with obstacles.

6. The mobile body path analysis method according to claim 1, wherein the data processing unit calculates an estimated self-position error that reflects a difficult-to-recognize area, in which the rate of increase in estimated self-position error that occurs in an area where it is difficult to recognize the environment and where high-precision self-position estimation processing is difficult is set higher than the rate of increase in estimated self-position error in normal areas other than the difficult-to-recognize area; analyzes whether the area of ​​the calculated estimated self-position error that reflects the difficult-to-recognize area overlaps with an obstacle; and determines that a route without overlap is a route that the mobile body can travel on without coming into contact with or colliding with an obstacle.

7. The mobile body path analysis method described in claim 6, wherein the data processing unit performs a calculation process to add a predetermined correction error R to the estimated self-position error in a normal area other than an area where environmental recognition is difficult and where high-precision self-position estimation processing is difficult, thereby calculating the estimated self-position error reflected in the area where recognition is difficult.

8. A method for analyzing a moving body path as described in claim 6, wherein the data processing unit calculates an estimated self-position error that reflects a difficult-to-recognize area, in which the area of ​​the estimated self-position error that reflects a difficult-to-recognize area is set to gradually expand in accordance with the movement of the moving body in the difficult-to-recognize area of ​​the environment.

9. A method for analyzing a path of a moving body as described in claim 6, wherein the difficult-to-recognize area is an area that differs depending on the type of sensor used by the moving body for environmental recognition, and the data processing unit calculates the estimated self-position error reflecting the difficult-to-recognize area based on the difficult-to-recognize area that is determined depending on the type of sensor used by the moving body for environmental recognition.

10. The mobile body path analysis method described in claim 6, wherein the data processing unit determines a path that does not overlap with an area of ​​the estimated self-position error that reflects the difficult-to-recognize area as the optimal path for the mobile body, and generates drive control information for moving the mobile body along the optimal path.

11. A mobile object path analysis method as described in claim 6, wherein the data processing unit calculates the estimated self-position error reflecting the difficult-to-recognize area by referring to a recognition-related heat map that can identify the difficult-to-recognize area of ​​the environment.

12. The mobile object path analysis method described in claim 6, wherein the data processing unit generates a recognition heat map capable of identifying the difficult-to-recognize area using a map set with attribute information that can determine whether high-precision self-position estimation processing is difficult or not.

13. A method for analyzing a moving body path as described in claim 12, wherein the difficult-to-recognize area is an area that differs depending on the type of sensor used by the moving body for environmental recognition, and the data processing unit generates a recognition-convenience heat map that makes it possible to identify the difficult-to-recognize area corresponding to each type of sensor used by the moving body for environmental recognition.

14. A mobile body path analysis method as described in claim 6, wherein the data processing unit generates a path as the movement path of the mobile body, with landmarks, which are objects or areas for which location information can be obtained, set as intermediate points.

15. A method for analyzing a moving body path as described in claim 14, wherein the data processing unit analyzes whether or not the area of ​​the estimated self-position error reflecting the difficult-to-recognize area overlaps with an obstacle for a route that has the landmark set as a stopover, and if it confirms that the route does not overlap, it determines that the route that has the landmark set as a stopover is a route that the moving body can travel on without coming into contact with or colliding with an obstacle.

16. The mobile object path analysis method according to claim 6, wherein the data processing unit generates a recognition-friendly heat map that makes it possible to identify the difficult-to-recognize areas of the environment and landmarks, which are objects or areas for which location information can be obtained.

17. The mobile object path analysis method described in claim 16, wherein the data processing unit generates the recognition-convenient heat map, which records landmarks that are objects or areas for which location information can be obtained, using a map in which location information is set as attribute information for each object or area.

18. A mobile body path analysis method as described in claim 6, wherein the data processing unit generates display data that makes it possible to identify at least one of the difficult-to-recognize area and the area of ​​the estimated self-position error reflecting the difficult-to-recognize area, and whether or not it overlaps with an obstacle.

19. A mobile body path analysis device having a data processing unit that calculates an estimated self-position error that includes a specified range of error in the estimated self-position calculated in the mobile body's self-position estimation process, analyzes whether the calculated estimated self-position error area overlaps with obstacles, and determines that a route without overlap is a route that the mobile body can travel without coming into contact with or colliding with obstacles.

20. A program that causes a moving body path analysis device to execute a moving body path analysis process, causing a data processing unit of the moving body path analysis device to execute the following processes: a process of calculating an estimated self-position error that includes a predetermined range of error in the estimated self-position calculated in the moving body's self-position estimation process; a process of analyzing whether the calculated area of ​​estimated self-position error overlaps with an obstacle; and a process of determining that a route without overlap is a route that the moving body can travel on without coming into contact with or colliding with an obstacle.

Images