Inspection route setting system and inspection route setting method

The system identifies and corrects unstable regions in inspection routes to maintain accurate self-position estimation, addressing the issue of degraded accuracy in existing systems and ensuring successful equipment inspection operations.

JP2024158196AActive Publication Date: 2024-11-08HITACHI LTD

Patent Information

Application Number
JP2023073184
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-04-27
Publication Date
2024-11-08
Estimated Expiration
2043-04-27

AI Technical Summary

Technical Problem

Existing inspection route setting systems fail to accurately identify regions where self-position estimation accuracy is degraded, leading to potential failures in equipment inspection operations by mobile objects.

Method used

An inspection route setting system that utilizes sensors and self-position estimation to detect areas with low accuracy by defining unstable regions based on predetermined thresholds, and identifies causes of instability, allowing for route adjustments to maintain accurate self-position estimation.

Benefits of technology

Enables appropriate detection of regions with reduced self-position estimation accuracy, allowing for route corrections and ensuring accurate equipment inspection operations.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024158196000001_ABST
    Figure 2024158196000001_ABST
Patent Text Reader

Abstract

To properly detect information on a decrease region of self-position estimation precision of an inspection route for facilities.SOLUTION: An inspection route setting system 1 comprises: a self-position estimation part 13 which estimates a self-position on a three-dimensional point group map including a previously set inspection route of facilities based upon information on an observation value by an outside world sensor 12 provided at a moving body 10 in a case in which the moving body travels in the inspection route; and an instability cause identification part 21 which determines, as an instable region where self-position estimation precision becomes unstable, a region where reliability of a self-position estimation result by the self-position estimation part 13 is equal to or less than a predetermined threshold and a variance value of the self-position estimation result is equal to or larger than a predetermined threshold.SELECTED DRAWING: Figure 1
Need to check novelty before this filing date? Find Prior Art

Description

[Technical Field]

[0001] The present invention relates to an inspection route setting system and an inspection route setting method. [Background technology]

[0002] Conventionally, equipment inspections have been performed using mobile objects such as robots. In general, the route along which the mobile object travels when inspecting equipment (hereinafter also referred to as an "inspection route") is generated by a user on a terminal device or the like, for example, by having the mobile object actually travel a pre-designed route and using information acquired by sensors provided on the mobile object.

[0003] However, there may be areas on an inspection route where the accuracy of self-location estimation by a mobile object decreases depending on the location of the equipment, the condition of the inspection route road surface, etc. In order to set an inspection route that can avoid such areas, it is necessary to understand information about the areas where the accuracy of self-location estimation decreases.

[0004] For example, Patent Document 1 discloses a method for detecting unstable factors in the control of autonomous driving of a vehicle based on information on the correlation between noise added to the vehicle's estimated self-position, obtained by tracing the estimated covariance matrix of the self-position, and the number of moving objects around the vehicle. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] International Publication No. 2019 / 203022 Summary of the Invention [Problem to be solved by the invention]

[0006] However, the technology described in Patent Document 1 does not check the validity of the self-location estimation result, so even if the self-location estimation result falls into a local optimum, for example, the estimation result is used as is. If information on the self-location estimation with low accuracy is used, there is a possibility that the mobile object will not be able to properly inspect the equipment.

[0007] The present invention has been made in consideration of the above circumstances, and an object of the present invention is to enable appropriate detection of information on areas where the accuracy of self-location estimation decreases on an inspection route for equipment. [Means for solving the problem]

[0008] An inspection route setting system according to one embodiment of the present invention comprises a self-position estimation unit that estimates its own position on a three-dimensional map including the inspection route based on information on observation values ​​from a sensor installed on the mobile body when the mobile body travels along a predetermined inspection route for equipment, and an unstable area identification unit that identifies an area where the reliability of the self-position estimation result by the self-position estimation unit is below a predetermined threshold and the variance value of the self-position estimation result is above a predetermined threshold as an unstable area where the self-position estimation accuracy is unstable. [Effects of the Invention]

[0009] According to at least one aspect of the present invention, it becomes possible to appropriately detect information about an area in which the accuracy of self-location estimation decreases on an inspection route for equipment. Problems, configurations, and effects other than those described above will become apparent from the following description of the embodiments. [Brief explanation of the drawings]

[0010] [Figure 1] 1 is a diagram illustrating an example of a schematic configuration of an inspection route setting system according to a first embodiment of the present invention. [Figure 2] 1 is a block diagram showing an example of the hardware configuration of each device constituting an inspection route setting system according to a first embodiment of the present invention. [Figure 3]1 is a diagram showing an outline of a 3D point cloud map constituting point cloud map information according to a first embodiment of the present invention. FIG. [Figure 4] 1 is a diagram showing an example of forming an inspection route within a 3D point cloud map according to the first embodiment of the present invention. FIG. [Figure 5] FIG. 3 is a diagram showing an example of inspection route information set on an inspection route according to the first embodiment of the present invention. [Figure 6] 1 is a diagram showing a 3D point cloud map according to the first embodiment of the present invention, a correspondence example between the position and orientation of a moving body, and observation values ​​obtained by an external sensor of the moving body. [Figure 7] 1 is a diagram showing an image of the distance between a target inspection route and the actual location of a moving body, and the sum of the distances, according to the first embodiment of the present invention. FIG. [Figure 8] 2 is a diagram showing an example of an environment including a position of a target to which height information is to be assigned by an instability cause identifying unit according to the first embodiment of the present invention; FIG. [Figure 9] 3A to 3C are diagrams illustrating an example of a 3D point cloud map and sensor observation values ​​when an environmental change occurs according to the first embodiment of the present invention. [Figure 10] FIG. 3 is a diagram showing an example of display of evaluation result information according to the first embodiment of the present invention. [Figure 11] 10 is a flowchart illustrating an example of a procedure for calculating reliability information of a result of self-position estimation by a moving object according to the first embodiment of the present invention. [Figure 12] 5 is a flowchart illustrating an example of a processing procedure performed by an instability cause identifying unit according to the first embodiment of the present invention. [Figure 13] 10 is a flowchart illustrating an example of a procedure for an instability cause identification process performed by an instability cause identification unit according to the first embodiment of the present invention. [Figure 14] FIG. 10 is a diagram illustrating an example of a schematic configuration of an inspection route setting system according to a second embodiment of the present invention. [Figure 15] 10 is a flowchart showing an example of a procedure for a self-position return operation addition process performed by a self-position return operation addition unit according to the second embodiment of the present invention. [Figure 16]10A and 10B are diagrams illustrating an example of adding a self-position return operation by a self-position return operation adding unit according to the second embodiment of the present invention. [Figure 17] FIG. 10 is a diagram illustrating an example of a schematic configuration of an inspection route setting system according to a third embodiment of the present invention. [Figure 18] 11 is a flowchart showing an example of the procedure of an inspection route correction process performed by an inspection route correction unit according to the third embodiment of the present invention. [Figure 19] FIG. 11 is a diagram showing an example of voxels that are targets for determining travelling capability according to the third embodiment of the present invention. [Figure 20] FIG. 11 is a diagram showing an example of generation of a corrected inspection route candidate by an inspection route correction unit according to the third embodiment of the present invention. [Figure 21] FIG. 11 is a diagram showing an example of a presentation of a corrected inspection route candidate to a user according to the third embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, examples of modes for carrying out the present invention (hereinafter referred to as "embodiments") will be described with reference to the accompanying drawings. The present invention is not limited to the embodiments, and various numerical values ​​in the embodiments are merely examples. Furthermore, in this specification and drawings, identical components or components having substantially the same functions will be designated by the same reference numerals, and redundant explanations will be omitted.

[0012] 1. First embodiment <Outline of the inspection route setting system> First, the configuration of an inspection route planning system according to a first embodiment of the present invention will be described with reference to Fig. 1. Fig. 1 is a diagram showing an example of the schematic configuration of an inspection route planning system 1.

[0013] 1, the inspection route setting system 1 includes a mobile body 10 configured as an autonomous mobile robot or the like, and a terminal device 20 configured as a PC (Personal Computer) or the like. Based on point cloud map information 30, inspection route information 50, and each piece of information transmitted from the mobile body 10, the terminal device 20 identifies a cause of instability that is causing a decrease in the accuracy of self-position estimation by the mobile body 10, and presents the identified cause of instability to a user via a display device or the like.

[0014] The mobile object 10 and the terminal device 20 are communicably connected via a wired or wireless network. Alternatively, information may be transmitted from the mobile object 10 to the terminal device 20 via a USB (Universal Serial Bus) memory (not shown) or the like.

[0015] The mobile object 10 includes an internal sensor 11 , an external sensor 12 , a self-position estimation unit 13 , a travel inspection operation control unit 14 , a self-position estimation success / failure determination unit 15 , and a reliability estimation unit 16 .

[0016] The internal sensor 11 is a sensor that acquires the amount of movement or movement speed, movement direction, turning speed, etc. of the mobile object 10 based on information such as the rotation speed of wheels (not shown) of the mobile object 10 and the number of rotations of a motor (not shown), and is configured, for example, by an IMU (Inertial Measurement Unit) or a wheel encoder, etc. The internal sensor 11 outputs the acquired information such as the movement speed and turning speed to the self-position estimation unit 13.

[0017] The external sensor 12 is a sensor capable of acquiring three-dimensional point cloud information, such as a LiDAR (Light Detection And Ranging) sensor. The detection values ​​by the internal sensor 11 and the observation values ​​(three-dimensional point cloud information) by the external sensor 12 are acquired according to a predetermined data acquisition cycle.

[0018] The self-position estimation unit 13 estimates the self-position and attitude (hereinafter also simply referred to as "self-position") of the moving body 10 based on the point cloud map information 30, the detection values ​​by the internal sensor 11, and the observation values ​​by the external sensor 12. The self-position estimation unit 13 outputs the self-position estimation result to the traveling inspection operation control unit 14, the self-position estimation success / failure determination unit 15, and the reliability estimation unit 16.

[0019] The point cloud map information 30 is a three-dimensional map that describes the inspection route along which the mobile object 10 travels to inspect equipment and information about the surrounding area. The point cloud map information 30 will be described in detail later with reference to FIG. 3.

[0020] The traveling inspection operation control unit 14 travels along the inspection route and performs inspection operations on equipment on the inspection route based on information such as observation values ​​from the external sensor 12, control values ​​40 input from a control device (not shown), inspection route information 50, and the vehicle's own position estimated by the self-position estimation unit 13. The inspection route information 50 includes information on each node N that constitutes the inspection route and the edges E (see FIG. 4) that connect the nodes N. The inspection route information 50 also includes information on the positions (nodes) where the mobile object 10 should perform inspections, the inspection operations, etc. The inspection route information 50 will be described in detail with reference to FIGS. 4 and 5 below.

[0021] The self-location estimation success / failure determination unit 15 generates success / failure determination data based on the self-location estimation result of the moving object 10 input from the self-location estimation unit 13, the point cloud map information 30, and the information of the observation values ​​by the external sensor 12, and determines the success / failure of the self-location estimation based on the success / failure determination data. The self-location estimation success / failure determination unit 15 outputs the determination result of the success / failure of the self-location estimation to the reliability estimation unit 16. Details of the processing by the self-location estimation success / failure determination unit 15 will be described in detail with reference to FIG. 6 described later.

[0022] The reliability estimation unit 16 calculates the reliability of the result of the self-position estimation by the self-position estimation unit 13 based on the self-position of the moving object 10 estimated by the self-position estimation unit 13 and the result of the determination of the success or failure of the self-position estimation by the self-position estimation success or failure determination unit 15. The reliability indicates the degree of probability that the self-position estimation is successful. The reliability calculation process by the reliability estimation unit 16 will be described later.

[0023] Furthermore, the reliability estimation unit 16 calculates the confidence level of the self-location estimation result by the self-location estimation unit 13. The confidence level is information indicating whether the variance value (covariance value) of the self-location estimation result falls within a predetermined threshold range. For example, the reliability estimation unit 16 determines that the confidence level is low when the variance value σ of the error of the self-location estimation result in the x and y directions and orientation is 0.2 or more, and determines that the confidence level is high when it is less than 0.2.

[0024] When the value of the self-location certainty is low and the reliability is low, it can be determined that the self-location estimation result has converged to an incorrect position (fallen into a local optimum solution). In other words, when both the reliability and the certainty are below the threshold, the reliability estimation unit 16 determines that the estimation accuracy of the self-location at the position where these values ​​are calculated is low. Therefore, according to this embodiment, it is guaranteed that the accuracy of the self-location estimation falls within an expected range. Note that the threshold value for determining the certainty is an example, and an appropriate value can be set.

[0025] Information on the reliability and certainty (hereinafter also simply referred to as "reliability") of the self-location estimation result calculated by the reliability estimation unit 16 is linked to the self-location estimation result by the self-location estimation unit 13 and the observation value by the external sensor 12, and output to the instability cause identification unit 21 of the terminal device 20.

[0026] The instability cause identifying unit 21 (an example of an unstable area identifying unit) of the terminal device 20 extracts an area where the reliability of the self-location estimation result of the mobile object 10 is low, i.e., where the estimation of the self-location by the self-location estimation unit 13 is unstable. Then, the instability cause identifying unit 21 identifies the cause of the instability in the extracted area, mainly the cause attributable to the topographical information. The instability cause identifying unit 21 identifies the cause of instability based on the self-location estimation result by the self-location estimation unit 13, the reliability of the self-location estimation result calculated by the reliability estimation unit 16, the observation value by the external sensor 12, and the inspection route information 50. The instability cause identifying unit 21 outputs the identified instability cause and various information used to identify the instability cause to the evaluation result drawing unit 22. Details of the processing by the instability cause identifying unit 21 will be described in detail with reference to FIGS. 8 and 9 described later.

[0027] The evaluation result drawing unit 22 (an example of an unstable area information output unit) presents the various pieces of information input from the instability cause identification unit 21 to the user as evaluation result information. The evaluation result information presented to the user by the evaluation result drawing unit 22 includes inspection route information 50, the position where the mobile object 10 actually traveled, the reliability and certainty of the self-location estimation result, the cause of the decrease in the reliability or certainty, information on the area where the self-location estimation became unstable, etc. A display example of the evaluation result information will be described in detail with reference to FIG. 10 described later.

[0028] <Example of computer hardware configuration> Next, the configuration (hardware configuration) of the control system of each device that constitutes the inspection route planning system 1 shown in FIG. 1 will be described with reference to FIG.

[0029] Fig. 2 is a block diagram showing an example of the hardware configuration of each device that constitutes the inspection route planning system 1. A calculator 200 shown in Fig. 2 is hardware used as a so-called computer.

[0030] The computer 200 includes a control unit 210, a nonvolatile storage 220, a display unit 230, an operation input unit 240, and a communication I / F (Interface) 250, which are all connected to a bus B.

[0031] The control unit 210 includes a central processing unit (CPU) 211, a read only memory (ROM) 212, and a random access memory (RAM) 213.

[0032] The CPU 211 reads out program code of software that realizes each function according to this embodiment from the ROM 212, expands it in the RAM 213, and executes it. Variables, parameters, etc. that are generated during the calculation process are temporarily written to the RAM 213.

[0033] The control unit 210 may include a processing device such as an MPU (Micro-Processing Unit) instead of the CPU 211. The control unit 210 may also be configured with an FPGA (Field-Programmable Gate Array), an ASIC (Application Specific Integrated Circuit), or the like.

[0034] The functions of the self-position estimation unit 13, driving inspection operation control unit 14, self-position estimation success / failure determination unit 15, and reliability estimation unit 16 of the moving body 10, and the functions of the instability cause identification unit 21 and evaluation result drawing unit 22 of the terminal device 20 are realized by the CPU 211 reading the corresponding program code from the ROM 212, expanding it into the RAM 213, and executing it.

[0035] The nonvolatile storage 220 may be, for example, a hard disk drive (HDD), a solid state drive (SSD), a flexible disk, an optical disk, a magneto-optical disk, a CD-ROM, a CD-R, a nonvolatile memory card, etc. In addition to an operating system (OS) and various parameters, programs for operating the computer 200 are recorded in the nonvolatile storage 220. The programs may be stored in the ROM 212.

[0036] The display unit 230 is, for example, a monitor configured with an LCD (Liquid Crystal Display) or the like, and displays the results of the processing performed by the computer 200, the evaluation result information generated by the evaluation result drawing unit 22, and the like.

[0037] The operation input unit 240 is configured with, for example, a keyboard, a mouse, a touch sensor, etc., and generates an operation signal according to an operation by a user and supplies it to the CPU 211. The display unit 230 and the operation input unit 240 may be integrated into a touch panel.

[0038] The program is stored in the form of a computer-readable program code, and the CPU 211 sequentially executes operations in accordance with the program code. In other words, the ROM 212 or the non-volatile storage 220 is used as an example of a computer-readable non-transitory recording medium that stores a program to be executed by a computer.

[0039] The communication I / F 250 may be, for example, a network interface card (NIC), and may transmit and receive various data to and from external devices via a network or communication line.

[0040] <Point cloud map information> Next, the point cloud map information 30 will be described with reference to Fig. 3. Fig. 3 is a diagram showing an outline of a 3D point cloud map (an example of a 3D map) constituting the point cloud map information 30. Fig. 3A is a diagram showing a schematic diagram of an environment (area) in which a moving object 10 actually travels, and Fig. 3B shows a 3D point cloud map 101 generated based on observation values ​​by an external sensor 12 of the environment shown in Fig. 3A.

[0041] When a 3D point cloud map is generated by measuring the area 100 shown in Fig. 2A, as shown in Fig. 2B, objects shown by dashed lines with the same size and shape as the objects are generated at positions corresponding to the positions of the objects in the area 100. The objects shown by dashed lines are configured as a collection of points 102, as shown in the object shown at the right end of the figure.

[0042] The three-dimensional point cloud map is expressed by a coordinate system 103 having an origin O at an arbitrary position in three-dimensional space, for example, the position where collection of three-dimensional point cloud information is started. Then, using this coordinate system 103 as a reference, objects in the area 100 observed by the external sensor 12 are placed within the three-dimensional point cloud map. The three-dimensional point cloud map may be generated based on a 3D CAD (Computer Aided Design) drawing or the like of the area including the inspection route.

[0043] <Inspection route information> Next, the configuration of the inspection route information 50 will be described with reference to Fig. 4 and Fig. 5. Fig. 4 is a diagram showing an example of the formation of an inspection route R within a three-dimensional point cloud map, and Fig. 5 is a diagram showing an example of inspection route information set for the inspection route R.

[0044] As shown in Fig. 4, inspection route R is represented by a directed graph made up of nodes N and edges E. Inspection route information 51 and inspection route information 52 shown in Fig. 5 are set to the nodes N and edges E that make up inspection route R, respectively. That is, the inspection route information is made up of inspection route information 51 set to node N and inspection route information 52 set to edge E.

[0045] As shown in FIG. 5A, the inspection route information 51 set for node N has the following items: "Node ID," "Coordinates," and "Connected Edge List." The "Node ID" item stores an ID that can uniquely identify the node. The "Coordinates" item stores the coordinates of node N expressed in the coordinate system of the 3D point cloud map. The "Action ID" item stores IDs assigned to specific inspection actions, such as photographing the equipment to be inspected, or entering a narrow road. The "Connected Edge List" item stores the identification ID of edge E connected to that node N.

[0046] As shown in Figure 5B, the inspection route information 52 set for edge E has the following items: "Edge ID," "Starting node ID," "Ending node ID," "Distance," "Speed," "Road width," "Relay point," and "Cause of instability."

[0047] The "Edge ID" field stores an ID that can uniquely identify the edge E, the "Start Node ID" field stores the node ID of the node N that is the start point of the edge E, the "End Node ID" field stores the node ID of the node N that is the end point of the edge E, the "Distance" field stores information about the length of the edge E, and the "Speed" field stores information about the target speed when the moving body 10 travels along the edge E.

[0048] The "Road width" field stores the width of the inspection route, i.e., road width information. The "Relay point" field stores coordinate information of the edge relay point set between edges E and E. The "Instability factor" field stores the identification ID of the instability factor identified by the instability cause identification unit 21 of the terminal device 20.

[0049] <Self-position estimation process by self-position estimation unit> Next, a self-location estimation process performed by the self-location estimation unit 13 of the moving object 10 will be described. The self-location estimation unit 13 can estimate the self-location of the moving object 10 using a known method that uses scan matching, such as Monte Carlo localization (MCL), iterative closest point (ICP), or normal distribution transform (NDT). In this embodiment, the self-location estimation unit 13 estimates the self-location using MCL. MCL is a method of estimating a position by approximating a probability distribution using particles.

[0050] Furthermore, the self-location estimation unit 13 updates the value of a reliability attenuation parameter that attenuates the reliability of the self-location estimation result estimated by the reliability estimation unit 16 described below. Since the reliability of the self-location estimation result decreases according to the magnitude of the movement amount and the amount of rotation of the mobile object 10, the value of the reliability attenuation parameter is updated at a predetermined period according to the model shown in the following equation (1).

[0051]

number

[0052] "r' in the above formula (1) t ” is the updated reliability decay parameter, “r t " is the reliability decay parameter before updating, "Δdt" is the error expected when the moving body 10 translates, "Δθt" is the error expected when turning, and "β1" and "β2" are non-negative parameters. "Δθt" and "Δθt" may be constant values ​​or may be changed depending on the condition of the road surface.

[0053] <Outline of self-location success / failure determination process by self-location estimation success / failure determination unit> Next, we will explain the outline of the self-location estimation success / failure determination process performed by the self-location estimation success / failure determination unit 15. As described above, the self-location estimation success / failure determination unit 15 generates success / failure determination data, and determines the success / failure of self-location estimation based on the success / failure determination data.

[0054] When generating the success / failure determination data, the self-location estimation success / failure determination unit 15 first converts the three-dimensional point cloud information acquired by the external sensor 12 into information in a map coordinate system, using the information on the self-location (posture) estimated by the self-location estimation unit 13. By performing such processing by the self-location estimation success / failure determination unit 15, the coordinate system of the sensor observation value coincides with the coordinate system in the three-dimensional point cloud map.

[0055] 6A and 6B are diagrams showing an example of correspondence between a 3D point cloud map, the position and orientation of a moving body 10, and observation values ​​obtained by an external sensor 12 of the moving body 10. Fig. 6A shows three objects 104a to 104c in the 3D point cloud map and observation points 105a to 105c obtained by observing each of the objects 104a to 104c. The moving body 10 is assumed to be present at a position indicated by a rectangle including an arrow mark indicating a direction, with the orientation indicated by the arrow mark (the orientation with the upward direction in the figure as the direction of travel).

[0056] In the example shown in FIG. 6A, the moving body 10 recognizes two sides (the right side and the bottom side in the figure) that make up the object 104a, as well as one side (the bottom side in the figure) that makes up the object 104b, and one side (the right side in the figure) that makes up the object 104c.

[0057] The left half of Figure 6B shows the correspondence between the 3D point cloud map and the observation values ​​by the external sensor 12 of the moving body 10 when the self-location estimation accuracy is high, and the right half shows the correspondence between the 3D point cloud map and the observation values ​​by the external sensor 12 of the moving body 10 when the self-location estimation accuracy is low.

[0058] When the self-localization accuracy is high, the observation points (observation points 105a1 to 105c1) of the external sensor 12 of the moving body 10 are projected near the objects 104a to 104c in the 3D point cloud map, as shown on the left side of Fig. 6B. On the other hand, when the self-localization accuracy is low, the observation points (observation points 105a2 to 105c2) of the external sensor 12 of the moving body 10 are projected at positions and angles that are shifted from the positions of the objects 104a to 104c in the 3D point cloud map, as shown on the right side of Fig. 6B.

[0059] The success / failure determination data is made up of information that changes when the accuracy of self-position estimation is reduced, and may use, for example, the following information (1) to (4). (1) The sum of the distances between the target inspection route that the mobile unit 10 should travel and the actual location of the mobile unit 10. (2) Mean absolute error between the target points on the point cloud map corresponding to the observation values ​​by the external sensor 12 and the actual observation points (3) A success / failure judgment image generated from the results of projecting the observation values ​​onto a point cloud map (4) The likelihood of the particle estimated to have the highest likelihood (maximum likelihood particle) by the particle filter used for self-estimation

[0060] The above (1) will be explained with reference to Fig. 7. Fig. 7 is a diagram showing an image of the distance between the target inspection route and the location where the mobile object 10 is actually located, and the sum of these distances. The distance between the target inspection route and the location where the mobile object 10 is actually located can be calculated, for example, from the length (distance) of a perpendicular line 106 extending from the origin (center position) of the mobile object 10 to the edge E, as shown in Fig. 7A.

[0061] The sum of the distances between the target inspection route and the actual location of the mobile body 10 is the sum of the distance between object 104a and observation point 105a1, the distance between object 104b and observation point 105b1, and the distance between object 104c and observation point 105c1.

[0062] The mean absolute error E between the target points on the point cloud map corresponding to the observation values ​​by the external sensor 12 and the actual observation points in (2) above MAE can be calculated, for example, by the following formula (2).

[0063]

number

[0064] In the above equation (2), "K" indicates the number of observation points by the external sensor 12, and "e" indicates the difference between the position of the observation point and the point included in the 3D point cloud map that is located closest to the observation point.

[0065] The image for determining success or failure described above in (3) can be generated, for example, by projecting the observation values ​​onto a three-dimensional point cloud map, generating an image of the area around the moving object 10 as if it were photographed from directly above, and then filling in the area through which the laser of the external sensor 12 (LiDAR) has passed. Fig. 7C shows an example of generating an image for determining success or failure. In the example shown in Fig. 7C, the area 107 through which the laser of the external sensor 12 (LiDAR) has passed is indicated by a dot pattern.

[0066] The self-location estimation success / failure determination unit 15 determines the success or failure of the self-location estimation by a determiner that performs deep learning using success / failure determination data as input. When the success / failure determination image of (3) above is input, the determiner can be configured using a CNN (Convolutional Neural Network) based learning model or the like that detects similarity with a pre-trained success / failure determination image.

[0067] Furthermore, for example, when the likelihood of the maximum likelihood particle in (4) above is used as the data for determining success or failure, a two-class classifier that classifies input values ​​into two classes, success or failure, can be used. The two-class classifier can be constructed using an algorithm such as Adaboost.

[0068] The threshold value used by the self-location estimation success / failure determination unit 15 to determine the success or failure of the self-location estimation result may be dynamically changed based on information such as the width of the inspection route traveled by the mobile object 10. By dynamically changing the threshold value based on the width of the inspection route, it becomes possible to determine the success or failure of self-location estimation using conditions appropriate for the environment, even in environments with different conditions, such as narrow roads and open areas.

[0069] <Reliability estimation process by reliability estimation unit> Next, a description will be given of the reliability estimation process performed by the reliability estimation unit 16. The reliability estimation unit 16 updates the reliability of the result of self-location estimation performed by the self-location estimation unit 13 using the following equation (3).

[0070]

number

[0071] In the above formula (3), “r t " indicates the updated reliability, and "r' t " denotes the parameter for reliability decay calculated using equation (1). "p s " indicates the probability that the determination result will be a true positive when the self-location estimation success / failure determination unit 15 determines that the self-location estimation has been successful, and that the determination result will be a true negative when the self-location estimation has been unsuccessful. "p f " indicates the probability that the determination result will be a false positive when the self-location estimation success / failure determination unit 15 determines that self-location estimation has been successful, and the probability that the determination result will be a false negative when the self-location estimation has been unsuccessful.

[0072] <Instability cause identification process by the instability cause identification unit> Next, the instability cause identification process performed by the instability cause identification unit 21 of the terminal device 20 will be described. The instability cause identification unit 21 extracts an area in the point cloud map information 30 where the reliability and certainty calculated by the reliability estimation unit 16 are equal to or lower than a predetermined threshold, i.e., an area where the self-estimation accuracy is unstable. Since the reliability calculated by the above formula (3) takes a value between 0 and 1, the threshold for the reliability can be set to, for example, "0.3". Furthermore, the threshold for the certainty can be set to, for example, "0.2", which is the variance value of the error in the self-location estimation result.

[0073] After extracting a position where the self-location estimation accuracy becomes unstable, the instability cause identification unit 21 assigns height information to that position. Fig. 8 is a diagram showing an example of an environment including a position to which height information is assigned by the instability cause identification unit 21. Fig. 8A shows an example of the environment, and Fig. 8B shows a 3D point cloud map of the environment shown in Fig. 8A.

[0074] In the environment shown in Fig. 8A, equipment 110a to equipment 110g are arranged. The floor surface on which equipment 100e to 100g are arranged is located at a higher position than the floor surface on which equipment 100a to equipment 100d are arranged, and a step exists at boundary 122. In addition, in the environment shown in Fig. 8A, an inspection route R that passes by equipment 110 is set. Inspection route R includes a route that passes position 121 that is on boundary 122. Therefore, when passing position 121, moving object 10 (not shown in Fig. 8) needs to climb over the step in a downward direction.

[0075] If this position 121 is identified as a position where the self-location estimation accuracy becomes unstable, the instability cause identification unit 21 performs a process of assigning height information to the position 121. In the height information assignment process, the instability cause identification unit 21 first performs clustering on the 3D point cloud map using information on normals and reflection intensity information. Known clustering methods such as Region Growing Segmentation, Difference of Normals Based Segmentation, and Conditional Euclidean Clustering can be used as the clustering method.

[0076] Region Growing Segmentation is a method for dividing regions using information on the normals and curvature of the point cloud surface, Difference of Normals Based Segmentation is a method for clustering using information on the difference in normal direction calculated using different thresholds, and Conditional Euclidean Clustering is a method for clustering based on information on the difference in normal direction and information on the reflection intensity of the point cloud.

[0077] Fig. 8B shows an example of a 3D point cloud map 123 after clustering. In the 3D point cloud map 123, points belonging to the same surface are clustered as points on the same plane. In the 3D point cloud map 123 shown in Fig. 8B, each element such as facility 110 existing in the environment shown in Fig. 8A is shown with a different pattern.

[0078] Next, the instability cause identification unit 21 detects planes from the clustered 3D point cloud map 123. A known method such as RANSAC (RANdom Sample Consensus) can be used to detect planes. RANSAC is a method for learning the parameters of a mathematical model from data that includes outliers, while excluding the influence of outliers. If the value of the z-direction normal of the detected plane is, for example, 0.92 or greater, the instability cause identification unit 21 determines that the plane is a plane on which the mobile object 10 can travel. A normalized value is used for the z-direction normal value. The above value of 0.92 is an example, and an appropriate value can be set based on the size of the step that the mobile object 10 can overcome, the angle of inclination, etc.

[0079] Next, the instability cause identification unit 21 divides each edge E that makes up the point cloud route R into lengths of, for example, 25 cm, and sets the height of the plane directly below the edge end of each divided edge as the height of the edge end. The division unit of edge E, "25 cm," is just an example, and any other value may be set as long as it is an appropriate value for detecting changes in road surface height. The instability cause identification unit 21 sets the generated information about the edge end in the "relay point" item of the inspection route information 52 for edge E shown in FIG. 5.

[0080] The instability cause identifying unit 21 performs a process of setting height information for a position where the self-location estimation accuracy is unstable, and then performs a process of identifying the cause of the instability of the self-location estimation accuracy. In the process of identifying the cause of instability, the instability cause identifying unit 21 first extracts a difference between the 3D point cloud map and the observation value by the external sensor 12 at the position where the self-location estimation accuracy is unstable.

[0081] For example, if an object that does not exist in the 3D point cloud map newly appears in the environment in which the mobile body 10 is traveling, the point cloud observed for the newly appeared object will not exist on the 3D point cloud map. On the other hand, if a specific object disappears from the environment in which the mobile body 10 is traveling, the object that exists on the 3D point cloud map will not exist in the observation values ​​by the external sensor 12. Therefore, by extracting the difference between the 3D point cloud map and the observation values ​​by the external sensor 12, it is possible to detect such environmental changes. The difference between the 3D point cloud map and the observation values ​​by the external sensor 12 can be calculated using, for example, a known method that uses a voxel map and nearest neighbor search.

[0082] FIG. 9 illustrates an example of a 3D point cloud map and sensor observation values ​​when an environmental change occurs. FIG. 9A illustrates an example of an environment, and FIG. 9B illustrates a 3D point cloud map of the environment illustrated in FIG. 9A. If equipment 110d and equipment 110g disappear from the environment illustrated in FIG. 9A, the areas corresponding to equipment 110d and equipment 110g in the 3D point cloud map illustrated in FIG. 9B are extracted as the difference between the 3D point cloud map and the observation values ​​obtained by the external sensor 12. When such an environmental change is detected, the instability cause identification unit 21 sets information about an "environmental change" in the "cause of instability" field of the inspection route information 52 for edge E illustrated in FIG. 5. In other words, according to this embodiment, when an environmental change is the cause of instability in the self-localization estimation accuracy of the mobile object 10, this fact can be identified.

[0083] Next, the instability cause identifying unit 21 performs a process of checking whether the cause of instability in the self-location estimation accuracy is climbing over a step. The instability cause identifying unit 21 compares the height of the position (edge ​​relay point) extracted as the position where the self-location estimation accuracy is unstable with the height of each edge relay point before and after the edge relay point. If a difference is detected by the comparison and the magnitude of the difference is equal to or greater than a preset threshold, the instability cause identifying unit 21 estimates that there was a sudden change in height during the movement of the mobile object 10, and determines that the cause of instability is climbing over a step. Then, the instability cause identifying unit 21 sets the information of "climbing over a step" in the "cause of instability" field of the inspection route information 52 for edge E shown in FIG. 5.

[0084] If the difference detected by the comparison does not exceed a predetermined threshold, the instability cause identifying unit 21 estimates that a gradual change in height occurred during the movement of the mobile object 10. Then, the instability cause identifying unit 21 calculates the inclination angle of the edge E by determining the difference between the height at the start point of the edge E while the mobile object 10 is traveling and the height at the end point. If the calculated inclination angle is equal to or greater than a preset threshold, the instability cause identifying unit 21 sets information about "uphill driving" or "downhill driving" in the "cause of instability" field of the inspection route information 52 for edge E shown in FIG. 5 based on information about the gradient of the inclination. In other words, according to this embodiment, if the cause of instability in the self-location estimation accuracy of the mobile object 10 is climbing over a step or traveling on a sloped route, it can be identified.

[0085] <Evaluation result information> Next, the evaluation result information generated by the evaluation result drawing unit 22 will be described with reference to Fig. 10. Fig. 10 is a diagram showing an example of display of the evaluation result information.

[0086] 10, a vertically long oval mark 141 indicates to the user an area where the accuracy of self-location estimation has decreased due to going over a step, and a dotted rectangular area 142 indicates to the user an area where the accuracy of self-location estimation has decreased due to the slope of the inspection route. Furthermore, a solid arrow 143 indicates the route actually traveled by the mobile object 10. A dashed frame 144 indicates to the user information about equipment that has been confirmed to be missing.

[0087] Such a display allows the user to visually grasp information about areas where the accuracy of self-location estimation of the mobile body 10 is reduced. Therefore, the user can take measures such as recreating an inspection route that avoids these areas or instructing the mobile body 10 to perform an operation to restore the accuracy of self-location estimation.

[0088] The evaluation result display unit 145 is an area that displays the history of the reliability and certainty of the self-location estimation results estimated while the mobile object 10 was traveling along an edge selected by the user. In the evaluation result display unit 145, the history of the reliability is shown by a line graph with the reliability (0 to 1) on the vertical axis and the time (sec) on the horizontal axis. The history of the certainty is shown by a line graph with the time (sec) on the horizontal axis. With such a display, the user can consider measures to improve the accuracy of the self-location estimation of the mobile object 10 based on the information on the history of the reliability and / or certainty of the self-location estimation result of the mobile object 10. Note that the display in the evaluation result display unit 145 is not limited to the example shown in FIG. 10 , and other information may be displayed in other display formats.

[0089] The area 146 labeled "Map Update" is an area that displays a message urging the user to update the information on the 3D point cloud map. This message is displayed when an environmental change, such as the disappearance of equipment, occurs. By updating the information on the user's 3D point cloud map based on this message, the environment in which the moving body 10 is actually traveling can be matched with the information on the 3D point cloud map.

[0090] <Processing for calculating reliability information of self-location estimation results by a moving body> Next, a process of calculating reliability information of the result of self-location estimation by the moving body 10 will be described with reference to Fig. 11. Fig. 11 is a flowchart showing an example of the procedure of a process of calculating reliability information of the result of self-location estimation by the moving body 10.

[0091] First, the self-position estimation unit 13 of the moving body 10 acquires point cloud map information 30 of the environment in which the moving body 10 is traveling, and reads the 3D point cloud map (step S1). Next, the self-position estimation unit 13 reads inspection route information 50 of the inspection route along which the moving body 10 is traveling (step S2). Next, the self-position estimation unit 13 acquires the observation values ​​obtained by the external sensor 12, i.e., the 3D point cloud information (step S3). In step S3, the self-position estimation unit 13 also acquires information on the moving speed and turning speed of the moving body 10, which are calculated based on the detection values ​​obtained by the internal sensor 11.

[0092] Next, the self-position estimation unit 13 estimates the self-position (and attitude) on the 3D point cloud map (step S4). Next, the self-position estimation success / failure determination unit 15 generates data for determining the success / failure of the self-position estimation based on the 3D point cloud map and information on the self-position estimated in step S4 (step S5). Next, the self-position estimation success / failure determination unit 15 determines the success / failure of the self-position estimation performed in step S4 using the success / failure determination data generated in step S5 (step S6). Next, the reliability estimation unit 16 calculates the reliability and certainty of the self-position estimation result based on the success / failure determination result of the self-position estimation in step S6 (step S7).

[0093] Next, the traveling inspection operation control unit 14 of the mobile body 10 identifies which node N / edge E the mobile body 10 is traveling through (step S8).Since the mobile body 10 moves along the edge from the starting node of the inspection route, the node N / edge E through which the mobile body 10 is traveling can be estimated based on the travel distance calculated based on the self-position estimation result of the mobile body 10.

[0094] Next, the traveling inspection operation control unit 14 determines whether the moving object 10 has reached the terminal node of the inspection route (step S9). If it is determined in step S9 that the terminal node has been reached (if step S9 returns YES), the reliability estimation unit 16 stores information on the reliability and certainty of the self-location estimation result by the moving object 10 calculated in step S7 (step S10). After processing step S10, the process of calculating reliability information on the self-location estimation result by the moving object 10 ends.

[0095] On the other hand, if it is determined in step S9 that the moving object 10 has not reached the terminal node (if step S9 is determined as NO), the traveling inspection operation control unit 14 determines whether the node N where the moving object 10 is located is a node N in which a specific operation, i.e., an inspection operation, is registered (step S11).If it is determined in step S11 that the node N is a node in which a specific operation is registered (if step S11 is determined as YES), the traveling inspection operation control unit 14 performs the specific operation registered in the node N (step S12).

[0096] After processing step S12, or if step S11 is determined to be NO, the traveling inspection operation control unit 14 calculates the control amount of the mobile object 10 required to move along the inspection route to the next node N (step S13). The calculation of the control amount required to travel while following the route can be performed using known techniques such as model predictive control and sliding mode control. After processing step S13, the traveling inspection operation control unit 14 resumes the movement of the mobile object 10 and returns to step S3.

[0097] <Processing of the instability cause identification part> Next, the processing by the instability cause identifying unit 21 will be described with reference to Fig. 12. Fig. 12 is a flowchart showing an example of the procedure of the processing by the instability cause identifying unit 21.

[0098] First, the instability cause identification unit 21 of the terminal device 20 reads the 3D point cloud map (step S21). Next, the instability cause identification unit 21 reads the inspection route information 50 traveled by the mobile object 10 (step S22). Next, the instability cause identification unit 21 reads the information on the reliability and certainty of the self-location estimation result of the mobile object 10 saved in step S10 of FIG. 11 (step S23).

[0099] Next, the instability cause identifying unit 21 extracts a position where the self-location estimation of the moving body 10 has become unstable based on the information read in steps S21 to S23 (step S24). Specifically, the instability cause identifying unit 21 extracts a position where the reliability and certainty are equal to or less than a preset threshold value as an unstable position.

[0100] Next, the instability cause identification unit 21 calculates and assigns height information to the unstable positions extracted in step S24 (step S25). Next, the instability cause identification unit 21 performs an instability cause identification process, which is a process for identifying the cause of instability in self-location estimation (step S26). The process of step S26 is performed for all positions extracted in step S24. The instability cause identification process of step S26 will be described in detail with reference to the following FIG. 13. Next, the instability cause identification unit 21 saves the instability cause identified in step S26 (step S27). After the process of step S27, the process by the instability cause identification unit 21 ends.

[0101] <Process to identify the cause of instability> Next, the instability cause identification process performed in step S26 of Fig. 12 will be described with reference to Fig. 13. Fig. 13 is a flowchart showing an example of the procedure for the instability cause identification process performed by the instability cause identification unit 21.

[0102] First, the instability cause identification unit 21 determines whether the reliability and certainty read in step S23 of Fig. 12 are each smaller than a threshold value (step S31). If it is determined in step S31 that the reliability and certainty are each smaller than the threshold value (YES in step S31), the instability cause identification unit 21 performs processing to extract the difference between the 3D point cloud map and the observation value by the external sensor 12 (step S32).

[0103] Next, the instability cause identifying unit 21 determines whether or not a difference has been extracted in step S32 (step S33). If it is determined in step S33 that a difference has been extracted (if the determination in step S33 is YES), the instability cause identifying unit 21 sets "environmental change" as the cause of the instability of the self-estimation accuracy (step S34).

[0104] After the processing of step S34, if step S31 is judged as NO or if step S33 is judged as NO, instability cause identifying unit 21 determines whether or not a sudden change in height has occurred at the position where the self-estimation accuracy has become unstable (step S35). Instability cause identifying unit 21 determines whether or not a sudden change in height has occurred based on whether or not a difference between the height set at the position (node) where the self-estimation accuracy has become unstable and the heights set at the nodes before and after the node exceeds a predetermined threshold. If it is judged in step S35 that a sudden change in height has occurred (if step S35 is judged as YES), instability cause identifying unit 21 sets "going over a step" as the cause of the instability of the self-estimation accuracy (step S36).

[0105] On the other hand, if it is determined in step S35 that there is no sudden change in height (if step S35 is determined as NO), the instability cause identifying unit 21 determines whether there has been a gradual change in height at the position where the self-estimation accuracy became unstable (step S37). The instability cause identifying unit 21 calculates the inclination angle of the edge E from information on the difference between the height at the start point and the height at the end point while the moving body 10 is traveling, and determines whether there has been a gradual change in height based on the magnitude of the inclination angle.

[0106] If it is determined in step S37 that there is a gradual change in height (if step S37 is determined as YES), the instability cause identifying unit 21 sets "uphill" or "downhill" as the cause of the instability of the self-estimation accuracy. The instability cause identifying unit 21 can determine whether the cause of instability is an uphill or a downhill slope based on the inclination direction of the edge.

[0107] After the process of step S36, after the process of step S38, or if the determination in step S37 is NO, the instability cause identifying unit 21 performs the process of step S27 in Fig. 12. That is, the identified instability cause is stored.

[0108] In the above-described embodiment, the instability cause identification unit 20 identifies, as an unstable region where the self-location estimation accuracy is unstable, a region where the reliability indicating the probability that the self-location estimation by the self-location estimation unit 13 is successful is equal to or less than a predetermined threshold and the variance value of the self-location estimation result is equal to or greater than a predetermined threshold. Therefore, according to this embodiment, it is possible to appropriately detect information about a region where the self-location estimation accuracy is reduced on an inspection route of an equipment.

[0109] Furthermore, since information regarding the unstable positions identified by the instability cause identifying unit 20 is presented to the user by the evaluation result drawing unit 22, the user can evaluate and / or modify the inspection route based on information about the area where the self-location estimation accuracy of the mobile body 10 becomes unstable. Furthermore, the user can generate a route that is less likely to cause instability in the self-location estimation accuracy of the mobile body 10.

[0110] Furthermore, according to the above-described embodiment, even a user with little knowledge about the moving body 10 can appropriately set (modify) the inspection system based on the evaluation result information presented by the instability cause identification unit 20.

[0111] 2. Second embodiment <Outline of the inspection route setting system> Next, a second embodiment of the present invention will be described. Fig. 14 is a diagram showing a schematic configuration example of an inspection route setting system 1A according to the second embodiment. The inspection route setting system 1A shown in Fig. 14 differs from the inspection route setting system 1 according to the first embodiment in that it includes specific action information 60 and a self-position return action adding unit 23. The configuration other than the specific action information 60 and the self-position return action adding unit 23 is the same as that of the inspection route setting system 1 shown in Fig. 1, so a description thereof will be omitted.

[0112] The specific action information 60 is a table that defines information on specific actions that require high self-position estimation accuracy, such as an inspection action, an action of entering a narrow road, etc. In the specific action information 60, the content of the specific action is managed in association with a specific action ID (not shown).

[0113] The self-position return operation adding unit 23 causes the moving body 10 to perform an operation (hereinafter also referred to as a "self-position return operation") that enables recovery (return) of the self-position estimation accuracy before the moving body 10 performs a specific operation. The operation that enables recovery of the self-position estimation accuracy includes, for example, an operation of turning on the spot, an operation of moving finely within a range of about 1 meter and then stopping, and the like.

[0114] For example, assume that the posture of the mobile object 10 changes due to climbing over a step or the like, causing the observed value of the external sensor 12 to become abnormal, resulting in an error in the self-position estimation result. In this case, by the mobile object 10 performing the return operation described above, the external sensor 12 can stably acquire observed values, and the self-position estimated by the self-position estimation unit 13 converges again.

[0115] <Self-position return operation addition process by self-position return operation addition unit> Next, the self-position return operation adding process by the self-position return operation adding unit 23 will be described with reference to Fig. 15. Fig. 15 is a flowchart showing an example of the procedure of the self-position return operation adding process by the self-position return operation adding unit 23.

[0116] First, the self-position return operation adding unit 23 extracts a node (hereinafter also referred to as a "specific operation setting node") in which a specific operation that requires high self-position estimation accuracy, such as an inspection operation or an operation of entering a narrow road, is set (step S41). The self-position return operation adding unit 23 can extract the specific operation setting node by, for example, comparing the "operation ID" of the inspection route information 51 for node N shown in Fig. 5 with the "specific operation ID" in the specific operation information 60.

[0117] The processing from step S42 onwards is repeated the number of times equal to the number of specific operation setting nodes extracted in step S41. After the processing of step S41, the self-position return operation adding unit 23 determines whether or not there is an edge in which a cause of instability in self-position estimation is set, among edges that point in the direction of the specific operation setting node and whose distance from the specific operation setting node is within a preset distance threshold (step S42). That is, in step S42, the self-position return operation adding unit 23 determines whether or not there is an edge in which a cause of instability is set, immediately before a specific operation setting node that requires high self-position estimation accuracy.

[0118] If it is determined in step S42 that there is an edge to which an instability cause is set (if step S42 is determined as YES), the self-position return operation adding unit 23 determines whether or not another node exists between the edge to which the instability cause is set and the specific operation setting node (step S43). That is, in step S43, the self-position return operation adding unit 23 determines whether or not there is an edge that is closer to the specific operation setting node than the edge to which the instability cause is set.

[0119] If it is determined in step S43 that another node exists (if step S43 is determined to be YES), the self-position return operation addition unit 23 adds a specific operation ID associated with the self-position return operation to the inspection route information 51 of the other node (step S44).

[0120] On the other hand, if it is determined in step S43 that no other node exists (if step S43 is determined as NO), the self-position return operation adding unit 23 adds a specific operation ID associated with the self-position return operation to the inspection route information 51 of the specific operation setting node (step S45). After the processing of step S44, after the processing of step S45, or if step S42 is determined as NO, the self-position return operation adding process by the self-position return operation adding unit 23 ends.

[0121] 15 is performed by the self-position return operation addition unit 23, so that even when traveling along an inspection route where a specific operation such as an inspection operation needs to be performed after climbing over a step, the self-position return operation addition unit 23 performs an operation to return the self-position estimation accuracy of the mobile body 10 before performing the specific operation. This allows the mobile body 10 to perform an inspection operation using the self-position estimation result after the estimation accuracy has been restored, thereby preventing failure of the inspection operation based on a self-position estimation result with low accuracy.

[0122] It is possible that both the specific operation and the self-returning operation are set for one node through the self-returning operation addition process by the self-returning operation adding unit 23. In this case, the moving object 10 is to perform the self-returning operation first.

[0123] Next, an example of adding a self-position return action by the self-position return action adding unit 23 will be described with reference to Fig. 16. Fig. 16 is a diagram showing an example of adding a self-position return action by the self-position return action adding unit 23.

[0124] Fig. 16 shows the environment in which the moving object 10 travels. The environment shown in Fig. 16 is the same as the environment shown in Fig. 8A. That is, the floor surface on which the equipment 1003e to 100g are arranged is located at a higher position than the floor surface on which the equipment 100a to 100d are arranged, and a step exists at the boundary 122.

[0125] 16, the specific operation is set to node N5 as an operation of entering a narrow passage between facilities 110h and 110i, and the specific operation is set to node N2 as an inspection operation. Also, the moving object 10 is assumed to perform an operation of climbing over a step just before node N2 and just before node N5.

[0126] The narrow road entry operation set in node N5 is an operation that is performed immediately after going over a step, and there is no other node between the position where the step (boundary 122) is passed and node N5. Therefore, self-position return operation adding unit 23 sets the self-position return operation in node N5, which is a specific operation setting node.

[0127] On the other hand, at node N2 where an inspection operation is set, another node N1 exists between the location where the step climbing operation is performed and node N2. Therefore, the self-position return operation adding unit 23 sets a self-position recovery operation at node N1. By performing such processing by the self-position return operation adding unit 23, it becomes possible to create an inspection route that also includes the setting of the self-position return operation.

[0128] 3. Third embodiment Next, a third embodiment of the present invention will be described. Fig. 17 is a diagram showing a schematic configuration example of an inspection route setting system 1B according to the third embodiment. The inspection route setting system 1B shown in Fig. 17 differs from the inspection route setting system 1 according to the first embodiment only in that it includes an inspection route correction unit 24. The configuration other than the inspection route correction unit 24 is the same as that of the inspection route setting system 1 shown in Fig. 1, so a description of these components will be omitted.

[0129] The inspection route correction unit 24 generates an inspection route that avoids areas where self-location estimation may become unstable. That is, the inspection route is corrected based on information about the instability cause identified by the instability cause identification unit 21.

[0130] FIG. 18 is a flowchart showing an example of the procedure for the inspection route correction process performed by the inspection route correction unit 24. First, the inspection route correction unit 24 clusters the 3D point cloud map. As a clustering method, the same method as that performed by the instability cause identification unit 21 can be used. Next, the inspection route correction unit 24 divides the 3D point cloud map clustered in step S51 into voxels (an example of divided regions) (step S52). The division unit into voxels can be set to, for example, 1 meter per side.

[0131] Next, for each voxel divided in step S52, the inspection route correction unit 24 determines the traversal capability of the mobile body 10 through that voxel (step S53). Specifically, the inspection route correction unit 24 determines the traversal capability of the mobile body 10 based on information such as the presence or absence of steps on the route when the mobile body 10 enters from above, below, left, right, or diagonally and passes through that voxel.

[0132] Here, an example of a voxel that is the target of travelling capability determination will be described with reference to FIG. 19. FIG. 19 is a diagram showing an example of a voxel that is the target of travelling capability determination. FIG. 19 shows a voxel 160 that includes a step 161 that exists in the vertical direction in the figure. When the mobile object 10 travels along a path 162 that proceeds from bottom to top in the figure, the mobile object 10 can travel through the voxel 160 without any problems because there is no step on the travelling path. Therefore, the inspection path correction unit 24 determines that the mobile object 10 can travel through the voxel 160 from the path 162.

[0133] When the moving body 10 travels along a path 163 that extends diagonally from the lower right to the upper right in the drawing, the moving body 10 crosses a step 161 diagonally, and therefore the moving body 10 cannot travel through the voxel 160. Therefore, the inspection route correction unit 24 determines that the moving body 10 cannot travel through the voxel 160 from the path 163.

[0134] When the mobile object 10 travels along the path 164 going from right to left in the figure, the angle of entry to the step 161 is perpendicular to the step 161, and therefore the mobile object 10 can travel along the path. Therefore, the inspection path correction unit 24 determines that the mobile object 10 can travel through the voxel 160 from the path 164. However, when entering from the path 164, a step must be overcome during travel, and therefore the inspection path correction unit 24 records the height of the step along with the determination result. Note that if a change in height is detected between the entry position and exit position of the voxel by the mobile object 10, the inspection path correction unit 24 records the height change information along with the determination result of the travelability.

[0135] Returning to FIG. 18, the explanation will be made. After the processing of step S53, the inspection route correction unit 24 extracts a specific operation setting node (step S54). The extraction of the specific operation setting node can be performed by the same method as the method described in step S41 of FIG. 15. Next, if there is a route that has a section including an edge that makes the accuracy of self-position estimation unstable among the routes connecting the specific operation setting nodes extracted in step S54, the inspection route correction unit 24 generates an inspection route correction candidate that avoids that route (step S55). After the processing of step S55, the inspection route correction processing by the inspection route correction unit 24 ends.

[0136] In step S55, the inspection route correction unit 24 first identifies a voxel that includes a specific operation setting node. The voxel to which the node belongs can be determined by, for example, determining which voxel coordinates the node's coordinates, divided into voxel sizes, correspond to. Next, the inspection route correction unit 24 generates a correction candidate inspection route based on the travelability information determined in step S53. The inspection route correction unit 24 generates the correction candidate inspection route using only routes that pass through voxels that are determined to be travelable. This has the effect of allowing the inspection route to be set as a route that does not involve climbing over steps or the like, which can easily destabilize the self-estimation accuracy of the mobile object 10.

[0137] The inspection path correction unit 24 can generate a candidate inspection path using a known method such as the Dijkstra algorithm. When using the Dijkstra algorithm, the inspection path correction unit 24 sets the cost to the time required to pass through a voxel. The time required for the mobile object 10 to pass through a voxel can be calculated using the following equation (4).

[0138]

number

[0139] In the above formula (4), "t v ” indicates the time required to pass through a voxel, and “t r " indicates the time required for the moving object 10 to pass through the voxel at the assumed speed. a " indicates the time required for the self-returning operation.

[0140] The inspection route correction unit 24 sets a route that passes through all nodes for which an inspection operation is set as a route for the correction candidate inspection route. Furthermore, nodes for which no inspection operation is set are selected as nodes connecting nodes for which an inspection operation is set. Therefore, the inspection route correction unit 24 does not select nodes that are not included in the route connecting nodes for which an inspection operation is set as nodes that constitute the route for the correction candidate inspection route. Furthermore, information about nodes that the mobile body 10 does not pass through is deleted from the inspection route information 50 of the inspection route R.

[0141] Next, an example of generating corrected inspection route candidates by the inspection route corrector 24 will be described with reference to Fig. 20. Fig. 20 is a diagram showing an example of generating corrected inspection route candidates by the inspection route corrector 24.

[0142] FIG. 20 shows the environment in which the mobile body 10 travels. The environment shown in FIG. 20 is the same as the environment shown in FIG. 8A. FIG. 20A shows an example of an inspection route R before correction, and FIG. 20B shows an inspection route R' that is a candidate for the correction of the inspection route. In FIG. 20, nodes for which no specific operation is set are indicated by white circles, and specific operation-set nodes are indicated by black circles.

[0143] In the inspection route R before correction shown in FIG. 20A, a step (boundary 122) is crossed on each of the edge E12 connecting the node N12 and the node N13 and the edge E19 connecting the node N19 and the node N.

[0144] On the other hand, in the inspection route R' of the correction candidate for the inspection route shown in Fig. 20B, edge E12' that goes over a low step 171 is set instead of edge E12 where a step occurs, and edge E19' that goes over a low step 172 is set instead of edge E19 where a step occurs. The inspection route R' of the correction candidate for the inspection route is a longer route than the inspection route R before correction, but considering the time required to perform the step climbing and self-returning operations that must be performed on the inspection route R before correction, it is expected that the travel time will be shorter.

[0145] FIG. 21 is a diagram showing an example of an inspection route correction candidate presented to a user. In the screen showing the inspection route correction candidate shown in FIG. 21, the upper half shows information about the current inspection route R, and the lower half shows information about the inspection route R' of the inspection route correction candidate. In the current inspection route R shown in the upper half of the screen, vertically long elliptical marks 181 and 182 are superimposed on areas where the accuracy of self-location estimation becomes unstable due to the occurrence of climbing over a step, i.e., near the end of edge E12 and at edge E19, respectively. This display allows the user to visually and appropriately grasp information about areas on the inspection route R where the accuracy of self-location estimation of the mobile object 10 becomes unstable.

[0146] In the inspection route R', a candidate for the corrected inspection route, shown in the lower half of the screen, a dot pattern indicating that the inspection route has been corrected is superimposed on the edge E12' and edge E18' that have been reconstructed to avoid crossing steps. This display allows the user to visually and appropriately grasp the information about the area where the correction has been made. The user can compare the corrected inspection route R' displayed on the screen with the inspection route that they have set, and select a more preferable inspection route.

[0147] According to the above-described third embodiment, if there is an area in the inspection route where the self-position estimation of the mobile body 10 becomes unstable, the inspection route setting system 1C can suggest to the user information on a corrected inspection route that avoids this area.

[0148] Furthermore, according to the present embodiment, route candidates that are less likely to reduce the accuracy of self-location estimation are proposed to the user, thereby reducing the burden on the user of creating point routes and reducing the variation in the quality of the created inspection routes.

[0149] In the third embodiment described above, the inspection path correction unit 24 generates the inspection path correction candidate using information about the path that passes through voxels that require crossing steps. However, the present invention is not limited to this. For example, the inspection path correction unit 24 may first attempt to generate the inspection path correction candidate using only information about voxels that do not require crossing steps, and if this generation fails, may generate the inspection path correction candidate using information about the path that can traverse the voxels that require crossing steps. By performing such processing, it is possible to further reduce the possibility that the inspection path will include a path that includes a position where the accuracy of the self-location estimation of the mobile object 10 becomes unstable.

[0150] In the above-described embodiments, the information required for evaluating the accuracy of self-location estimation is acquired by having the mobile object 10 actually travel along an inspection route, but the present invention is not limited to this. The accuracy of self-location estimation may be evaluated on a simulator that reproduces the environment in which the mobile object 10 travels, and the inspection route may be corrected by the inspection route setting system 1 (1A, 1B) before introducing the mobile object 10 into the site.

[0151] Furthermore, the above-described embodiments provide detailed and specific descriptions of the configurations of the devices and systems in order to clearly explain the present invention, and are not necessarily limited to those having all of the configurations described.

[0152] 1, 2, 14, and 17, the control lines or information lines indicated by solid lines or arrows are those considered necessary for explanation, and do not necessarily show all control lines or information lines in the product. In reality, it can be considered that almost all components are interconnected.

[0153] Furthermore, in this specification, processing steps describing chronological processing include not only processing that is performed chronologically in the order described, but also processing that is not necessarily performed chronologically but is performed in parallel or individually (for example, parallel processing or processing by objects).

[0154] Furthermore, each component of the elevator system according to one embodiment of the present invention described above may be implemented in any hardware as long as the respective hardware can transmit and receive information to each other via a network. Furthermore, the processing performed by a certain processing unit may be realized by a single piece of hardware, or may be realized by distributed processing using multiple pieces of hardware. [Explanation of symbols]

[0155] 1, 1A, 1B, 1C... inspection route setting system, 10... moving body, 11... internal sensor, 12... external sensor, 13... self-position estimation unit, 14... driving inspection operation control unit, 15... self-position estimation success / failure determination unit, 16... reliability estimation unit, 20... terminal device, 21... instability cause identification unit, 22... evaluation result drawing unit, 23... self-position return operation addition unit, 24... inspection route correction unit

Claims

1. a self-location estimation unit that estimates a self-location on a three-dimensional map including a preset inspection route for equipment based on information of an observation value by a sensor provided on the mobile body when the mobile body travels along the preset inspection route for equipment; and an unstable area identifying unit that identifies an area in which a reliability indicating a high probability that the self-location estimation by the self-location estimation unit is successful is equal to or less than a predetermined threshold and a variance value of the estimation result of the self-location is equal to or more than a predetermined threshold as an unstable area in which the self-location estimation accuracy is unstable. Inspection route setting system.

2. a self-location estimation success / failure determination unit that determines whether the estimation of the self-location has been successful or unsuccessful by using a learning model in which success / failure determination data generated based on the self-location estimated by the self-location estimation unit is input; a reliability estimation unit that calculates a reliability of the estimation result of the self-location based on a result of the determination by the self-location estimation success / failure determination unit, and calculates a variance value of the estimation result of the self-location. The inspection route planning system according to claim 1 .

3. When the unstable area identification unit extracts a difference between an observation value obtained by the sensor in the unstable area and a corresponding position of the observation value in the three-dimensional map, the unstable area identification unit determines that a cause of an error in the self-location estimation in the area where the difference is detected is a change in a traveling environment of the mobile body. The inspection route planning system according to claim 2 .

4. An unstable region information output unit that outputs information on the unstable region identified by the unstable region identification unit, When the cause of the error is a change in a traveling environment of the moving body, the unstable area information output unit issues a notification to prompt updating of information of a three-dimensional map including the inspection route. The inspection route planning system according to claim 3 .

5. The unstable area identification unit acquires height information in the unstable area and the surrounding area, and when a difference is detected between the height in the unstable area and the height in the surrounding area, determines that the cause of instability of the self-location estimation in the area where the difference is detected is the moving body climbing over a step or an inclination of the inspection route. The inspection route planning system according to claim 2 .

6. The method further includes a self-position return operation addition unit that causes the moving body to perform an operation for returning self-position estimation accuracy in a section between the unstable area detected by the unstable area identification unit and an inspection operation execution point that is located ahead of the unstable area in the traveling direction of the moving body.

6. An inspection route planning system according to claim 3 or 5.

7. The present invention further includes an inspection route correction unit that determines a travelability of the moving body in each divided area obtained by dividing the three-dimensional map into voxel units when the moving body enters the divided area from a plurality of different approach directions, and generates a correction candidate for the inspection route based on the determined travelability information.

6. An inspection route planning system according to claim 3 or 5.

8. a step of estimating a self-position on a three-dimensional map including a preset inspection route for equipment based on information of observation values ​​by a sensor provided on the mobile body when the mobile body travels along the preset inspection route for equipment; and a step of identifying an area in which a reliability indicating a high probability that the self-location estimation is successful is equal to or less than a predetermined threshold and a variance value of the estimation result of the self-location is equal to or more than a predetermined threshold as an unstable area in which the self-location estimation accuracy is unstable. How to set up inspection routes.

Citation Information

Patent Citations

  • Map generation method, own position estimation method, robot system and robot

    JP2017045447A

  • Autonomous mobile body

    JP2017188067A

  • Mobile body system, map creation system, route creation program and map creation program

    JP2020166702A

  • Self position estimation device and self position estimation method

    JP2021117893A

  • Own location estimating method

    JP2022042630A

Cited By

  • Information processing method, information processing device, and program

    WO2025253893A1