Robots performing equipment maintenance
The integration of a railway robot and drones with customizable sensors and shapes addresses the limitations of existing unmanned maintenance technologies, enabling efficient inspection and repair of infrastructure targets beyond their usual range, thus reducing maintenance time and costs.
Patent Information
- Application Number
- JP2021072085
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2021-04-21
- Publication Date
- 2024-11-07
- Estimated Expiration
- 2041-04-21
AI Technical Summary
Existing unmanned maintenance technologies for infrastructure systems face challenges in inspecting and repairing targets beyond the range of robots or drones due to environmental conditions and complex infrastructure configurations, leading to prolonged maintenance times and increased costs.
A system comprising a railway robot and drones that collaboratively perform maintenance tasks, where the railway robot transports drones to targets outside its range, and customizes them with appropriate sensors or shapes to ensure effective inspection and repair, utilizing dynamic scheduling and real-time environmental data for optimal task allocation.
This approach significantly reduces maintenance work time by enabling efficient inspection and repair of targets previously inaccessible, thereby enhancing the reliability and efficiency of infrastructure maintenance.
Smart Images

Figure 0007579197000001 
Figure 0007579197000002 
Figure 0007579197000003
Abstract
Description
[Technical field]
[0001] The present invention relates to a technique for performing unmanned maintenance of equipment. [Background technology]
[0002] Social infrastructure systems such as electric power, railways, water supply and sewerage, and urban transportation are composed of multiple subsystems. For example, a railway system is composed of more than 100 subsystems, such as trains, tracks, electric power, bridges, signals, and tunnels. Many of the subsystems are located in remote locations.
[0003] Such social infrastructure systems require continuous operational improvements. For example, when it comes to urban transportation maintenance, while maintenance costs are on the rise due to the aging of facilities, transportation revenues are expected to decrease due to population decline. Therefore, there is a need to devise policies to reduce maintenance costs without compromising transportation safety.
[0004] Thousands of kilometers of track and associated electrical equipment require frequent inspection and repair. It is difficult to dispatch enough people to remote locations to complete inspection and repair of tracks and associated equipment in time. For example, after an earthquake or bad weather, rail operators must carefully check everything: rails, signals, power, bridges, tunnels, etc. Long inspection and repair times cause delays to many trains.
[0005] In recent years, there have been many research and attempts to use robots for railway inspection and repair. A robot that runs on railroad tracks (railroad robot) can quickly inspect railroad tracks and overhead lines. However, due to the limited scanning range of the sensors installed on railroad robots, many important objects may be excluded from the target list. For example, objects that are too high up, such as cables and tunnels, or too far away, such as pylons and railway bridges, may be excluded.
[0006] One solution (see, for example, Patent Document 3) is railroad inspection using drones. Drones can fly to locations where targets are beyond the range of robots. A typical problem with maintenance using drones is that infrastructure targets are complex: they are in different locations, have different shapes, have different sizes, are subject to different regulations, have different obstacles around them, and, one of the most important factors, are in different environments, such as weather. Therefore, it is difficult to use one type of drone to cover all targets.
[0007] Another solution (see, for example, Patent Document 1) combines robots and 3D printers to perform maintenance in remote locations. When a part of a system breaks down, it is quicker to print a part of the equipment than to order a new one. Another method (see, for example, Patent Document 2) uses a truck to carry multiple drones for different tasks such as delivering packages. [Prior art documents] [Patent documents]
[0008] [Patent Document 1] U.S. Patent No. 10,518,411 [Patent Document 2] US Patent Application Publication No. 2018 / 0233053 [Patent Document 3] JP 2020-006916 A Summary of the Invention [Problem to be solved by the invention]
[0009] Most monitoring data from infrastructure systems includes data from hundreds or thousands of sensors. In addition to being high-dimensional, the monitoring data includes millions of rows of data. Furthermore, infrastructure systems have high robustness, with anomalies rarely occurring, e.g., the anomaly occurrence rate is about 1 in 100,000 or less. This makes it very difficult to detect and predict anomalies.
[0010] The technique of Patent Document 1 uses a robot system to monitor and manage the health of assets. A 3D printer mounted on the robot repairs malfunctions. However, Patent Document 1 does not mention how to maintain targets that are outside the range of the robot.
[0011] The method of Patent Document 2 schedules multiple tasks and obtains a final order for execution using multiple drones. It controls the traffic of multiple drones for task management and flight path setting. However, Patent Document 2 does not mention a method for processing tasks that are out of range of drones with different environmental conditions.
[0012] Patent Document 3 discloses a management device that can easily and safely fly an unmanned aerial vehicle by creating a flight route using the area where the railway track is laid and owned by the railway operator. It is safe and easy to fly a drone in a managed area. However, Patent Document 3 does not mention a method of flying a drone to various targets outside the range of the drone.
[0013] In many cases, there are several obstacles in the development of unmanned technology for the maintenance of infrastructure systems. (1) The scope of inspection and repair by railway robots is limited. For example, it is difficult for the camera of a railway robot to clearly capture a small crack in a railway bridge. (2) It is difficult for the railway robot to determine what to inspect depending on the on-site environment. For example, there are cases where the camera cannot capture a clear image of the target overhead wire due to fog.
[0014] Therefore, there is a demand for unmanned technology that can shorten the time required for maintenance work, whether during regular maintenance or after a disaster. [Means for solving the problem]
[0015] One aspect of the present invention is a robot that performs maintenance on a facility, the robot including one or more arithmetic devices and one or more storage devices. The robot transports one or more drones that perform maintenance and backup parts for the one or more drones. The one or more storage devices store a task schedule that instructs one or more maintenance tasks and configuration information for each of the one or more drones. The task schedule indicates a maintenance target position and maintenance content. The one or more arithmetic devices fly a first drone of the one or more drones toward a first target of a first maintenance task, and determine whether the first drone can perform maintenance on the first target based on a distance between the first drone and the first target determined based on a measurement result of the first drone and a maintenance range of the first drone indicated by the configuration information, and customize the first drone using the backup parts when it is determined that the first drone cannot perform maintenance on the first target. Effect of the Invention
[0016] According to one aspect of the present invention, the maintenance work time can be reduced. [Brief description of the drawings]
[0017] [Figure 1] 1 shows a schematic diagram of an example of the overall configuration of a railway maintenance system. [Diagram 2] 1 shows an example of the hardware configuration of a maintenance management system. [Diagram 3] An example of the logical configuration of a railway robot system implemented in a railway robot is shown below. [Figure 4A] 13 shows an example of the configuration of drone parts management information. [Figure 4B] 1 shows an example of the configuration of drone configuration management information. [Figure 4C] 13 shows an example of the configuration of drone current configuration management information. [Diagram 5] The diagram shows the maintenance flow using railway robots and drones. [Figure 6] This shows an example in which a drone and a railway robot calibrate their inspection ranges by measuring each other's effective inspection ranges. [Figure 7] This shows an example where the maintenance target exceeds the inspection range of the railway robot when RTmin>d2max. [Figure 8] This shows an example where the maintenance target is beyond the drone's inspection range when DTmin>dmax. [Figure 9] The conditions for drone sensor customization are shown below. [Figure 10] Specifies the conditions for customizing the drone shape (including size). [Figure 11] Examples of maintenance schedules for rail robots and drones are shown below. [Figure 12] A flowchart of the operation of a rail robot that employs drones for maintenance tasks is shown. [Figure 13] Illustrates scheduling of time-bound maintenance tasks. [Figure 14] A conceptual diagram showing a railway robot carrying a drone. [Figure 15] A schematic diagram of an example of drone customization using a railway robot. [Figure 16] 1 shows examples of targets inside and outside the maintenance range of a rail robot. [Figure 17] This shows a drone performing maintenance on a target that is outside the maintenance range of a rail robot. [Figure 18] An example is shown in which the environment changes to fog or rain, and the maintenance range of the rail robot 7 changes in response to the change in environment. [Figure 19] This shows an example in which the environment changes to fog or rain and the drone maintenance range changes accordingly. [Figure 20] Here is an example of customizing a drone as the environment changes to fog or rain. [Figure 21]This is an example of maintenance being performed by a customized drone due to the environment changing to fog or rain. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0018] Hereinafter, an embodiment of the present invention will be described with reference to the accompanying drawings. In the accompanying drawings, functionally identical elements are indicated by the same numbers. Note that the accompanying drawings show specific embodiments according to the principles of the present invention, but these are for understanding the present invention and are by no means used to interpret the present invention in a limiting manner.
[0019] In the following, when necessary for convenience, the description will be divided into multiple sections or examples, but unless otherwise specified, they are not unrelated to each other, and one is related to the other as a partial or complete modification, detail, supplementary explanation, etc. Furthermore, in the following, when the number of elements, etc. (including the number, numerical value, amount, range, etc.) is mentioned, it is not limited to the specific number, and may be more or less than the specific number, unless otherwise specified or clearly limited in principle to a specific number, etc.
[0020] The system may be a physical computer system (one or more physical computers) or a system built on a group of computing resources (multiple computing resources) such as a cloud infrastructure. The computer system or the group of computing resources includes one or more interface devices (including, for example, a communication device and an input / output device), one or more storage devices (including, for example, a memory (main memory) and an auxiliary storage device), and one or more processors.
[0021] When a function is realized by a program being executed by a processor, the defined processing is performed using a storage device and / or an interface device, etc. as appropriate, and therefore the function may be at least a part of the processor. Processing described with a function as the subject may be processing performed by a processor or a system having the processor. The program may be installed from a program source. The program source may be, for example, a program distribution computer or a computer-readable storage medium (for example, a computer-readable non-transitory storage medium). The description of each function is an example, and multiple functions may be combined into one function, or one function may be divided into multiple functions.
[0022] An embodiment of an equipment maintenance system according to the present specification will be described below. Maintenance of railway equipment, which is an example of social infrastructure equipment, will be described below. The technology of the embodiment of the present specification can be applied to maintenance of equipment other than railway equipment systems, for example, other social infrastructure equipment such as water supply and sewerage equipment and urban transportation equipment, and equipment other than social infrastructure equipment.
[0023] 1 shows a schematic diagram of an example of the overall configuration of a railway maintenance system. The railway maintenance system includes a maintenance management system 1, a railway robot 7, and a drone (unmanned aerial vehicle) 9. A railway schedule management system 10 and a weather system 12 are external systems to the railway maintenance system.
[0024] The maintenance management system 1 manages a rail robot 7 and a drone 9 for unmanned maintenance of rail infrastructure. The rail robot 7 can travel along rails and carry one or more drones 9 to one or more destinations for maintenance. The drones 9 are placed on the carrier of the rail robot 7.
[0025] The railroad robot 7 can travel on or along rails by means of a movable moving device such as wheels or caterpillar tracks. In Fig. 1, one of the multiple railroad robots is indicated by reference numeral 7 as an example. Also, one of the multiple drones is indicated by reference numeral 9 as an example.
[0026] The maintenance management system 1 includes a maintenance task management system 2, a dynamic maintenance scheduling system 3, a railway robot and drone management database 4, and a real-time inspection system 5.
[0027] The maintenance task management system 2 manages the maintenance tasks of all maintenance targets. Maintenance of a target involves performing one or both of inspection and repair of the target. The dynamic maintenance scheduling system 3 performs dynamic scheduling of the maintenance tasks. The dynamic maintenance scheduling system 3 obtains information on the railway schedule from the railway schedule management system 10, and determines the task schedule for automatic maintenance by the railway robot 7 and drone 9 based on the information.
[0028] The railroad robot and drone management database 4 stores railroad robot management information that manages all railroad robots, and drone management information that manages all drones. The railroad robot management information includes, for example, an identifier of the railroad robot, a waiting position of the railroad robot, a sensor mounted on the railroad robot, a distance that can be repaired by the railroad robot, a shape (including size) of the railroad robot, an identifier of the drone mounted on the railroad robot, and an identifier of a replacement part for the drone mounted on the railroad robot.
[0029] The drone management information includes, for example, a drone identifier, a drone's onboard sensor, a distance that the drone can perform repairs, the shape (including size) of the drone, and the identifiers and specifications of each part that can be mounted on the drone. The drone management information further stores information on the shape (including size) of each drone consisting of a different combination of parts.
[0030] As described below, each railway robot 7 obtains at least railway robot management information related to the device itself, and drone management information for the drones that it transports and controls, from the railway robot and drone management database 4, and stores the information in the management information database.
[0031] The real-time inspection system 5 verifies the results of the inspections and repairs performed by the rail robot 7 and drone 9. The maintenance management system 1 and rail robot 7 obtain weather information from a weather system 12. The weather information that can be provided includes current weather information for the area where maintenance is to be performed, as well as weather forecasts for that area. As described below, the weather information is referenced in task scheduling by the dynamic maintenance scheduling system 3 and in the execution of maintenance by the rail robot 7.
[0032] The components other than the drones 9 shown in Fig. 1 can communicate with each other via a network 6. Each of the rail robots 007 can communicate with each of the drones 9 that it manages via a network 8. In Fig. 1, the network between one rail robot 7 and one or more drones 9 that it manages and controls is indicated by the reference numeral 8 as an example.
[0033] 2 shows an example of a hardware configuration of the maintenance management system 1 according to an embodiment of the present specification. The maintenance management system 1 may have, for example, a computer configuration. The maintenance management system 1 includes a processor 101, which is a computing device having computing performance, and a DRAM 102, which is a main memory device providing a volatile temporary storage area for storing programs and data executed by the processor 101. Furthermore, the maintenance management system 1 includes an auxiliary storage device 104 that provides a permanent information storage area using an HDD (Hard Disk Drive), flash memory, or the like. The DRAM 102, the auxiliary storage device 104, and a combination thereof are each a storage device.
[0034] The maintenance management system 1 further includes a communication device 103 that performs data communication with other devices, an input device 105 that accepts operations from a user, and a monitor 106 (an example of an output device) that presents the output results of each process to the user. These components can communicate with each other via a bus. The number of each of the components shown in FIG. 2 is arbitrary, and some of the components, for example, the input device 105 and the monitor 106, may be omitted.
[0035] 1, specifically, the maintenance task management system 2, the dynamic maintenance scheduling system 3, and the real-time inspection system 5, may be implemented, for example, by a processor 101 that executes a program including instruction codes, or may be implemented in different computers.
[0036] Programs for implementing the functional units are stored, for example, in the auxiliary storage device 104. The programs executed by the processor 101 and data to be processed are loaded from the auxiliary storage device 104 to the DRAM 102. At least some of the functions in the system may be implemented by circuits for specific functions, instead of a general-purpose processor that operates according to a program.
[0037] The maintenance management system may be a physical computer system (one or more physical computers) as shown in Fig. 2, or a system built on a group of computing resources (multiple computing resources) such as a cloud infrastructure. The computer system or the group of computing resources includes one or more interface devices (including, for example, a communication device and an input / output device), one or more storage devices (including, for example, a memory (main memory) and an auxiliary storage device), and one or more processors.
[0038] When a function is realized by executing a program by a processor, the defined processing is performed using a storage device and / or an interface device, etc., as appropriate, and therefore the function may be considered to be at least a part of the processor. Processing described using a function as the subject may be considered to be processing performed by a processor or a system including the processor.
[0039] The program may be installed from a program source. The program source may be, for example, a program distribution computer or a computer-readable storage medium (for example, a computer-readable non-transitory storage medium). The description of each function is an example, and multiple functions may be combined into one function, or one function may be divided into multiple functions.
[0040] The functional units of the railway robot 7, which will be described later, may be realized by one or more processors that operate according to programs stored in one or more storage devices, as described above.
[0041] 3 shows an example of a logical configuration of a railway robot system 201 implemented in a railway robot 7. The railway robot system 201 includes a railway robot management system 202 and a drone management system 203. The railway robot management system 202 includes a database 204, a communication system 205, a robot arm control system 206, and a monitoring and analysis system 207.
[0042] The database 204 stores data from the railroad robot 7 and the drone 9. The communication system 205 is a system for communicating with other devices. The robot arm control system 206 controls the robot arm of the railroad robot 7 to perform maintenance work. The monitoring and analysis system S07 performs real-time inspection and analysis of the inspection results.
[0043] The drone management system 203 includes a drone information management system 208, a maintenance range calibration system 209, a drone customization system 210, and a drone control system 211.
[0044] The drone information management system 208 stores and manages information about the drones installed on the vehicle. The information about the drones includes information about drone parts and finished drones. The maintenance range calibration system 209 calibrates the sensors of the rail robot 7 and the drones 9. The drone customization system 210 performs customization of the drones 9. The drone control system 211 controls the flight of the drones 9 and maintenance by the drones 9.
[0045] The railway robot system 201 may be, for example, a system including one or more processors and one or more storage devices that execute a program including an instruction code, as described with reference to Fig. 2. The functions of the railway robot system 201 may be realized by one or more processors that operate according to the corresponding programs, and information may be stored in one or more storage devices. At least some of the functions of the railway robot system 201 may be realized by a computing device other than a general-purpose processor.
[0046] 4A-4C show examples of information stored in the drone information management system 208. Fig. 4A shows an example of the configuration of drone parts management information 250. The drone parts management information 250 manages parts of the drone 9 mounted on the railway robot 7. In the example of Fig. 4A, the drone parts management information 250 indicates a target 251, a part ID 252, a weight 253, a size 254, a resolution / range 255, and a usage status 256.
[0047] Target 251 indicates the action target of the drone part. Part ID 252 indicates the ID of the drone part. Importance 253 indicates the weight of the drone part. Size 254 indicates the size of a given portion of the drone part. Resolution / Range 255 indicates, for example, the resolution of the sensors (including the camera) or the range of motion of the arm. Usage Status 256 indicates whether the drone part is currently being used by the drone or is a backup part.
[0048] FIG. 4B shows an example of the configuration of drone configuration management information 260. The drone configuration management information 260 shows configuration information for each combination of drone component parts. In the example of FIG. 4B, the drone configuration management information 260 shows a part combination ID 261, combined parts 262, total weight 263, and size 264. The part combination ID 261 shows the ID of the combination of parts that make up the drone. The combined parts 262 show the IDs of each drone part in the combination. The total weight 263 shows the total weight of the drone, and the size 264 shows the size of the drone, which in this example is the length in three directions.
[0049] Fig. 4C shows an example of the configuration of drone current configuration management information 270. The drone current configuration management information 270 shows the current configuration of the drone 9 mounted on the railway robot 7. In the example of Fig. 4C, the drone current configuration management information 270 shows a drone ID 271, combined parts 272, total weight 273, and size 274. The drone ID 271 shows the ID of the drone. The combined parts 272 show the IDs of each drone part that makes up the drone. The total weight 273 shows the total weight of the drone, and the size 274 shows the size of the drone, which in this example shows the lengths in three directions.
[0050] 5 shows a flow of maintenance using a railway robot and a drone. In step S101, the dynamic maintenance scheduling system 3 obtains maintenance tasks to be performed from the maintenance task management system 2. Information on each maintenance task to be performed is stored in, for example, the auxiliary storage device 104 and managed by the maintenance task management system 2.
[0051] In step S102, a maintenance schedule for each rail robot is determined. The maintenance schedule may indicate the location of each target, the shape of each target, the estimated arrival time at each target, the estimated maintenance time for each target (the time required for maintenance), the entity that performs each task (rail robot or drone), and the estimated end time of all maintenance tasks, etc.
[0052] A dynamic maintenance scheduling system 3 determines the maintenance schedule for each rail robot based on information obtained from a rail robot and drone management database 4, a rail schedule management system 10 and a weather system 12. Each maintenance task can command the inspection of a target and its repair if a defect is already known.
[0053] For example, the dynamic maintenance scheduling system 3 determines a maintenance schedule so that all tasks of each railway robot are completed before the start time of railway service obtained from the railway schedule management system 10. The dynamic maintenance scheduling system 3 may determine the entity that will perform a task based on a weather forecast for the scheduled execution time of each task. For example, when poor visibility is predicted due to rain or fog, maintenance of targets far from the railway robot may be assigned to a drone.
[0054] Next, in step S103, the dynamic maintenance scheduling system 3 sets a maintenance schedule (task) for each railroad robot 7. The railroad robot 7, together with the drone carried thereon, departs toward a target according to the set maintenance schedule.
[0055] In step S104, the rail robot 7 reaches the target position and starts calibration of the maintenance range (checking the maintenance range). This allows proper maintenance to be performed. An example of performing calibration of an inspection range, which is one of the maintenance ranges, will be described below. Calibration of a repair range can also be performed in a similar manner. For example, if the task instructs inspection only, calibration of the inspection range is performed, and if inspection and repair are instructed, calibration of the inspection range and repair range is performed.
[0056] In the following description, it is assumed that the railroad robot 7 is designated as the subject of the task to be executed. If the drone 9 is designated to perform the maintenance, the following steps S105 and S106 are omitted. Note that the subject to perform the maintenance does not have to be designated. In that case, the maintenance (task) of the target is executed according to this flow.
[0057] In step S105, the maintenance range calibration system 209 determines whether RTmin>d2max. d2max indicates the maximum distance (maximum inspection range) from the railway robot 7 to the target at the minimum allowable inspection resolution. A method for calculating d2max will be described in detail later. RTmin is the minimum distance that the railway robot 7 can approach the target. The maintenance range calibration system 209 can calculate RTmin based on the position of the target indicated by the task information and the position of its own device on the track.
[0058] If RTmin is equal to or less than d2max (S105: NO), in step S106, the monitoring and analysis system 207 of the railway robot 7 executes inspection (maintenance) of the target. The analysis result is transmitted to the real-time inspection system 5 of the maintenance management system 1 via the communication system 205. This completes the maintenance of one target.
[0059] If RTmin is greater than d2max (S105: YES), the maintenance range calibration system 209 determines to perform maintenance of the target using a drone and selects a drone to perform the maintenance. This enables maintenance of the target. The selected drone may be any one or a specific drone may be selected according to a preset rule.
[0060] In step S107, the drone control system 211 sets a target position and a flight path for the selected drone 9. The drone 9 takes off toward the target according to the set information. The drone control system 211 can set control information for instructing the operation of the drone 9 before takeoff, and can communicate with the drone 9 during flight to obtain surrounding information obtained by the drone 9 and instruct the operation of the drone 9.
[0061] In step S108, the maintenance range calibration system 209 determines whether DTmin>dmax. dmax indicates the maximum distance (maximum inspection range) from the drone 9 to the target at the minimum allowable inspection resolution. The method of calculating dmax will be described in detail later. DTmin is the minimum distance that the drone 9 can approach the target. The drone 9 flies toward the target and measures the distance that can be most easily reached by the target.
[0062] If DTmin is equal to or less than dmax (S108: NO), in step S114, the drone 9 performs inspection (maintenance) of the target. Specifically, the monitoring and analysis system 207 of the railroad robot 7 receives an image of the target from the drone 9 and analyzes the received image. The analysis result is transmitted to the real-time inspection system 5 of the maintenance management system 1 via the communication system 205.
[0063] If DTmin is greater than dmax (S108: YES), in step S109, the drone customization system 210 customizes the sensor of the drone 9 with an appropriate sensor. The drone 9 returns to the rail robot 7 in response to an instruction from the drone control system 211. The drone customization system 210 estimates the resolution required for the new sensor based on the resolution (performance) of the sensor before customization, DTmin, and dmax, and selects a sensor that satisfies the estimated resolution or has the highest performance from the backup parts and replaces it.
[0064] In step S110, the maintenance range calibration system 209 starts inspection range calibration for the customized drone 9 and calculates dmax2. dmax2 indicates the maximum distance (maximum inspection range) from the customized drone 9 to the target at the minimum allowable inspection resolution of the drone 9. A method for calculating dmax2 will be described in detail later.
[0065] In step S111, the maintenance range calibration system 209 determines whether DTmin>dmax2. If DTmin is equal to or less than dmax2 (S111: NO), in step S113, the customized drone 9 performs inspection (maintenance) of the target. Specifically, the monitoring and analysis system 207 of the railway robot 7 receives an image of the target from the drone 9 and analyzes the received image. The analysis result is transmitted to the real-time inspection system 5 of the maintenance management system 1 via the communication system 205.
[0066] If DTmin is greater than dmax2 (S111: YES), in step S112, the drone customization system 210 customizes the drone into a new shape (including size). The drone 9 returns to the rail robot 7 in response to an instruction from the drone control system 211. The size (X, Y, Z) of the customized drone is customized to satisfy the following conditions. X<(l+2*l´), Y<(m+2*m´), Z<(n+2*n´).
[0067] Here, X, Y, and Z are the maximum dimensions of the drone in mutually perpendicular directions (called the X-direction, Y-direction, and Z-direction). l, m, and n are the minimum dimensions in the X-direction, Y-direction, and Z-direction of the passing tunnel (see Figure 9) detected by the drone's sensor. l', m', and n' are the preset detection errors for l, m, and n.
[0068] The drone customization system 210 can know the size of each part combination by referring to the drone configuration management information 260. The drone customization system 210 can know the current configuration of the drone 9 by referring to the drone current configuration management information 270, and can further know replaceable parts by referring to the drone parts management information 250.
[0069] The drone customization system 210 selects the part to be replaced so that the above size condition is satisfied. At this time, the replacement part is selected so that dmax2 does not change. When it is necessary to reduce dmax2 to satisfy the size condition, for example, the sensor with the highest performance within the range where the size condition is satisfied is selected. Note that shape customization may be performed before sensor customization. If DTmin after shape customization is equal to or less than dmax, sensor replacement is not necessary.
[0070] In step S113, the customized drone 9 performs inspection (maintenance) of the target. This completes the maintenance of one target. In the above example, the maximum inspection range is referenced as the maintenance arrangement. If the maintenance includes repair, the repairable range is compared with the minimum distance that the rail robot 7 or drone 9 can approach the target, and the maintenance subject and customization are determined. The repair range can be determined from the operating range of a finishing tool such as an arm attached to the device. If the inspection result indicates the need to repair a defect, the rail robot 7 may repair the defect using its own device or drone.
[0071] An example of the inspection range calibration method will be described below. Fig. 6 shows an example in which a drone 9 and a railroad robot 7 calibrate their inspection ranges by measuring the effective inspection ranges of each other.
[0072] In addition to parameters of real-time environmental conditions (weather conditions) such as temperature, humidity, wind direction, wind speed, air pressure, visibility, etc., there are parameters of fixed size (a, b, c) and moving speed v2 of the rail robot 7, fixed size (x, y, z), moving speed v1 and weight M of the drone 9, and dynamic distance d and direction Rd between the drone 9 and the rail robot 7. These parameters are applied to the calibration of the rail robot 7 for the maximum inspection range d2max, and the calibration of the drone 9 for the maximum inspection range dmax (or dmax2).
[0073] The environmental conditions can be obtained from the weather system 12. In another example, the rail robot 7 may be equipped with a device for observing the current environmental conditions. The size of the rail robot 7 is preset, and the size and weight of the drone 9 can be obtained from the drone current configuration management information 270. The dynamic distance and direction of the rail robot 7 and the drone 9 can be calculated from their respective current positions. The speed of the rail robot 7 and the drone 9 can be measured by a speedometer equipped in each.
[0074] First, the calibration of the inspection range of the railway robot 7 will be described. This calculates the longest distance that can be inspected for the sensors installed in the railway robot 7 under the current environmental conditions. The detection error of the size (x, y, z) of the drone 9 by the railway robot 7 is given as follows. f1, f2, and f3 are pre-defined functions. They may be common. Detection error x' = d * f1(v2, v1, T, H, Wd, v3, P, s) Detection error y' = d * f2(v2, v1, T, H, Wd, v3, P, s) Detection error z' = d * f3(v2, v1, T, H, Wd, v3, P, s) Furthermore, the relative error f(E2) is given by the following formula. f(E2) = d * (f1 + f2 + f3) / (x + y + z)
[0075] Here, in order to perform an appropriate inspection, it is necessary to satisfy the condition that the relative error f(E2) < L2. L2 is a pre-set value. From this condition, the maximum inspection range d2max of the railway robot 7 can be calculated as follows. d2max = L2 * (x + y + z) / (f1 + f2 + f3)
[0076] Next, the calibration of the inspection range of the drone 9 will be described. This calculates the longest distance that can be inspected for the sensors installed in the drone 9 under the current environmental conditions. The detection error of the size (a, b, c) of the railway robot 7 by the drone 9 is given as follows. f4, f5, and f6 are pre-defined functions. They may be common. Detection error a' = d * f4(v2, v1, T, H, Wd, v3, P, s, M) Detection error b' = d * f5(v2, v1, T, H, Wd, v3, P, s, M) Detection error c' = d * f6(v2, v1, T, H, Wd, v3, P, s, M) Furthermore, the relative error f(E) is given by the following formula. f(E) = d * (f1 + f2 + f3) / (a + b + c)
[0077] Here, in order to perform an appropriate inspection, it is necessary that the condition of relative error f(E) < L be satisfied. L is a preset value. From this condition, the maximum inspection range dmax of the drone 9 can be calculated as follows. dmax = L * (a + b + c) / (f4 + f5 + f6)
[0078] As described above, by measuring an object of known size, the maximum inspection range can be calculated more accurately. Also, by using the railway robot 7 and the drone 9, the maximum inspection range at any location can be accurately calculated. In other examples, the maximum inspection range may be estimated without performing actual measurements. For example, the maximum inspection range can be estimated based on environmental conditions and sensor performance. Note that the variables of the function for the calibration of each of the drone and the robot are not limited to the above variables, and some of these may be omitted or different variables may be added.
[0079] FIG. 7 shows an example where the maintenance target 303 exceeds the inspection range of the railway robot 7 when RTmin > d2max. RTmin is the minimum distance that the railway robot 7 can approach the target 303, and d2max is the maximum distance to the target 303 at the allowable minimum inspection resolution. In this example, the sensor mounted on the railway robot 7 cannot perform an inspection. The railway robot 7 uses the drone 9 to inspect the target 303.
[0080] FIG. 8 shows an example where the maintenance target 303 exceeds the inspection range of the drone 9 when DTmin > dmax. DTmin is the minimum distance that the drone 9 can approach the target 303, and dmax is the maximum distance to the target 303 at the minimum value of the allowable inspection resolution. In this example, the sensor mounted on the drone 9 cannot perform a sufficient inspection. The railway robot 7 customizes the drone 9 with an appropriate sensor or in an appropriate shape (including size) as described above.
[0081] 9 shows the conditions for sensor customization of the drone 9. An obstacle 305 exists in front of the maintenance target 303. Here, if dmax>DTmin is established, the drone 9 can photograph (inspect) the target 303 without customizing the sensor. dmax is the maximum distance to the target 303 at the minimum allowable inspection resolution of the sensor of the drone 9, and DTmin is the minimum distance at which the drone 9 can approach the target 303.
[0082] On the other hand, if dmax≦DTmin, the rail robot 7 needs to customize the sensor of the drone 9 to finish the inspection. The rail robot 7 searches for a backup sensor of the drone 9 for customization. The new limit value dmax2 of the sensor to be searched is one that satisfies dmax2>DTmin under the site conditions f(v1, v2, T, H, Wd, v3, P, s, M).
[0083] FIG. 10 shows the conditions for customizing the shape (including size) of the drone 9. An obstacle 305 exists in front of the target 303. The size of the passage tunnel to the target 303 is (l, m, n). The tunnel size (l, m, n) is a value detected by the sensor of the drone 9. If the drone size (x, y, z) satisfies x<(l+2*l'), y<(m+2*m'), z<(n+2*n'), the drone 9 can pass through the tunnel of the obstacle 305 without customization. Note that x, y, and z are the maximum sizes of the drone 9 in the directions of l, m, and n, respectively. l', m', and n' are the detection errors of the sensors set in advance for l, m, and n.
[0084] If the above conditions are not met, it is necessary to customize the shape of the drone 9 in order to pass through the obstacle 305. The rail robot 7 searches for a backup drone part for shape customization that meets the following conditions: The new drone size (X, Y, Z) is required to meet the following conditions: X<(l+2*l´), Y<(m+2*m´), Z<(n+2*n´)
[0085] FIG. 11 shows an example of a maintenance schedule for the rail robot 7 and drone 9. The dynamic maintenance scheduling system 3 allocates tasks to the rail robot 7 and its mounted drone 9 before the rail robot 7 starts moving. The allocation of tasks is based on (1) the distance from the rail robot 7 to the target, (2) real-time weather conditions and weather forecasts, (3) the maximum effective inspection range of the rail robot 7's sensors, (4) the maximum working range of the robot arm implemented on the rail robot 7, and (4) drones with different functions and shapes.
[0086] The dynamic maintenance scheduling system 3 separates all maintenance tasks 351 and assigns each separated task 352 to an assignment destination 353 of the rail robot 7 or the drone 9. The dynamic maintenance scheduling system 3 assigns each task to the rail robot 7 or the drone 9 whose target of the task is included in the maintenance range. For example, the dynamic maintenance scheduling system 3 assigns tasks of targets within the maintenance range of the rail robot 7 to the rail robot 7, and assigns other tasks to the drone 9.
[0087] Each task requires a time duration 354 to be performed. The time duration 354 can be calculated based on the target location, the performance of the maintenance entity, the predicted environmental conditions, and the maintenance content. The time required for each rail robot 7 (including the onboard drone 9) to complete all tasks is the estimated total time 361.
[0088] The dynamic maintenance scheduling system 3 determines the task schedule so that the time after the estimated total time 361 has elapsed since the start of the task for each railway robot does not exceed the completion time specified by the railway schedule management system 10.
[0089] In actual task execution, as described above, it may be necessary to change the task execution subject from the rail robot 7 to the drone 9, or to customize the drone 9. The time required for such processing is the additional task time 362. The time estimated to be required for the rail robot 7 to complete all tasks is the total time 363, which is the sum of the estimated total time 361 and the predicted value of the additional task time 362.
[0090] The railway schedule management system 10 may determine the task schedule in consideration of the occurrence of additional tasks so that the latest predicted task end time of all the railway robots 7 is a preset predetermined time before the designated completion time. The predetermined time is set based on the predicted value of the additional task time 362.
[0091] The additional task time 362 is based on (1) new tasks during the originally specified maintenance (inspection and repair) and (2) the performance of the executing entity and the current environmental conditions. The maintenance scope calibration system 209 of the rail robot 7 calculates the estimated additional task time when an additional task is required. Depending on the type of additional task, a method for calculating the additional task time is preset. The calculation method includes variables such as those described above.
[0092] For example, the time required to remove and install each drone part may be preset. The calculation of the calibration time may be calculated based on the drone's flight speed and environmental conditions. The maintenance scope calibration system 209 calculates an estimate of the additional task time 362 if additional tasks are required.
[0093] The maintenance range calibration system 209 refers to the task schedule and calculates the sum of the estimated values of the time required to execute the remaining tasks and the time for each additional task. The maintenance range calibration system 209 may estimate the completion time of the remaining tasks from this value and the current time, and compare it with the specified completion time of all tasks to determine whether or not to execute the additional tasks and the specified tasks that require the additional tasks. In another example, a priority may be assigned to the specified tasks. The maintenance range calibration system 209 may skip the specified tasks with a low priority so that the final destination position can be reached by the specified time for all tasks.
[0094] 12 shows a flowchart of the operation of the rail robot 7 to carry the drone 9 for a maintenance task. In step S151, the maintenance tasks for the rail robot 7 and the drone 9 are obtained. In step S502, the takeoff and landing points of the drone for the maintenance task by the drone 9 are set. These are specified in the task schedule.
[0095] In step S153, it is confirmed whether a maintenance task for the rail robot 7 exists. If a maintenance task exists (503: YES), in step S155, the speed of the rail robot 7 up to the takeoff point of the drone 9 is set to the inspection speed. If a maintenance task for the rail robot 7 does not exist (S153: NO), in step S154, the speed of the rail robot 7 up to the takeoff point of the drone 9 is set to the maximum speed.
[0096] In step S156, the rail robot 7 moves to the takeoff point of the drone 9, and the drone takes off. In step S157, it is confirmed whether the landing point of the drone 9 is the same as the takeoff point. If they are different (S157: NO), in step S158, the rail robot 7 moves to a new landing point. If the takeoff point and the landing point are the same (S157: YES), in step S159, the rail robot 7 waits at the same point until the drone 9 lands, and then moves to the next point. The next point is, for example, the base, which is the standard waiting position of the rail robot 7.
[0097] 13 shows the scheduling of a maintenance task with a time limit. In step S201, the dynamic maintenance scheduling system 3 obtains a maintenance task and a time limit from the maintenance task management system 2. In step S202, the dynamic maintenance scheduling system 3 calculates the number of rail robots and drones required, schedules the task based on the current weather conditions, and calculates the time required to complete the maintenance task in step S203.
[0098] For example, the dynamic maintenance scheduling system 3 divides the tasks to be performed into multiple groups according to the target positions. For example, tasks whose target positions are within a predetermined range are included in one group. The dynamic maintenance scheduling system 3 assigns one or more rail robots 7 to each task group. Assuming that different tasks in each group can be performed simultaneously by different drones or rail robots, the dynamic maintenance scheduling system 3 assigns tasks to rail robots based on the number of drones mounted on each rail robot 7 so that the total task completion time is minimized. At this time, the entity that will perform the tasks is selected according to, for example, weather conditions.
[0099] In step S204, the dynamic maintenance scheduling system 3 checks whether the required time calculated in step S203 exceeds the time limit acquired in step S201. If the required time does not exceed the time limit (S204: NO), in step S206, the dynamic maintenance scheduling system 3 ends the scheduling of the railway robot and the drone. If the required time exceeds the time limit (S204: NO), in step S205, it is checked whether all maintenance tasks have the highest priority (whether they have the same priority).
[0100] If any of the tasks has a low priority (S205: NO), in step S207, the dynamic maintenance scheduling system 3 removes the task with the lowest priority, and in step S208, adds the removed task to a waiting list. This allows the maintenance to be completed within the specified time, with higher priority tasks given priority. Returning to step S202, the dynamic maintenance scheduling system 3 calculates and checks the estimated time for the remaining tasks (S203). If all tasks have the same priority (S205: YES), in step S209, the dynamic maintenance scheduling system 3 notifies the maintenance task management system 2 that a delay will occur, along with the expected value of the delay.
[0101] 14 is a conceptual diagram showing a state in which a railroad robot 7 is carrying a drone 9. The railroad robot 7 runs on wheels on a track 803 and performs maintenance using sensors and a robot arm mounted thereon. The drone 9 flies to perform maintenance on a target 805 located beyond the maintenance range of the railroad robot 7.
[0102] FIG. 15 shows a schematic example of customization of a drone by a railroad robot. The railroad robot 7 can customize the sensor of the drone 9A, for example, by changing the visible light camera to an infrared camera, and can configure a new drone 9B. The railroad robot 7 can also customize the shape (including size) of the drone 9A, for example, by miniaturizing the drone 9A to pass through obstacles, and can configure a new drone 9C.
[0103] Figure 16 shows an example of targets inside and outside the maintenance range of the rail robot 7. In the example of Figure 16, target 1 and target 2 are outside the maintenance range of the rail robot 7, while the other targets 3, 4, and 5 are within the maintenance range of the rail robot 7. The rail robot 7 directly maintains targets 3, 4, and 5.
[0104] FIG. 17 shows drone 9 performing maintenance on targets 1 and 2, which are outside the maintenance range of rail robot 7.
[0105] Fig. 18 shows an example in which the environment changes to fog or rain, and the maintenance range of the rail robot 7 changes in response to the change in environment. Compared to the examples shown in Figs. 16 and 17, the target 5 is outside the maintenance range of the rail robot 7. The other targets are similar to the examples shown in Figs. 16 and 17.
[0106] 19 shows an example in which the environment changes to fog or rain and the maintenance range of the drone 9 changes accordingly. Targets 1 and 5 are within the maintenance range of the drone 9, but target 2 is outside the maintenance range of the drone 9. Targets 3 and 4 are within the maintenance range of the rail robot 7.
[0107] 20 shows an example of customizing a drone due to a change in the environment to fog or rain. The railroad robot 7 customizes one drone 9D into a drone 9E with a different configuration for the target 2.
[0108] 21 shows an example of maintenance being performed by a customized drone due to a change in the environment to fog or rain. Target 1 and target 5 are within the maintenance range of a non-customized drone 9E. Target 2 is within the maintenance range of a customized drone 9E. Target 3 and target 4 are within the maintenance range of a railroad robot 7.
[0109] Here is a concrete use case example: A railway bridge has six small cracks. These cracks were presumed to be safe for train operation. However, after an earthquake, the railway operator needs to inspect the railway bridge before allowing trains to run on it. However, because the railway bridge is in a remote location, it is difficult to send enough personnel and equipment to inspect it in time for trains to run.
[0110] Five cracks are outside the maintenance range of the railway robot. One railway robot with four drones arrives at the railway bridge. The railway robot sets up the four drones based on the new calibrated inspection range to inspect the four cracks. Four cracks are inspected and the third crack is found to be 0.2 cm wider and 2 cm longer. However, due to the unexpected fog, none of the drones can make a clear measurement on the fifth crack.
[0111] So, Rail Robot customizes one drone to a smaller size so that it can pass through the hole and get closer to the crack. The crack is found to be 5cm wider and 45cm longer. The inspection results show that it is unsafe to run trains on this railway bridge.
[0112] The present invention is not limited to the above-mentioned embodiment, but includes various modified examples. For example, the above-mentioned embodiment has been described in detail to clearly explain the present invention, and is not necessarily limited to those including all of the configurations described. It is also possible to replace a part of the configuration of one embodiment with the configuration of another embodiment, and it is also possible to add the configuration of another embodiment to the configuration of one embodiment. It is also possible to add, delete, or replace a part of the configuration of each embodiment with another configuration.
[0113] In addition, the above-mentioned configurations, functions, processing units, etc. may be realized in part or in whole by hardware, for example, by designing them as integrated circuits. In addition, the above-mentioned configurations, functions, etc. may be realized in software by a processor interpreting and executing a program that realizes each function. Information such as the program, table, file, etc. that realizes each function can be stored in a memory, a recording device such as a hard disk or SSD (Solid State Drive), or a recording medium such as an IC card or SD card.
[0114] In addition, the control lines and information lines shown are those that are considered necessary for the explanation, and not all control lines and information lines in the product are necessarily shown. In reality, it can be considered that almost all components are connected to each other. [Explanation of symbols]
[0115] 1 Maintenance Management System 2. Maintenance Task Management System 3 Dynamic Maintenance Scheduling System 4. Railway Robot and Drone Management Database 5 Real-time inspection system 7. Railway Robot 9. Drone 101 Processor 102 DRAM 104 Auxiliary storage 201 Railway Robot System 202 Railway Robot Management System 203 Drone Management System 202 Railway Robot Management System 204 Database 205 Communication Systems 206 Robot Arm Control System 207 Monitoring and Analysis System 208 Drone Information Management System 209 Maintenance Range Calibration System 210 Drone Customization System 211 Drone Control System
Claims
1. A robot for performing maintenance on equipment, one or more computing devices; one or more storage devices; The robot includes one or more drones that perform maintenance; and a backup component for the one or more drones; The one or more storage devices include a task schedule that indicates one or more maintenance tasks; and configuration information for each of the one or more drones; The task schedule indicates a maintenance target location and a maintenance content, The one or more computing devices, flying a first drone of the one or more drones toward a first target of a first maintenance task; determining whether the first drone can perform maintenance of the first target based on a distance between the first drone and the first target determined based on the measurement result of the first drone and a maintenance range of the first drone indicated by the configuration information; When the robot determines that the first drone is unable to perform maintenance on the first target, the robot customizes the first drone with the backup part.
2. The robot according to claim 1 , The one or more computing devices, determining a maintenance possible range of the robot based on environmental conditions; A robot that assigns the second task to a drone selected from the one or more drones when the maintenance range of the robot is smaller than a distance to a target of the second task specified for the robot by the task schedule.
3. The robot according to claim 1 , The one or more computing devices, acquiring a first image of an object of known size with a first sensor of the robot; Estimating a size of the object based on the first image; determining an inspection range of the first sensor based on a difference between the estimated size of the object and the known size; If the inspection range is smaller than a distance to a target of a third task specified for the robot by the task schedule, the robot assigns the third task to a drone selected from the one or more drones.
4. The robot according to claim 3, the object is a flying drone; The configuration information indicates a known size of the flying drone; The one or more computing devices determine an inspection range of the first sensor based on a difference between an estimated size of the flying drone and the known size indicated by the configuration information and current environmental conditions.
5. The robot according to claim 1 , The one or more computing devices, acquiring a second image of an object of known size by a second sensor of the first drone; Estimating a size of the object based on the second image; determining an inspection range of the second sensor based on a difference between the estimated size of the object and the known size; If the inspection range of the second sensor is smaller than the distance between the first drone and the first target, the robot determines that inspection of the first target is impossible and decides to replace the second sensor with another sensor.
6. The robot according to claim 1 , The one or more computing devices, acquiring a third image captured by the first drone of an obstacle on a flight path to the first target; determining a size of a tunnel through the obstacle based on the third image; The robot customizes a shape of the first drone based on a size of the tunnel.
7. The robot according to claim 5, The one or more computing devices, Comparing the inspection range of the other sensor with the distance from the first drone to the first target, which is determined by an obstacle that exists in a flight path to the first target and is impassable by the first drone; When the inspection range of the other sensor is smaller than the distance from the first drone to the first target, a third image of an obstacle present on a flight path to the first target is acquired by the first drone; determining a size of a tunnel through the obstacle based on the third image; The robot customizes a shape of the first drone based on a size of the tunnel.
8. The robot according to claim 1 ; a scheduling system for generating the task schedule; The scheduling system comprises: A maintenance system that selects one or more tasks from the plurality of maintenance tasks to be included in the task schedule based on the priority of the plurality of maintenance tasks and a specified time for completion of all tasks for the robot.
9. 1. A method for controlling a maintenance system of a facility, comprising: The maintenance system includes: One or more drones performing maintenance; a robot that transports the one or more drones and backup parts for the one or more drones and performs maintenance; The control system includes: a task schedule that indicates one or more maintenance tasks; Configuration information for each of the one or more drones; The task schedule indicates a maintenance target location and a maintenance content, The method further comprises the step of: flying a first drone of the one or more drones toward a first target of a first maintenance task; determining whether the first drone can perform maintenance of the first target based on a distance between the first drone and the first target determined based on the measurement result of the first drone and a maintenance range of the first drone indicated by the configuration information; and customizing the first drone using the backup part upon determining that the first drone is unable to perform maintenance on the first target.
Citation Information
Patent Citations
Flight body, living body search system, living body search method, program, and recording medium
JP2019059452A
Management device, unmanned flight body and program
JP2020006916A
Mobile fulfillment centers with intermodal carriers and unmanned aerial vehicles
JP2020514173A
Robotic repair or maintenance of an asset
US10518411B2
Ground movement system plugin for VTOL uavs
US20180044000A1
