Control server, information processing system, traveling body, traveling body control method, and recording medium
By switching the path to obtain the status information of different monitoring objects when an abnormality is detected by the traveling body, the problem of time consuming to determine the cause of the abnormality in the prior art is solved, and fast and reliable abnormality cause identification is achieved.
Patent Information
- Application Number
- CN202380028601.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2023-01-27
- Filing Date
- 2023-03-16
- Publication Date
- 2024-11-19
AI Technical Summary
The prior art is difficult to quickly and reliably determine the cause of abnormalities detected by the traveling body, resulting in the need of multiple analytical actions, time-consuming and labor-consuming.
By instructing the traveling body to travel on the first path, the status information of the first monitoring object is obtained. If an exception is detected, switch to a second path different from the first path, and the status information of the second monitoring object is obtained to determine the cause of the exception. .
It realizes rapid and reliable determination of the cause of abnormalities, shortens inspection time and improves efficiency.
Smart Images

Figure CN118984981A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to a control server, an information processing system, a moving object, a control method of the moving object, and a recording medium for controlling a moving object. Background Art
[0002] In order to carry out unattended transport and inspection of objects, a moving body that moves within a predetermined area has been introduced. Such a moving body has a sensor and can detect the current state or the occurrence of an abnormality based on the detection results of the sensor.
[0003] Sometimes the cause of an abnormality cannot be determined based on the sensor's detection results. For example, the sensor may be faulty. In this case, in order to determine the cause of the abnormality, the actions before and after the abnormality are reproduced and analyzed multiple times. However, determining the cause requires time and labor.
[0004] Therefore, a technology for quickly and reliably determining the cause of an abnormality has been proposed. For example, images of a monitored object (monitored object) and its vicinity are repeatedly captured, and the captured images are stored in a time series. When an abnormality is detected, the images are read and replayed from a storage location after a predetermined time from the time of detection (for example, see Patent Document 1).
[0005] Citation List
[0006] Patent Literature
[0007] [Patent Document 1]: Japanese Patent Application Laid-Open No. 9-182057 Summary of the invention
[0008] Technical issues
[0009] In order to solve the above-mentioned inconvenience, an object of the present disclosure is to provide information for facilitating determination of the cause of abnormality.
[0010] Solutions to the problem
[0011] In one aspect, a control server for controlling a moving body includes: an instruction unit that instructs the moving body to move on a first path and obtains state information of a first monitored object on the first path. Based on a determination that an abnormality has occurred in the state of the first monitored object from the obtained state information, the instruction unit instructs the moving body to move on a second path different from the first path and obtains state information of a second monitored object different from the first monitored object located on the second path.
[0012] In another aspect, an information processing system includes the above-mentioned control server, and one or more moving objects controlled by the control server.
[0013] In another aspect, the moving body includes an instruction unit to instruct the moving body to move on a first path and obtain status information of a first monitored object on the first path. Based on a determination that an abnormality has occurred in the status of the first monitored object from the obtained status information, the instruction unit instructs the moving body to move on a second path different from the first path and obtains status information of a second monitored object different from the first monitored object located on the second path.
[0014] Another aspect relates to a method for controlling a moving body. The method includes: instructing the moving body to move on a first path, and obtaining status information of a first monitored object on the first path, and according to a determination that an abnormality has occurred in the status of the first monitored object from the obtained status information, instructing the moving body to move on a second path different from the first path, and obtaining status information of a second monitored object different from the first monitored object located on the second path.
[0015] In another aspect, the recording medium stores a plurality of program codes which, when executed by one or more processors, cause the processors to perform the above-described method.
[0016] Effects of the Invention
[0017] According to aspects of the present disclosure, information is provided to facilitate determination of the cause of an abnormality. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] A more complete understanding of the embodiments of the present disclosure and many of its attendant advantages and features can be readily obtained and understood from the following detailed description taken in conjunction with the accompanying drawings.
[0019] Figure 1 FIG. 1 is a diagram showing the configuration of an information processing system according to a first embodiment.
[0020] Figure 2 The figure shows an example of a schematic structure of a traveling robot as a traveling body involved in the embodiment.
[0021] Figure 3 1 is a block diagram showing an example of the hardware configuration of the traveling robot according to the embodiment.
[0022] Figure 4 FIG. 1 is a block diagram showing an example of a hardware configuration of a management server as a control system according to the embodiment.
[0023] Figure 5 FIG. 1 is a block diagram showing an example of the functional configuration of the traveling robot, the management server, and the communication terminal according to the first embodiment.
[0024] Figure 6 The illustrated diagram is a first scenario diagram of an information processing system according to an embodiment of the present invention.
[0025] Figure 7 FIG. 1 is a diagram showing an operation of controlling a traveling robot to travel at a location and registering candidate destinations that the traveling robot passes on an autonomous traveling path according to the first embodiment.
[0026] Figure 8 Shown is an example diagram of via points and check points.
[0027] Fig. 9 The following is an example of a routing point management table.
[0028] Fig.10 The following is an example diagram of the inspection object management table.
[0029] Fig.11 The figure shows an example of a path information management table.
[0030] Fig.12 FIG. 1 is a sequence diagram showing an example of processing executed by the information processing system according to the first embodiment.
[0031] Fig.13 FIG. 1 is a diagram showing an inspection operation for an abnormal situation performed at a designated inspection point on an abnormal situation path according to the first embodiment.
[0032] Fig.14 The flowchart shown is an example of the processing executed by the traveling robot in the inspection operation using the first method.
[0033] Fig.15 The flowchart shown is an example of the process executed by the management server in the inspection operation using the first method.
[0034] Fig.16 The figure shows an example of an abnormality information table used to determine whether the inspection object has abnormality.
[0035] Fig.17 Shown is an example diagram of a path database (DB) for abnormal situations based on the first method.
[0036] Fig.18 The flowchart shown is an example of the processing executed by the traveling robot in the inspection operation using the second method.
[0037] Fig.19 The flowchart shown is an example of the process executed by the management server in the inspection operation using the second method.
[0038] Fig. 20The figure shows another example of an abnormality information table used to determine whether the inspection object has an abnormality.
[0039] Fig.21 The figure shows an example of a path DB for abnormal situations based on the first method.
[0040] Fig. 22 The figure shows an example of setting up a monitoring area with multiple traveling robots and performing inspections.
[0041] Fig.23 The flowchart showing an example of processing executed by the traveling robot during the inspection operation of the traveling robot using the mechanism having the presentation information according to the first embodiment.
[0042] Fig.24 The flowchart shows an example of processing executed by the management server in the inspection operation of the traveling robot using the mechanism having the presentation information according to the first embodiment.
[0043] Fig.25 The figure shows another example of an abnormality information table used to determine whether the inspection object has an abnormality.
[0044] Fig.26 FIG. 2 is another sequence diagram showing another example of the process executed by the information processing system according to the first embodiment.
[0045] Fig.27A The figure shows an example of an information prompt of a moving robot.
[0046] Fig.27B The figure shows an example of an information prompt of a moving robot.
[0047] Fig.27C The figure shows an example of information prompts for a moving robot.
[0048] Fig.28 The figure shows an example of the inspection action when there is an impassable part in the path.
[0049] Fig.29 The figure shows an example of a path search method according to an embodiment.
[0050] Fig.30 The figure shows an example of the inspection result DB stored in the management server.
[0051] Fig.31 FIG. 1 is a flowchart showing an example of a route search according to the first embodiment.
[0052] Fig.32 Shown is an example of a graph structure used in path searching.
[0053] Fig.33 FIG. 1 is a flowchart showing an example of a route search process executed by the information processing system according to the first embodiment.
[0054] Fig.34 FIG. 2 is a diagram showing the configuration of an information processing system according to a second embodiment.
[0055] Fig.35 FIG. 1 is a block diagram showing a functional configuration of a traveling robot according to a second embodiment.
[0056] Fig.36 FIG. 1 is a flowchart showing an example of an inspection operation using the first method in the second embodiment.
[0057] Fig.37 FIG. 1 is a flowchart showing an example of an inspection operation using the second method in the second embodiment.
[0058] Fig.38 The flowchart shown is another example of the inspection operation of the traveling robot having the information presenting mechanism.
[0059] Fig.39 The diagram shows an example of an abnormality determination process in which a check point, image comparison information, and an action are stored for each abnormality ID in association with an abnormality path ID.
[0060] Fig.40 FIG. 2 is a diagram showing the configuration of an information processing system according to a third embodiment.
[0061] Fig.41 Shown is a block diagram showing the functional configuration of a traveling robot, a management server, and a communication terminal according to a third embodiment.
[0062] Fig.42 FIG. 2 is a diagram showing an example of degradation information managed in the degradation information management table in the degradation information DB.
[0063] Fig.43 FIG. 1 is a diagram showing an example of a process of determining a degradation state using degradation information.
[0064] Fig.44 The figure shows an example of a screen displaying a text message related to degradation.
[0065] The accompanying drawings are intended to illustrate the embodiments of the present disclosure and should not be interpreted as limiting the scope thereof. Unless explicitly stated, the accompanying drawings should not be considered to be drawn to scale. In addition, in all the drawings, the same or similar reference numerals represent the same or similar components. DETAILED DESCRIPTION
[0066] When describing the embodiments shown in the drawings, specific words are used for clarity. However, the disclosure of this specification is not intended to be limited to the specific words selected, and it should be understood that each specific element includes all technical equivalents having similar functions, operating in a similar manner and achieving similar results.
[0067] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.
[0068] Although the embodiments of the present disclosure are described below, the embodiments of the present invention are not limited to the following embodiments.
[0069] Figure 1 FIG. 1 is a diagram showing the configuration of an information processing system according to the present embodiment. Figure 1 The information processing system 10 shown sets a prescribed area as a monitoring area (location of an object). The information processing system 10 includes a traveling robot 11 as a traveling body traveling in the monitoring area, and a management server 12 as a control system (or control server) for controlling the traveling robot 11. The traveling robot 11 and the management server 12 are connected to a communication network 13, and communicate with each other via the communication network 13.
[0070] The communication network 13 includes the Internet, a mobile communication network, and a local area network (LAN), and may be wired or wireless.
[0071] The traveling robot 11 is disposed in the monitoring area and is capable of autonomously traveling in the monitoring area. Autonomous traveling may be autonomous movement on a specified path in the monitoring area, autonomous movement in the monitoring area using a technique such as line tracking, or autonomous movement using the result of machine learning of the paths that the traveling robot 11 has moved in the past. The traveling robot 11 may be manually operated by an operator.
[0072] The traveling robot 11 includes various sensors and performs prescribed tasks such as inspection and maintenance. The traveling robot 11 may include an arm capable of grasping an object and may perform tasks such as transportation and light work. The traveling robot 11 may be any robot capable of autonomous traveling, such as a car, a drone, a multicopter, or an unmanned aerial vehicle.
[0073] The traveling robot 11 includes a detection device (detection means) to detect the state of the monitored object, which is used for monitoring the monitoring area. As the detection device, for example, a camera device (camera means), a gas sensor (gas detection means), a recording device (recording means), etc. can be cited. The camera device photographs the monitored object. When the monitored object is a water or gas meter, a water or gas flow meter, or a liquid level meter, the camera device captures an image of the scale or display value. The camera device can also capture images of holes, obstacles, etc. on the road as surrounding state information representing the state surrounding the traveling robot 11. In the case where the monitored object is a pipe, a tank, etc., the gas sensor measures, for example, the concentration of harmful gases leaked from the pipe, the tank, etc. as the state around the traveling robot 11. The recording device records the sound of the mechanical action of the equipment involving the action of valves, pumps, compressors, etc. The state of the monitored object may be the temperature or humidity at a specified position, and the traveling robot 11 may include a temperature and humidity sensor as a detection device.
[0074] The monitoring area is an area where the traveling robot 11 is set (also called the location of the object, or simply the location). As the monitoring area, for example, outdoor areas such as business premises, factories, chemical plants, construction sites, substations, farms, fields, cultivated land, disaster sites, and indoor areas such as offices, schools, factories, warehouses, commercial facilities, hospitals, nursing facilities, etc. The monitoring area can be any place where the traveling robot 11 is required to perform tasks that were once done by people. The number of traveling robots 11 monitoring the monitoring area is not limited to one, and multiple traveling robots 11 can cooperate to monitor the monitoring area. In this case, for example, traveling robot A monitors the first area of the monitoring area, traveling robot B monitors its second area, and traveling robot C monitors its third area. In the following description, it is assumed that the traveling robot 11 includes multiple traveling wheels and a camera device (camera) as a detection device.
[0075] The management server 12 instructs the traveling robot 11 to capture an image of the monitored object while traveling on the first path via the communication network 13. The first path is the path that the traveling robot 11 follows along the first path under normal conditions (a path for normal conditions). The number of monitored objects is not limited to one, but may be multiple. The management server 12 receives image data of the monitored object captured by the traveling robot 11. The management server 12 analyzes the received image data and determines whether there is any abnormality. In the case where the monitored object is a flow meter and the normal flow rate is 1 to 10 m3 / s, if the flow rate is outside this range (for example, 0.5 m3 / s), it is determined that an abnormality exists.
[0076] In addition, even when an abnormality is detected from the image data of a specific monitored object (first monitored object), the cause of the abnormality cannot be determined only from the image data of the specific monitored object. Therefore, the management server 12 instructs the traveling robot 11 to switch the path to a second path different from the first path. Then, the management server 12 instructs the traveling robot 11 to capture an image of a specified part (second monitored object) related to the state of the specific monitored object on the second path. In addition, the first path and the second path may also partially overlap each other. The first path and the second path may be different from each other in only one or both of the starting point and the end point.
[0077] As in the above example, when the flow meter (first monitoring object) indicates a flow rate lower than the normal flow rate, it is estimated that a blockage has occurred in some part. Therefore, the path used for normal monitoring (first path) is switched to the path used for abnormal situations (second path), and an image of the valve (second monitoring object) as a designated part can be captured in order to confirm the valve failure. In addition, this is just an example, and parts related to other reasons can also be captured, and the sound of mechanical operation can also be recorded.
[0078] A communication terminal 14 such as a laptop computer, a personal computer (PC), a tablet computer, etc. operated by an operator is connected to the communication network 13. The communication terminal 14 is set at the management location. The communication terminal 14 communicates with the management server 12 via the communication network 13, and can display an image captured by the traveling robot 11 and received from the management server 12.
[0079] The communication terminal 14 receives the image data of the designated part obtained by the traveling robot 11 that has switched to the second path, which is instructed by the management server 12 for detecting the abnormality. Then, the communication terminal 14 can display the image represented by the image data. In the case of referring to the image of the valve as the designated part, when the opening of the valve is smaller than the normal opening, it can be detected that the valve closure is the cause of the abnormality of the decrease in flow rate represented by the flow meter.
[0080] Thus, without the intervention of the operator, since the traveling robot 11 captures and provides an image of the portion that the operator wishes to observe, the operator can identify the cause of the abnormality in the monitoring area and can quickly take the best countermeasure. If image capture is performed for all portions to be inspected during a normal inspection, the inspection time will be long. In contrast, in the present embodiment, since image capture of a specified portion is performed when an abnormality is found, the inspection points can be narrowed in the normal inspection, thereby shortening the inspection time.
[0081] exist Figure 1 In the illustrated example, the motion of the traveling robot 11 is controlled by the management server 12 , but a part of the control of the motion of the traveling robot 11 may be performed in the traveling robot 11 .
[0082] Figure 2 1 is an example diagram of the structure of a traveling robot. The traveling robot 11 includes a housing 21 having a control unit 20, a camera 22, a support member 23, a moving mechanism 24, and a prompting mechanism 25. The control unit 20 controls the processing or action of the traveling robot 11. The camera 22 captures an object located at a location where the traveling robot 11 is set or a landscape at the location as a subject, and generates a captured image. The camera 22 may be a digital camera capable of generating a planar image, such as a digital single-lens reflex camera or a compact digital camera, or a special image capturing device capable of capturing a spherical (360°) panoramic image and generating a captured image.
[0083] The special image capturing device is, for example, a spherical image capturing device that captures a subject to generate two hemispherical images and synthesizes the two hemispherical images into a spherical panoramic image. Alternatively, the special image capturing device may be a wide-angle camera or a stereo camera that is capable of acquiring a wide-angle image having a viewing angle equal to or greater than a predetermined value. As an alternative to using a special image capturing device, a conventional digital camera that captures planar images may be used. For example, a conventional digital camera captures images while rotating so as to cover all directions of the location. The captured images are then synthesized to generate a spherical image. In either method, the special image capturing device synthesizes a plurality of captured images through image processing to generate a spherical image.
[0084] The image captured by the camera 22 may be a still image, an animation, or both a still image and an animation. The camera 22 may record sound when capturing an image, and acquire sound data together with the image data. The camera 22 has a pan-tilt-zoom (PTZ: full range of movement and lens magnification, zoom control) function for capturing a wide range through one device. Panning is a function of moving the orientation of the lens of a camera (camera) in the horizontal direction (left and right). Tilt is a function of moving the orientation of the lens of a camera in the vertical direction (up and down). Zoom is a function of changing the visual distance from the subject and increasing the viewing angle. Thus, even if the camera 22 is not located at the same height or the same lateral position as the subject, the camera 22 can make the lens face the subject by panning and tilting and can capture the image of the subject. Even if the subject is located in a deeper position, the camera 22 can capture the image of the subject in a desired size by, for example, enlarging.
[0085] The support member 23 is a member in which the camera 22 is installed in the housing 21 of the traveling robot 11. The support member 23 may be, for example, a rod-shaped rod fixed to the housing 21 to extend in the vertical direction. The support member 23 may be a movable member so as to adjust the image capturing direction (direction of the lens) and position (height of the lens) of the camera 22.
[0086] The moving mechanism 24 (travel means) is a unit for moving the traveling robot 11. The moving mechanism 24 includes wheels, a traveling motor, a traveling encoder, a steering motor, and a steering encoder, and can be called a drive system. The traveling motor causes the traveling robot 11 to travel. The traveling encoder detects the rotation direction, position, and rotation speed of the traveling motor. The steering motor changes the direction of the traveling robot 11. The steering encoder detects the rotation direction, position, and rotation speed of the steering motor. The rotation direction and rotation speed detected by the traveling encoder and the steering motor are input to the control unit 20, and the traveling motor and the traveling encoder are controlled to obtain an appropriate traveling speed, direction, etc.
[0087] The prompting mechanism 25 is used as a prompting means (prompting unit) for prompting information about the abnormality when it is determined that there is an abnormality around the traveling robot 11. The control unit 20 determines whether there is an abnormality around the traveling robot 11 based on the image taken by the camera device 22. The traveling robot 11 may include a gas sensor, and the control unit 20 may determine whether there is an abnormality based on the concentration of harmful gases, etc. detected by the gas sensor. When the control unit 20 determines that there is an abnormality and determines that a prompt is required through the prompting mechanism 25, the control unit 20 instructs the prompting mechanism 25 to prompt information about the abnormality. In response to the instruction of the control unit 20, the prompting mechanism 25, for example, raises a flag to prompt the surrounding personnel with information about the abnormality.
[0088] Figure 3 Although the traveling robot 11 includes the control section 20 in the housing 21 , the control section 20 may be provided outside the traveling robot 11 or may be provided as a separate device from the traveling robot 11 .
[0089] The control section 20 includes a central processing unit (CPU) 30, a read-only memory (ROM) 31, a random access memory (RAM) 32, a hard disk drive (HDD) controller 33, an HD 34, a media interface (I / F) 35, an input / output I / F 36, a sound input / output I / F 37, a network I / F 38, a short-distance communication circuit 39, an antenna 40 of the short-distance communication circuit 39, an external device I / F 41, and a bus 42. The HDD 33 controls the HDD having the HD 34.
[0090] The CPU 30 controls the entire motion of the traveling robot 11. The CPU 30 is a processor that loads a program or data stored in the ROM 31 or the HD 34 onto the RAM 32 and executes processing to realize the functions of the traveling robot 11.
[0091] ROM 31 is a nonvolatile memory that keeps the storage of programs or data even after the power is turned off. RAM 32 is used by CPU 30 as a work area for executing programs to perform various processes. HDD controller 33 controls reading and writing (storage) of data to HD 34 under the control of CPU 30. HD 34 stores various data such as programs.
[0092] The medium I / F 35 controls the reading or writing of data to the recording medium 43, such as a universal serial bus (USB) memory, a memory card, an optical disc or a flash memory. The input / output I / F 36 is an interface for inputting and outputting characters, numbers, and various instructions to various external devices. The input / output I / F 36 controls the display of various information such as a cursor, a menu, a window, a text, an image, etc. on a display 44 such as a liquid crystal display (LCD). In one example, the display 44 is a touch panel display provided with an input device (input means). In addition to the display 44, input devices such as a mouse and a keyboard can be connected to the input / output I / F 36.
[0093] The sound input / output I / F 37 is a circuit that processes the input and output of sound signals between the microphone 45 and the speaker 46 under the control of the CPU 30. The microphone 45 is an example of a built-in sound collecting device that can input sound signals under the control of the CPU 30. The speaker 46 is an example of a reproducing device that outputs sound signals under the control of the CPU 30.
[0094] The network I / F 38 is an interface for communicating with other devices and apparatuses via the communication network 13. The network I / F 38 is, for example, a communication interface of a wired or wireless LAN or the like. The short-range communication circuit 39 is a communication circuit conforming to a protocol such as near field communication (NFC) or Bluetooth (BLUETOOTH). The external device I / F 41 is an interface for connecting the control unit 20 to another device.
[0095] Examples of the bus 42 include, but are not limited to, an address bus and a data bus electrically connected to components such as a CPU, etc. The bus 42 transmits an address signal, a data signal, and various control signals.
[0096] A drive motor 47 , an acceleration and direction sensor 48 , a global positioning system (GPS) sensor 49 , an imaging device 22 , a battery 50 , and a sensor 51 such as a gas sensor are connected to the control section 20 via an external device I / F 41 .
[0097] The driving motor 47 drives the moving mechanism 24 to rotate according to the command from the CPU 30, thereby moving the traveling robot 11 on the ground. The acceleration and direction sensor 48 includes various sensors, such as an electromagnetic compass, a gyrocompass, and an accelerometer that senses geomagnetism. The GPS sensor 49 receives GPS signals from GPS satellites. The battery 50 is a unit that supplies power to the traveling robot 11 as a whole. In addition to the battery built into the main body of the traveling robot 11, the battery 50 may also include an external battery that supplies auxiliary power from the outside.
[0098] Figure 4 1 is a block diagram showing an example of the hardware configuration of the management server 12. Since the communication terminal 14 has the same hardware configuration as that of the management server 12, the description of the hardware configuration of the communication terminal 14 will be omitted.
[0099] The management server 12 is implemented by a general-purpose computer. The management server 12 includes a CPU 60, a ROM 61, a RAM 62, an HD 63, an HDD controller 64, a display 65, an external device I / F 66, a network I / F 67, a bus 68, a keyboard 69, a pointing device 70, a sound input / output I / F 71, a microphone 72, a speaker 73, a camera 74, a digital versatile disk rewritable (DVD-RW) drive 75, and a medium I / F 76.
[0100] The CPU 60 controls the overall action of the management server 12. The ROM 61 stores programs such as an initial program loader (IPL) to start the CPU 60. The RAM 62 provides a work area for the CPU 60. The HD 63 stores various data such as programs. The HDD controller 64 controls the reading and writing of data to the HD 63 under the control of the CPU 60. The display 65 displays various information such as a cursor, a menu, a window, text, and an image. In one example, the display 65 is a touch panel display provided with an input device. The display 65 can be an external device with a display function, such as an electronic induction whiteboard or an interactive whiteboard (IWB). Alternatively, the display 65 can also be a flat portion on which an image from a projector or a head-up display (HUD) is projected (for example, a ceiling or wall of a management location, a windshield of a car, etc.).
[0101] The external device I / F 66 is an interface for connecting various external devices. The network I / F 67 is an interface for data communication via the communication network 13. The bus 68 is, for example, an address bus or a data bus that electrically connects components such as the CPU 60.
[0102] The keyboard 110a is an example of an input device including a plurality of keys for inputting characters, numbers or various instructions. The pointing device 70 is an example of an input device that allows a user to select or execute various instructions, select a processing item, or move a cursor being displayed. The input device is not limited to the keyboard 69 and the pointing device 70, but also includes a touch panel and a voice input device. The sound input / output I / F 71 is a circuit that processes the input and output of sound signals between a microphone 72 and a speaker 73 under the control of the CPU 60. The microphone 72 is an example of a built-in sound collection device that receives sound input. The speaker 73 is an example of a built-in output device that outputs sound signals.
[0103] The camera 74 is an example of a built-in image capturing device for capturing an image of a subject to obtain image data. The microphone 72, the speaker 73, and the camera 74 may be external devices of the management server 12 rather than built-in devices. The DVD-RW drive 75 controls reading or writing various types of data to a DVD-RW 77 as an example of a removable recording medium. The removable recording medium is not limited to the DVD-RW 77, but may also be a recordable DVD (DVD-R) or a BLU-RAY disk. The medium I / F 76 controls reading or writing data from or to a recording medium 78 such as a flash memory.
[0104] Figure 5 FIG. 1 is a block diagram of an example of the functional configuration of the traveling robot 11, the management server 12, and the communication terminal 14. The functions of the traveling robot 11, the management server 12, and the communication terminal 14 can be realized by one or more processing circuits. The storage units 97, 106, and 128 are composed of HD34 ( Figure 3 )、HD63( Figure 4 ) etc. Here, the term "processing circuit or circuit" includes a programmed processor that performs each function by software, such as a processor implemented by an electronic circuit, and devices such as an application specific integrated circuit (ASIC), a digital signal processor (DSP), a field programmable gate array (FPGA), and conventional circuit modules arranged to perform the functions.
[0105] The traveling robot 11 includes a control unit 20. The control unit 20 includes, as functional units, a sending and receiving unit 80, a determination unit 81, a camera control unit 82, a state information acquisition unit 83, a position information acquisition unit 84, a candidate destination acquisition unit 85, and a path information generation unit 86. The control unit 20 also includes a destination setting unit 87, a traveling control unit 88, an image recognition unit 89, a mode setting unit 90, an autonomous traveling unit 91, a manual operation processing unit 92, a task execution unit 93, an image processing unit 94, a learning unit 95, a storage and reading unit 96, and a storage unit 97.
[0106] The transmission and reception unit 80 serving as transmission means (transmission unit) and reception means (reception unit) performs transmission and reception of various data and information to and from other devices such as the management server 12 and the communication terminal 14 via the communication network 13 .
[0107] The determination unit 81 as determination means performs various determinations. The image capture control unit 82 controls the image capture processing performed by the image capture device 22. The image capture control unit 82 sets the PTZ setting value to the image capture device 22 and instructs the image capture device 22 to perform image capture processing.
[0108] The state information acquisition unit 83 serving as state information acquisition means acquires information on the state of the traveling robot 11 and information on the state of the surroundings from various sensors including the image sensor of the imaging device 22 .
[0109] The state information acquisition unit 83 acquires optical information (image data) from the image sensor of the camera 22 as state information. The state information acquisition unit 83 acquires distance data representing the measured distance from the obstacle detection sensor to the object (obstacle) existing around the traveling robot 11 as state information, and the state information acquisition unit 83 acquires the direction facing the traveling robot 11 from the acceleration and direction sensor 48 as state information. The state information acquisition unit 83 acquires the gas concentration from the gas sensor as state information. The determination unit 81 can determine whether there is an abnormality in the surroundings based on the state information acquired by the state information acquisition unit 83.
[0110] The position information acquisition unit 84 acquires position information indicating the current position of the traveling robot 11 using the GPS sensor 49. The position information is coordinate information indicating the latitude and longitude of the current position of the traveling robot 11.
[0111] The candidate destination acquisition unit 85 acquires an image of a candidate destination indicating a candidate for the destination of the traveling robot 11. The candidate destination acquisition unit 85 acquires a captured image acquired by the imaging control unit 82 as an image of a candidate destination.
[0112] The path information generating unit 86 generates path information (path data) indicating the path (travel path) along which the traveling robot 11 travels. The path information generating unit 86 generates path information indicating the path from the current position to the final destination based on the position of the candidate destination selected by the operator of the traveling robot 11. Example methods of generating the path information include a method of connecting waypoints from the current position to the final destination with straight lines, and a method of generating a path for avoiding obstacles and minimizing the travel time using information about obstacles acquired by the state information acquiring unit 83. A waypoint is a freely selected point on the path from the travel start position to the final destination.
[0113] The destination setting unit 87 sets the destination of the traveling robot 11. For example, based on the current position of the traveling robot 11 acquired by the position information acquiring unit 84 and the path information generated by the path information generating unit 86, the destination setting unit 87 sets one of the destination candidates selected by the operator of the traveling robot 11 as the traveling destination to which the traveling robot 11 is to travel.
[0114] The traveling control unit 88 controls the traveling of the traveling robot 11 by driving the moving mechanism 24. For example, the traveling control unit 88 controls the traveling robot 11 to travel in response to a driving instruction of the autonomous traveling unit 91 or the manual operation processing unit 92.
[0115] The image recognition unit 89 performs image recognition on the captured image acquired by the camera control section 82. For example, the image recognition unit 89 performs image recognition to determine whether a specific object is captured in the acquired captured image. The specific object is, for example, an obstacle on or around the travel path of the traveling robot 11, an intersection such as a crossroad or an L-shaped road, or a sign or signal at the location.
[0116] The mode setting unit 90 sets an operation mode indicating an operation for moving the traveling robot 11. The mode setting unit 90 sets one of an autonomous traveling mode in which the traveling robot 11 autonomously travels and a manual traveling mode in which the traveling robot 11 travels, according to a manual operation by an operator.
[0117] The autonomous traveling unit 91 controls the autonomous traveling process of the traveling robot 11. For example, the autonomous traveling unit 91 outputs an instruction to the traveling control unit 88 to drive the traveling robot 11 so that the traveling robot 11 travels on the traveling path indicated by the path information generated by the path information generating unit 86.
[0118] The manual operation processing unit 92 controls the manual operation processing of the traveling robot 11. For example, the manual operation processing unit 92 outputs an instruction for driving the traveling robot 11 to the traveling control unit 88 in response to a manual operation command transmitted from the communication terminal 14.
[0119] The task execution unit 93 controls the traveling robot 11 to execute a specified task in response to a request from an operator. The specified task is, for example, the capture of an image for the inspection of equipment at the location. When the traveling robot 11 includes a movable arm, the specified task may include light physical labor by the movable arm.
[0120] The image processing unit 94 generates an image to be displayed on the communication terminal 14. For example, the image processing unit 94 processes a captured image acquired by the imaging control unit 82 to generate an image for display.
[0121] The learning unit 95 learns a travel path for autonomous travel of the traveling robot 11. For example, the learning unit 95 performs machine learning of the travel path for autonomous travel based on a captured image acquired by a travel operation in the manual operation mode of the manual operation processing unit 92 and detection data acquired by the state information acquisition unit 83. The autonomous traveling unit 91 performs autonomous travel of the traveling robot 11 based on the learning data of the result of the machine learning of the learning unit 95.
[0122] The storage and reading unit 96 stores various types of data in the storage unit 97, and reads out various types of data from the storage unit 97. The storage unit 97 stores various types of data under the control of the storage and reading unit 96.
[0123] The travel of the traveling robot 11 is controlled by the management server 12 based on the path information (via point information). The via point information is information about the path point (coordinate information represented by latitude and longitude). The travel of the traveling robot 11 is controlled so as to sequentially track the via point information. Image capture of the camera device 22 is controlled based on the position data and the PTZ information.
[0124] When the traveling robot 11 reaches the image capturing position according to the position data, image capturing is performed by setting the setting value of the PTZ information in the imaging device 22. When performing image capturing, the traveling robot 11 may keep moving or temporarily stop at the image capturing position.
[0125] The management server 12 includes, as functional units, a transmission and reception unit 100 , a determination unit 101 , an instruction unit 102 , a map information management unit 103 , a route information management unit 104 , a storage and reading unit 105 , and a storage unit 106 .
[0126] The transmitting and receiving unit 100 serving as transmitting means and receiving means receives captured images, sensor detection results, etc. acquired by the traveling robot 11, and transmits instructions to the traveling robot 11. The transmitting and receiving unit 100 transmits the captured images, sensor detection results, etc. to the communication terminal 14.
[0127] The storage unit 106 includes a candidate destination management DB 107, a map information management DB 108, a learning data management DB 109, and a path information management DB 110. The candidate destination management DB 107 stores the candidate destination data acquired by the candidate destination acquisition unit 85 of the traveling robot 11. The candidate destination data stored in the candidate destination management table associates, for each location identifier (ID) for identifying the location where the traveling robot 11 is set, a candidate ID for identifying the candidate destination, location information indicating the location of the candidate destination, and a captured image obtained by capturing a specific area of the location as the candidate destination. The location information is coordinate information of the latitude and longitude of the location of the candidate destination representing the location. The candidate destinations of the traveling robot 11 include not only candidates for the destination of the traveling robot 11, but also candidates for places to be excluded from the traveling path of the traveling robot 11.
[0128] The map information management DB 108 uses a map information management table to store map information. The map information is map information of an environment map of a location where the traveling robot 11 is installed. In the map information management table, a location ID for identifying a location where the traveling robot 11 is installed, a location name, and a storage location of an environment map of the location are stored in association with each other. The map information management unit 103 uses the map information management DB 108 to manage the map information of the location where the traveling robot 11 is installed.
[0129] The learning data management DB 109 uses a learning data management table to store learning data. The learning data is the result of learning the autonomous travel path by the learning unit 95 of the traveling robot 11. In the learning data management table, captured images, sensor detection results, etc. acquired from the traveling robot 11 are accumulated, and the results of machine learning are stored as learning data for each location or each traveling robot 11. These DBs are in the storage unit 106 of the management server 12, but the location thereof is not limited thereto. These DBs may also be in the traveling robot 11.
[0130] The path information management DB110 uses a path information management table to store path information representing the travel path of the traveling robot 11. The path information management DB110 stores, for each location ID identifying the location where the traveling robot 11 is installed, a path ID identifying the travel path of the traveling robot 11 and path information representing the travel path of the traveling robot 11 in association with each other. The path information represents the travel path of the traveling robot 11, and is used to reach the next destination one by one in sequence. When the traveling robot 11 starts to travel, the path information is generated by the path information generation unit 86. Specifically, the path information generation unit 86 generates path information for normal conditions and path information for abnormal conditions. In this example, the path information management DB110 is in the storage unit 106 of the management server 12, but its location is not limited thereto, and may also be in the traveling robot 11. The path information management unit 104 manages the path information by using the path information management DB110.
[0131] The determination unit 101 as a determination means determines whether there is an abnormality in the state of the monitored object based on the captured image or sensor detection results obtained from the traveling robot 11. The storage unit 106 also stores a standard for determining the presence or absence of an abnormality. Therefore, the determination unit 101 determines the presence or absence of an abnormality based on the determination standard stored in the storage unit 106. For example, when a flow meter for measuring the flow rate of a liquid is set as a monitored object, whether the flow rate is within a specified range is set as a determination standard. When the determination standard is set to a specified range of 2 to 10 ml / s for a normal flow rate, when the flow meter indicates 0 ml / s, the determination unit 101 determines that an abnormality exists. The determination standard is not limited to the above example. In addition, the information processing system 10 may further include an extraction unit for extracting the flow rate to be determined from the captured image. The extraction unit uses a known image recognition technology to extract the flow rate from the position of the meter needle. Image recognition technology is well known in the art, so a detailed description is omitted.
[0132] The instructing unit 102 as an instructing means instructs the traveling robot 11. The instructing unit 102 instructs the traveling robot 11 to detect the state of the monitored object via the transmitting and receiving unit 100 along the path (first path) of the normal condition. For example, the instructing unit 102 instructs the traveling robot 11 to travel along the path of the normal condition and to capture an image of the monitored object while traveling. For example, under normal conditions, the traveling robot 11 may be controlled to follow a path for capturing an image of an indicator of a measuring instrument such as a flow meter.
[0133] When the determination unit 101 determines that an abnormality exists, the instruction unit 102 instructs the traveling robot 11 to switch the path of normal conditions to the path where the abnormality occurs (hereinafter referred to as the "path for abnormal situations") as the second path, and detects the state of the designated part related to the state of the monitored object. In other words, the instruction unit 102 instructs the switch to the path for abnormal situations via the sending and receiving unit 100, and instructs the shooting of an image of the designated position. At this time, the instruction unit 102 can instruct the traveling robot 11 to record the sound, and can instruct the camera device 22 to shoot the image of the monitored object or the designated part. The designated part is another object in the location that is different from the monitored object. The monitored object is the subject of the image shooting of which the image is shot when the traveling robot 11 follows the path of normal conditions. When there is an abnormality in the flow, it is possible that the abnormality has occurred in the opening and closing of the valve or the pump. In this case, the path can be changed to a path for the traveling robot 11 to shoot the image of the valve and record the sound of the pump. The path of normal conditions and the path for abnormal situations are not limited to these examples.
[0134] The transmitting and receiving unit 100 receives captured images, sensor detection results, and the like from the traveling robot 11 , and transmits the received information to the communication terminal 14 .
[0135] The communication terminal 14 is installed at the management location and operated by the operator. The communication terminal 14 includes a transmission and reception unit 120, a reception unit 121, a display control unit 122, a determination unit 123, a manual operation command generation unit 124, an autonomous travel request generation unit 125, an image processing unit 126, a storage and reading unit 127, and a storage unit 128 as functional units.
[0136] The sending and receiving unit 120 sends and receives various data or information to and from the traveling robot 11 and the management server 12. The receiving unit 121 receives various selections and inputs from the operator. The display control unit 122 displays various screens on the display. The image captured by the traveling robot 11, the detection results detected by the sensor, etc. are displayed on the display. The determination unit 123 performs various determinations.
[0137] The manual operation command generation unit 124 generates a manual operation instruction for moving the traveling robot 11 by manual operation according to the input operation of the operator. The autonomous traveling request generation unit 125 generates an autonomous traveling request for autonomously traveling the traveling robot 11. For example, the autonomous traveling request generation unit 125 generates an autonomous traveling request to be sent to the traveling robot 11 based on the information of the candidate destination selected by the operator.
[0138] The image processing unit 126 generates a display image to be displayed on the display. For example, the image processing unit 126 performs processing on an image captured by the camera 22 of the traveling robot 11, and generates a display image. Although the image processing unit is provided in both the traveling robot 11 and the communication terminal 14 in this example, the image processing unit may alternatively be provided in one of the traveling robot 11 and the communication terminal 14.
[0139] The storage and reading unit 127 stores various data in the storage unit 128 , and reads various data from the storage unit 128 .
[0140] When the traveling robot 11 is used to monitor the monitoring area, the image captured by the traveling robot 11 is presented to the operator, so that the operator can remotely control the traveling robot 11 and confirm the surrounding conditions of the traveling robot 11 in real time. Using the image of the area in the location of the candidate destination, the area is pre-registered as a candidate destination. Then, the destination of the traveling robot 11 is set using the candidate destination, and the traveling robot 11 is set to the autonomous traveling mode, and path information is generated. At this time, a traveling path is generated so that the traveling robot 11 autonomously travels in the order of the captured images of the destination selected by the operator. The method of generating path information is not limited to the above-mentioned example method.
[0141] Next, the information processing system 10 for monitoring the monitored object using the traveling robot 11 in a specific scenario will be described in detail.
[0142] Figure 6 Shown is a diagram of a first scenario in which the information processing system is applied. Figure 6 The figure shows an inspection operation in a monitoring area such as a chemical plant by the traveling robot 11. The chemical plant includes a tank 200, a pump 201, valves 202 and 203, and flow meters 204 and 205. The liquid flowing in the pipe is sucked by the pump 201 and discharged to the tank 200. The flow rate of the liquid supplied to the pump 201 is measured by the flow meter 204 and controlled by the valve 202.
[0143] The flow meter 204 is set as an object to be inspected (inspection object). The traveling robot 11 travels according to the path data, and faces the flow meter 204 in order to capture an image of the flow meter 204. The traveling robot 11 stops at the inspection point D001 facing the flow meter 204, sets the setting value of the PTZ of the camera 22, and captures an image of the flow meter 204.
[0144] When the flow rate indicated by the flow meter 204 is controlled within the range of 2 to 10 ml / s, the judgment standard is set to judge the flow rate within the range as normal and the flow rate outside the range as abnormal. Therefore, the management server 12 extracts the numerical value representing the flow rate from the image taken by the traveling robot 11, and judges whether there is an abnormality based on the judgment standard. When there is an abnormality, the occurrence of the error is reported to the communication terminal 14 and displayed. As a result, the operator at the management location can recognize that there is an abnormality in the flow rate indicated by the flow meter 204.
[0145] Hereinafter, possible causes of abnormality will be described based on information indicating abnormality in the measured value of the flow meter 204 .
[0146] (1) The flow meter 204 is faulty.
[0147] (2) Although the flow meter 204 is operating normally, the liquid does not flow in the pipe due to a blockage somewhere.
[0148] (3) The flow meter 204 operates normally, but the nearby valve 202 is closed and there is no flow.
[0149] (4) The flow meter 204 is functioning normally, but the nearby pump 201 fails, inhibiting the flow.
[0150] Therefore, an operation for confirming each abnormality when an abnormality occurs is set in advance. When an abnormality occurs, the traveling robot 11 travels along the abnormality path and captures an image of the specified PTZ setting value at a specified image capturing position.
[0151] Specifically, at checkpoint D002, in order to confirm the blockage of the flow path upstream of the flowmeter 205, an image of the flowmeter 204 is captured. Next, in order to confirm whether the valve in the flow path is closed, an image of the valve 203 is captured at checkpoint D003 to confirm the open and closed state of the valve 203. Next, in order to confirm whether there is any abnormality in the operation of the pump 201, the operation sound of the pump 201 is recorded at checkpoint D004. The traveling robot 11 travels to these points in this order and captures images. The captured images and collected sounds are sent to the communication terminal 14 operated by the operator and played. As a result, the operator can determine where there is an abnormality and can respond to the abnormality quickly.
[0152] In order to travel along such a route and perform image capture, it is necessary to set a candidate for a destination and set an autonomous travel route at the current location. The destination is a transit point or a checkpoint for image capture.
[0153] Figure 7The figure shows the action of controlling the traveling robot 11 to travel at the location and registering the candidate destinations (via points or checkpoints) that the traveling robot 11 passes on the autonomous traveling path. The operator performs a prescribed input operation from the communication terminal 14 of the management location, starts the operation of the traveling robot 11, controls the traveling robot 11 to display the map information of the object location and starts image capture. The operator remotely controls the traveling robot 11 on the operation screen, and searches for the candidate destination while confirming the current location of the traveling robot 11 and viewing the image captured by the camera device 22. Then, the operator registers the candidate destination by pressing the candidate destination registration button.
[0154] exist Figure 7 In the example shown, the traveling robot 11 is controlled to travel in each of the four areas 1 to 4, and the operator searches for candidates for via points and check points. The found via points and check points are registered as candidate destinations.
[0155] Figure 8 The figure shows an example of waypoints and checkpoints. In order to inspect areas 1 to 4, waypoints P0 to P21 and checkpoints D001 to D004 are registered as candidate destinations. The traveling robot 11 sets waypoint P0 as the inspection start position and registers waypoints P1 to P21 as candidate destinations. Checkpoint D001 is registered as a candidate destination as a checkpoint in area 1. In addition, checkpoints D002 to D004 are registered as positions where images are to be taken when an abnormality occurs.
[0156] Therefore, in the inspection of area 1 under normal conditions, the inspection point D001 is registered to capture an image of the flow meter 204. If there is an abnormality in the flow rate indicated by the flow meter 204, images of other flow meters and valves can be captured at the inspection points D002 to D004 that are not captured in the normal inspection, and the operation sound of the pump can be stored.
[0157] Fig. 9 1 is a diagram showing an example of a via point management table. The via point management table stores position information (latitude and longitude) as via point information, a point ID for identifying a point, and a file name of image data of a captured image in association with each other. The point ID is a number or code freely assigned at each login. The position information indicates the latitude and longitude of the traveling robot 11 measured by the GPS sensor 49. Instead of the file name, any information for identifying image data may be used.
[0158] Fig.104 is a diagram showing an example of an inspection object management table. The inspection object management table stores, for each inspection object, an inspection object ID, a name of the inspection object, and location information in association with each other. Like the point ID, the inspection object ID is a number or code freely assigned at each login. The location information indicates the latitude and longitude of the traveling robot 11 measured by the GPS sensor 49. The name is information for identifying the inspection object, such as "meter 1", "meter 2", "valve 1", or "pump 1".
[0159] In addition to the above-mentioned items such as the name, the inspection object management table also stores the actions to be performed at the time of inspection (inspection actions), the settings of the inspection actions, etc. in association with each other. The inspection actions include, for example, image shooting and acquisition of sound sensor information (action sounds, etc.). When the inspection action is image shooting, the settings include, for example, pan, tilt, and zoom settings. When the inspection action is acquisition of sound sensor information, the settings include sensor settings such as the setting value of the sound sensor for collecting action sounds, etc.
[0160] Fig.11 1 is a diagram showing an example of a path information management table. In order to generate path information, an image in the information stored in the candidate destination management table is presented to the operator, and the operator's selection of the image is received to set the path. The path information includes data of passing points and checkpoints arranged in the order of travel of the traveling robot 11. In the path information management table, each time path data is registered, a path ID is freely assigned. The path ID and the path data are also associated with an area ID that identifies an area to be checked.
[0161] The path data associated with the path ID "R001" indicates the inspection path under normal conditions. The path information "R001" indicates that the inspection is performed at the inspection point D001 on the way from the via point P8 to the via point P10, and if there is no abnormality, the traveling robot 11 returns to the via point P8 and returns to the via point P0 as the inspection start position. When the inspection point is included in the path data, the inspection action and setting are read out with reference to the inspection object management table, and the inspection action is executed with the changed setting.
[0162] When there is an abnormality in the inspection action at the inspection point D001, the traveling robot 11 is controlled to travel with a path instead of the path data associated with the specified path ID according to the abnormal situation, and the inspection action of the specified point is performed. As described above, the first path and the second path may partially overlap each other. The first path and the second path may be different from each other in only one or both of the starting point and the end point. In addition, even if the starting point, the passing point, and the end point in the entire path are the same in the first path and the second path, when the inspection object between the first path and the second path is different, the second path can also be considered to be different from the first path.
[0163] When an abnormality is detected in the flow indicated by the meter 1 based on the image of the meter 1 captured by the camera device 22 at the check point D001, the path is switched from the path used for normal conditions (returning from the via point P10 to the via point P8) to the path for abnormal conditions. The path for abnormal conditions includes the shooting location for capturing the image of the meter 2, valve 1, and pump 1. In the case of capturing the meter 2, valve 1, and pump 1, the path for abnormal conditions includes the check points D002, D003, and D004. The traveling robot 11 may stop at the check point D004, or may return to the path for normal conditions via the above-mentioned via point P10.
[0164] Fig.12 1 is a timing diagram of a process performed by the information processing system 10. The operator operates the communication terminal 14 to set a path and instruct the start of an inspection. The communication terminal 14 sends an instruction including information about the set path to the management server 12 to start the inspection (S1). The management server 12 sends the inspection start instruction together with the path information managed by the path information management table to the traveling robot 11 (S2). The traveling robot 11 receives the instruction from the management server 12 and starts the inspection action (S3).
[0165] The traveling robot 11 travels on a path indicated by the management server 12, and performs inspection actions at designated inspection points. Whenever the traveling robot 11 captures an image at a checkpoint, the traveling robot 11 transmits the captured image to the management server 12 (S4). The management server 12 determines whether the inspection object is normal or abnormal based on the captured image and the judgment criteria. When the inspection object is judged to be normal, the management server 12 sends the captured image as a result to the communication terminal 14, and causes the communication terminal 14 to display the captured image (S5). On the other hand, when the inspection object is judged to be abnormal, the management server 12 notifies the communication terminal 14 of the abnormality, and causes the communication terminal 14 to display a dialog box for selecting an action (S6).
[0166] The communication terminal 14 receives the selection of an action in the displayed dialog box from the operator (S7). Upon receiving the selection of an action on the abnormal situation path, the communication terminal 14 sends an instruction to the management server 12 to perform an inspection action on the abnormal situation path. After receiving the instruction, the management server 12 sends the abnormal situation information and an instruction to perform an inspection action when an abnormality occurs to the traveling robot 11 (S8).
[0167] When there is an abnormality in the flow rate (reading value) indicated by meter 1, the traveling robot 11 is instructed to take an image of meter 2 to confirm blockage in the path upstream of meter 1. Next, in order to confirm whether valve 1 on the path is closed, an image showing the open and closed states of valve 2 is taken. Next, in order to confirm whether there is an abnormality in the operation of pump 1, the operation sound of pump 1 is recorded.
[0168] In the above-mentioned inspection action (photographing at a designated inspection point) performed on the abnormality path, for example, an abnormality is detected because a blockage occurs due to the closing of valve 1. In this case, even if the cause is not found from the image captured by meter 2, the cause of the abnormality can be identified by capturing an image showing the open and closed state of the next valve 1. When there are multiple points to be inspected when an abnormality occurs, each inspection site can be inspected in sequence. At this time, as a first method for identifying the cause of the abnormality, the inspection action is terminated when the cause of the abnormality is identified.
[0169] As a second method for identifying the cause of the abnormality, the inspection action is continued until all of the plurality of inspection points are inspected.
[0170] The first method for identifying the cause of the abnormality will be described in detail. Fig.13 The diagram shows the inspection action performed when an abnormality is detected at a checkpoint in area 1. Specifically, the route is switched to the abnormality route and the inspection is performed at the designated checkpoint. The normal route is a route that continues from via point P8 to via point P10 via checkpoint D001 and returns from via point P10 to via point P8. In this example, an abnormality is detected from the inspection performed at checkpoint D001 and the route is switched to the abnormality route.
[0171] The abnormality path is performed by sequentially checking at the check points D002, D003, and D004. In the first method, when the cause of the abnormality is identified in the middle of the check, the subsequent check is omitted. Therefore, the via points P30, P31, and P32 are set so that the check can be performed at each of the check points D002, D003, and D004.
[0172] Fig.14Shown is a flowchart of an example of processing performed by the traveling robot 11 in an inspection action using the first method. The traveling robot 11 is set at a position to start the inspection action, and the inspection is started. In step S101, the traveling robot 11 receives path information for normal conditions and instructions for traveling on the path for normal conditions from the management server 12. In step S102, the traveling robot 11 travels to the inspection point based on the path information. In step S103, the traveling robot 11 performs an inspection action at the inspection point. The inspection action is image capture of the inspection object, etc. In step S104, the traveling robot 11 sends the acquired status information such as the captured image to the management server 12.
[0173] In step S105, the traveling robot 11 determines whether the path information for abnormal situation is received from the management server 12. When the path information for abnormal situation is received, the process returns to step S102, and the traveling robot 11 travels on the path for abnormal situation based on the path information.
[0174] If it is determined in step S105 that the path information for abnormality is not received, the processing continues to step S106 to determine whether there is a next checkpoint. If there is a next checkpoint, the processing returns to step S102 to move to the next checkpoint. If it is determined in step S106 that there is no next checkpoint, in step S107, the traveling robot 11 travels to the end point (end point). The traveling robot 11 notifies the management server 12 that it has reached the end point and ends the inspection action.
[0175] Fig.15 FIG. 1 is a flowchart of an example of processing performed by the management server 12 in the inspection operation using the first method. In step S111, the management server 12 sends the path information used in normal times to the traveling robot 11 and instructs the traveling robot 11 to travel on the path used in normal times. In step S112, the management server 12 receives status information from the traveling robot 11.
[0176] In step S113, the management server 12 determines whether there is an abnormality in the inspection object based on the received state information and the judgment standard. For example, the inspection object is the reading value of the meter 1. When it is determined that there is an abnormality, in step S114, the management server 12 sends the path information for the abnormal situation and the instruction to travel on the abnormal situation path to the traveling robot 11. On the other hand, when it is determined that there is no abnormality, in step S115, the management server 12 instructs the traveling robot 11 to continue the inspection action.
[0177] In step S116 , the management server 12 receives a notification from the traveling robot 11 that the traveling robot 11 has reached the end point, and ends the inspection.
[0178] Fig.16 The figure shows an example of an abnormality information table used to determine whether the inspection object has abnormalities. The items of the abnormality information table include inspection points, area information, captured images and detection results, and these items are stored in association with the abnormality ID that identifies the abnormality. The abnormality ID is associated with the items "A. Gas concentration", "B. Image comparison", "C. Temperature", "D. Light detection and ranging (LIDAR, a sensor that measures the distance to a remote object or its shape)", "Judgment criteria" and "Action". Among the items of "Judgment criteria", when the result of "B. Image comparison" is pre-judged to be a standard with low matching degree among the items "A. Gas concentration", "B. Image comparison", "C. Temperature", and "D. LIDAR", the gas concentration and temperature of "A" and "C" are both outside the set range, etc., and when the standard is met, the management server 12 can determine the existence of abnormalities.
[0179] Fig.17 FIG. 1 is a diagram showing an example of an abnormal situation path DB based on the first method. The abnormal situation path DB is a database that the management server 12 stores in Fig.15 The abnormal situation path DB stores the path data indicating the abnormal situation path in which the via points and the check points are sequentially arranged in the abnormal situation path management table. The path data is associated with the abnormal situation path ID for identifying the abnormal situation path and the robot name for identifying the traveling robot traveling on the path. If there is only one traveling robot 11, it is not necessary to register the robot name in association with the traveling robot 11.
[0180] Path "ID201" is a path for performing an inspection at inspection point D004. Path "ID202" is a path for performing an inspection at inspection point D003. Path "ID203" is a path for performing an inspection at inspection point D002. Path "ID301" and path "ID403" are the same as Fig.13 The checking actions shown are not directly relevant and are provided as examples.
[0181] Next, the second method for identifying the cause of the abnormality will be described in detail. Fig.18Shown is a flowchart of an example of processing performed by the traveling robot 11 in an inspection action using the second method. The traveling robot 11 is set at a position to start the inspection action, and the inspection is started. In step S121, the traveling robot 11 receives path information for normal conditions and instructions for traveling on the path for normal conditions from the management server 12. In step S122, the traveling robot 11 travels to the inspection points in sequence based on the path information, and performs inspection actions at the inspection points. The inspection action is image capture of the inspection object, etc. In step S123, the traveling robot 11 sends the acquired status information to the management server 12.
[0182] In step S124, the traveling robot 11 determines whether the path information for abnormal situations is received from the management server 12. When the path information for abnormal situations is received, in step S125, the traveling robot 11 travels on the path for abnormal situations based on the path information and performs an inspection. In step S126, the traveling robot 11 travels to the end point, notifies the management server 12 that it has reached the end point, and ends the inspection operation.
[0183] In step S124, when no abnormality path information is received, in step S126, the traveling robot 11 travels to the end point and notifies the management server 12 of the arrival at the end point. Then, the inspection operation ends.
[0184] Fig.19 1 is a flowchart of an example of processing performed by the management server 12 in the inspection action using the second method. In step S131, the management server 12 sends the path information for normal conditions and instructions for traveling on the path for normal conditions and performing inspections at inspection points to the traveling robot 11. In step S132, the management server 12 receives status information from the traveling robot 11.
[0185] In step S133, the management server 12 determines whether there is an abnormality in the inspection object based on the received state information and the judgment standard. When it is determined that there is an abnormality, in step S134, the management server 12 sends the path information for the abnormal situation and the instruction to travel on the abnormal situation path to the traveling robot 11. In step S135, the management server 12 receives a notification from the traveling robot 11 that the traveling robot 11 has reached the end point, and ends the inspection. In contrast, when it is determined that there is no abnormality, in step S135, the management server 12 receives a notification from the traveling robot 11 that the traveling robot 11 has reached the end point, and ends the inspection.
[0186] Fig. 20Shown is a diagram of another example of an abnormality information table for determining whether an inspection object has an abnormality. The items of the abnormality information table include inspection points, area information, captured images, and detection results, and these items are stored in association with an abnormality ID that identifies the abnormality. The abnormality ID is associated with the items "A. Gas concentration", "B. Image comparison", "C. Temperature", "D. Light detection and ranging (LIDAR, a sensor that measures the distance to a remote object or its shape)", "Judgment criteria" and "Action". Among the items of "Judgment criteria", when among the items "A. Gas concentration", "B. Image comparison", "C. Temperature", and "D. LIDAR", the result of "B. Image comparison" is pre-judged to be a standard with a low degree of match, the gas concentration and temperature of "A" and "C" are both outside the set range, etc., and when the standard is met, the management server 12 can determine the existence of an abnormality.
[0187] Fig.21 FIG. 1 shows an example of an abnormal situation path DB based on the first method. The abnormal situation path DB is a database stored in the management server 12. Fig.19 The abnormal situation path DB stores the path data indicating the abnormal situation path in which the via points and the check points are sequentially arranged in the abnormal situation path management table. The path data is associated with the abnormal situation path ID for identifying the abnormal situation path and the robot name for identifying the traveling robot traveling on the path. If there is only one traveling robot 11, it is not necessary to register the robot name in association with the traveling robot 11.
[0188] The path "ID200" is a path for performing inspections at inspection points D004, D003, and D002 in this order. The paths "ID201" and "ID203" are the same as Fig.13 The checking actions shown are not directly relevant and are provided as examples.
[0189] Fig. 22 FIG. 1 is an example of a monitoring area where a plurality of traveling robots 11 are set up and an inspection is performed. In a chemical plant or the like, areas are usually connected by long-distance pipelines. Fig. 22 As shown, when an abnormality is found in the tank area 1, it is necessary to confirm the status of the valve and meter in the valve area 3 in the remote area. In this case, by instructing not only one traveling robot 11 but also other traveling robots 11, the cause of the abnormality can be quickly determined.
[0190] Since the traveling robot 11 has identification information for uniquely identifying the traveling robot 11, such as a robot name, an Internet Protocol (IP) address, or a Media Access Control (MAC) address, instructions can be given to a target traveling robot 11 using the identification information.
[0191] In commercial facilities including chemical plants, many operations are performed every day. While the traveling robot 11 performs inspection operations in the chemical plant, workers can perform construction operations near where the traveling robot 11 travels.
[0192] In a chemical plant, dangerous gas may occur, and it is desirable to provide the traveling robot 11 with a gas sensor for detecting such gas. The gas may be a flammable gas, and there is a risk, for example, of explosion of the traveling robot 11. In view of such a risk, it is desirable to completely shut down the power supply of the traveling robot 11 when the concentration of the gas reaches a certain level or higher.
[0193] However, if the power of the traveling robot 11 is completely turned off, it is impossible to prompt information such as danger notification to the surroundings. In addition, even if a worker working nearby looks at the stopped traveling robot 11, the worker cannot know why the traveling robot 11 stops there. In addition, the worker will not be notified that there is dangerous gas there.
[0194] Therefore, the traveling robot 11 has a mechanism for prompting information (e.g., raising a sign) to the surroundings before the traveling robot 11 is turned off. After the prompting information, the traveling robot 11 is turned off. By adopting this structure, people nearby can know the status of the traveling robot 11 and take appropriate actions.
[0195] Fig.23 The flowchart of an example of the processing performed by the traveling robot 11 in the inspection action of the traveling robot 11 using the mechanism with prompt information is shown. The traveling robot 11 is set at a position to start the inspection action, and the inspection is started. In step S141, the traveling robot 11 receives path information for normal conditions and instructions for traveling on the path for normal conditions from the management server 12. In step S142, the traveling robot 11 travels to the inspection point based on the path information. In step S143, it is determined whether the traveling robot 11 detects gas while traveling.
[0196] When no gas is detected, in step S103, the traveling robot 11 performs an inspection action at the inspection point. The inspection action is image capture of the inspection object, etc. After the inspection action, in step S123, the traveling robot 11 sends the acquired state information to the management server 12. On the other hand, when gas is detected, in step S146, the traveling robot 11 notifies the management server 12 of the gas detection.
[0197] In step S147, the traveling robot 11 determines whether the path information for the abnormal situation is received from the management server 12. When it is determined that the path information for the abnormal situation is not received, in step S148, the traveling robot 11 determines whether the current checkpoint is the last checkpoint. In the case that it is not the last checkpoint, the process returns to step S142. When the current checkpoint is the last checkpoint, in step S149, the traveling robot 11 travels to the end point, notifies the management server 12 that the inspection action is completed, and ends the inspection action.
[0198] When receiving the abnormal path information in step S147, the traveling robot 11 determines whether to execute shutdown in step S151. Shutdown is an example of an action corresponding to an abnormality. When it is determined that shutdown is not to be executed, in step S142, the traveling robot 11 travels on the abnormal path and performs inspection.
[0199] When the shutdown is executed in step S151, the traveling robot 11 performs information presentation in step S152. Then, the traveling robot 11 shuts down in step S153.
[0200] Fig.24 1 is a flowchart of an example of processing performed by the traveling robot 11 in the inspection action using the management server 12 having the mechanism of prompt information. In step S161, the management server 12 sends the traveling robot 11 the path information for normal conditions and the instructions for traveling on the path for normal conditions and performing inspection at the inspection point. In step S162, the management server 12 determines whether a notification of gas detection is received from the traveling robot 11 until the traveling robot 11 reaches the inspection point. When no notification of gas detection is received, in step S163, the management server 12 receives status information from the traveling robot 11 at the inspection point.
[0201] In step S164, the management server 12 determines whether there is an abnormality in the inspection object based on the received status information and the judgment standard. When it is determined that there is no abnormality, in step S165, the management server 12 instructs the traveling robot 11 to continue the inspection action. At this time, the management server 12 can notify the traveling robot 11 that there is no abnormality and instruct the traveling robot 11 to continue the inspection action. The management server 12 can also send such a notification to the communication terminal 14. Then, in step S169, the management server 12 receives a notification of reaching the end point from the traveling robot 11 and ends the inspection.
[0202] When a notification is received in step S162, or when it is determined that an abnormality exists in step S164, in step S166, the management server 12 determines whether it is necessary to shut down the traveling robot 11. When it is determined that the running robot 11 needs to be shut down, in step S166, the management server 12 instructs the running robot 11 to perform the shutdown and end the inspection. The condition requiring shutdown is, for example, that the gas concentration is above the reference concentration. When it is determined that it is unnecessary to shut down the running robot 11, in step S168, the management server 12 sends the path information for the abnormal situation to the traveling robot 11. In step S169, the management server 12 receives a notification of arrival at the end point from the traveling robot 11, and ends the inspection.
[0203] Fig.25 The figure shows another example of an abnormality information table for determining whether the inspection object has an abnormality. In this table, in addition to the abnormal content, shutdown information is also logged. In this example, when the gas concentration in area 2 is 10 or more, it is logged to execute power shutdown.
[0204] Fig.26 1 is another sequence diagram of the processing performed by the information processing system 10. The operator operates the communication terminal 14 to set the path and instruct the start of the inspection. The communication terminal 14 sends an instruction including information about the set path to the management server 12 to start the inspection (S1). The management server 12 sends the inspection start instruction together with the path information managed by the path information management table to the traveling robot 11 (S2). The traveling robot 11 receives the instruction from the management server 12 and starts the inspection action (S3).
[0205] The traveling robot 11 travels on the path indicated by the management server 12, and performs inspection actions at designated inspection points according to set conditions. Whenever the traveling robot 11 captures an image at a checkpoint, the traveling robot 11 transmits the captured image to the management server 12 (S4). The management server 12 determines whether the inspection object is normal or abnormal based on the captured image and the judgment criteria. When the inspection object is judged to be normal, the management server 12 sends the captured image as a result to the communication terminal 14, and causes the communication terminal 14 to display the captured image (S5). On the other hand, when the inspection object is judged to be abnormal, the management server 12 notifies the communication terminal 14 of the abnormality, and causes the communication terminal 14 to display a dialog box for selecting an action (S6).
[0206] The communication terminal 14 receives the selection of the action of the displayed dialog box from the operator (S7). When receiving the selection of the action on the abnormal situation path, the communication terminal 14 sends an instruction to the management server 12 to perform the inspection action on the abnormal situation path. After receiving the instruction, the management server 12 sends the abnormal situation information and the instruction to perform the inspection action when the abnormality occurs to the traveling robot 11 (S8).
[0207] When there is an abnormality in the flow rate (reading value) indicated by meter 1, the traveling robot 11 is instructed to take an image of meter 2 to confirm blockage in the path upstream of meter 1. Next, in order to confirm whether valve 1 on the path is closed, an image showing the open and closed states of valve 2 is taken. Next, in order to confirm whether there is an abnormality in the operation of pump 1, the operation sound of pump 1 is recorded.
[0208] The management server 12 determines whether it is necessary to shut down based on the detection result of the gas concentration transmitted from the traveling robot 11 together with the captured image. When it is determined that it is necessary to shut down, the management server 12 controls the communication terminal 14 to display a text message that it is necessary to shut down and the traveling robot 11 will shut down (S9). In addition, the management server 12 sends a shutdown command to the traveling robot 11 (S10). The traveling robot 11 executes the information prompt action and shuts down.
[0209] Although the action of detecting gas and shutting down the traveling robot 11 are described above, when flammable gas or harmful gas is detected, in addition to sending a notification of gas detection to the communication terminal 14, in order to notify the surroundings of a gas leak and prevent the traveling robot 11 from being ignited, it is also desired that the traveling robot 11 travel to a place with low gas concentration.
[0210] Therefore, the traveling robot 11 returns to the path that the traveling robot 11 has traveled while outputting the alarm. Even if the traveling robot 11 returns to the path, there may be a situation where the gas concentration is equal to or higher than the reference value. In this case, the traveling robot 11 automatically shuts down so that the traveling robot 11 does not become a fire source. At this time, in order to indicate an alarm to the surroundings, it is desired to indicate the occurrence of an abnormality from a distance without using electricity.
[0211] Figures 27A to 27C FIG. 1 is a diagram showing an example of information presentation by the traveling robot 11 . Figures 27A to 27C Shown are three examples such as an example of raising an object such as a flag with good visibility, an example of spraying a powder such as a fire extinguishing agent without ignition hazard, and an example of raising a balloon. Note that the information prompt is not limited to these examples. The flag or balloon may carry a text message or symbol indicating, for example, "Danger" or "Leave".
[0212] There may be obstacles such as fallen objects, working vehicles, or damaged road surfaces on the path of the traveling robot 11. The traveling robot 11 captures an image of this situation, sends the captured image to the communication terminal 14, resets the path, and continues the inspection action. By viewing the image received and displayed by the communication terminal 14, the operator can identify obstacles that hinder the traveling of the traveling robot 11, and take measures to remove the identified obstacles.
[0213] If the traveling robot 11 waits for the obstacle to be removed, the inspection time will be longer. Therefore, the traveling robot 11 travels to the next inspection point through an alternative path instead of waiting for the obstacle to be removed. Such an alternative path can be set in advance. Alternatively, the operator can search the traveling path again in consideration of the obstacles on the current path and determine the alternative path.
[0214] Fig.28 Shown is an example diagram of an inspection action in the case where there is an impassable portion in the middle of a path. The traveling robot 11 travels in the direction indicated by the solid arrow in the tank area 3 to perform an inspection. However, an obstacle in the middle of the path prevents the traveling robot 11 from passing through the path. The traveling robot 11 detects the obstacle by means of LIDAR, which is a sensor for detecting the obstacle in a non-contact manner, and captures an image thereof using a camera device 22. The traveling robot 11 sends a notification of the detection of the obstacle and the captured image to the communication terminal 14 via the management server 12. The operator receives the notification of the detection of the obstacle by the LIDAR, observes the captured image, and determines whether the path is passable even if there is an obstacle.
[0215] When it is determined that the route is not passable, the operator instructs an alternative route or a route search.
[0216] Based on the instruction received from the communication terminal 14 via the management server 12, the traveling robot 11 performs a check on the instructed path, or searches for an alternative path and performs a check on the found path. The alternative path is a preset convenient path. In the path search, the order of the via points visited may or may not be specified.
[0217] Fig.29 The figure shows an example of a route search method. The route is used to capture images of checkpoint D001 in area 1, checkpoint D002 in area 2, checkpoints D003 and D005 in area 3, and checkpoint D004 in area 4. The current situation is that there is an obstacle on the road between the via points P12 and P14, and the road is impassable.
[0218] Based on the instruction from the management server 12 , the traveling robot 11 travels according to the path information and captures an image of the inspection object with the imaging device 22 as an inspection action at the inspection point D001 . The traveling robot 11 transmits the captured image to the management server 12 .
[0219] Fig.30 1 is a diagram showing an example of the inspection result DB stored in the management server 12. The inspection result DB stores the inspection point ID, coordinates, area name, and file name of the captured image in association with each other.
[0220] exist Fig.30 In the example shown, the image captured at the inspection point D001 is stored as the inspection result.
[0221] Fig.31 1 is a flowchart of an example of a path search. The path search is started in response to an instruction of a path search from an operator when it is determined that the path cannot pass through the traveling robot 11. The path search is performed by the traveling robot 11. Alternatively, the management server 12 may perform the search and notify the traveling robot 11 of the search result. The description is made assuming that there are four checkpoints, namely, checkpoints D001, D002, D003, and D005.
[0222] In step S180, the coordinates of the point ID (or inspection object ID) are registered as one of the via points, and the list of adjacent via points is corrected. When an inaccessible via point is found, the inaccessible via point is deleted. In step S181, the checkpoint list is retrieved to check the checkpoints D002, D003, and D005 associated with the inspection object ID selected as the destination or via point.
[0223] In step S182, the paths between the selected checkpoints are searched sequentially and the paths are determined.
[0224] Specifically, a path from check point D002 to D003 and a path from check point D003 to D005 are determined.
[0225] When there is a list structure of adjacent via points, the shortest path can be found by a known path finding algorithm. Examples of path finding algorithms include Dijsktra's algorithm and A* search. These path finding algorithms are well known and will not be described in detail here. Other known algorithms can be used in path searching.
[0226] Fig.32 FIG. 1 is a diagram showing an example of a graph structure used in a route search. In the graph structure, when a via point with a location ID of "P8" is registered, Fig.29In the example shown, there are via points P10 and P4 adjacent to via point P8. Therefore, "P10" and "P4" are registered as the first and second adjacent point IDs associated with point ID "P8". The operator selects one of the adjacent point IDs "P10" and "P4" to determine the path. In this example, "P4" is selected. Since via point P4 is adjacent to three via points P3, P8, and P5, its location ID is registered as the first, second, and third adjacent point IDs in association with location ID "P4". This process is repeated to determine the path.
[0227] Fig.33 1 is a flowchart of an example of the route search process. In step S191, the information processing system 10 receives input of point IDs associated with the inspection positions from the operator. It is assumed that "D001", "D002" and "D005" are input as point IDs.
[0228] In step S192, the management server 12 searches for a path and instructs the traveling robot 11 to start traveling. Alternatively, the traveling robot 11 may perform a path search. At the start of the inspection, a path is obtained that is traveled in the order of P4, P8, D001, P10, P12, P14, D002, P18, P19, P20, P21, P17, and D005.
[0229] In step S193, the traveling robot 11 travels on the retrieved path. In step S194, the traveling robot 11 detects obstacles and transmits the captured image, and the operator determines whether the path is passable.
[0230] When it is determined in step S194 that the path is impassable, the management server 12 lists the checkpoints that have not been checked yet in step S195. Then, the process returns to step S192. Therefore, the path is searched again, and the traveling robot 11 starts traveling. On the other hand, when the operator determines in step S194 that the path is passable, the inspection action is performed at the checkpoint in step S196. At the checkpoint, the traveling robot 11 captures an image with the camera 22 and transmits the captured image to the management server 12.
[0231] In step S197, the management server 12 determines whether the inspection object has an abnormality based on the received image and the determination criteria. When it is determined that there is an abnormality, in step S198, the management server 12 retrieves the inspection point to be inspected when the abnormality occurs, and instructs the traveling robot 11 to travel on the abnormal path and perform the inspection. Then, the process returns to step S192.
[0232] When the management server 12 determines that there is no abnormality in step S197, the management server 12 searches for the next checkpoint in step S199. In step S200, the management server 12 determines that there is a next checkpoint, and the process returns to step S193. The management server 12 controls the traveling robot 11 to travel on the path. If there is no next checkpoint, the inspection is ended.
[0233] If it is determined that the route is not passable in step S194, an alternative route is searched in step S192. At this time, the search is performed regardless of the order of the remaining checkpoints initially determined.
[0234] return Fig.29 To continue the explanation, in the initial search, the path from P12 to P14 is searched, and an obstacle is found between the via points P12 and P14 when the traveling robot 11 is traveling. If the obstacle makes the path impassable, although the checkpoint D001 is checked, the checkpoints D002 and D005 are not checked yet.
[0235] The path searched in the initial search is a path for checking the check points D002 and D005 in order. On the other hand, the path searched in the second search is a path for checking the check points D002 and D005 regardless of the order. The path obtained in this case is a path that proceeds in the order of P12, P22, P13, P15, D005, P17, P21, P20, P19, P18, P16, and D002.
[0236] Fig.34 FIG. 2 is a block diagram of an information processing system according to a second embodiment. Fig.34 In the example, the information processing system 10 includes a traveling robot 11, and the traveling robot 11 has Figure 1 All functions of the management server 12 shown in FIG. Figure 3 The hardware configuration of the traveling robot 11 has been explained, and thus a description thereof will be omitted.
[0237] Fig.35 Shown is a block diagram showing the functional configuration of the traveling robot 11 according to the second embodiment.
[0238] Fig.35 The traveling robot 11 shown includes Figure 5 All functional units of the traveling robot 11 shown in FIG. 1 and the functional units of the management server 12. Specifically, the traveling robot 11 also includes a map information management unit 87A, a path information management unit 87B, and an instruction unit 87C. Note that the functional units of the communication terminal 14 are the same as those of the management server 12. Figure 5 The functional units of the illustrated communication terminal 14 are the same, and thus description thereof will be omitted.
[0239] exist Figure 5 In the present invention, each of the traveling robot 11 and the management server 12 includes a sending and receiving unit, a determination unit, a storage and reading unit, and a storage unit as overlapping functional units. However, when the functional units of the management server 12 are included in the traveling robot 11, there is no need for duplicate functional units. Figure 5 Each functional unit has been described, so its description is omitted.
[0240] Fig.36 FIG. 1 is a flowchart showing an example of an inspection operation using the first method in the second embodiment.
[0241] The traveling robot 11 is set at a position to start the inspection action, and the inspection in the autonomous traveling mode is started. In step S211, the traveling robot 11 retrieves the inspection path. In step S212, the traveling robot 11 travels to the inspection point. In step S213, the traveling robot 11 performs the inspection action at the inspection point. The inspection action is the shooting of an image of the inspection object, etc.
[0242] In step S214, the traveling robot 11 determines whether there is an abnormality in the inspection object based on, for example, the captured image and the determination criteria. For example, the inspection object is the read value of the meter 1. When it is determined that there is an abnormality, in step S215, the traveling robot 11 retrieves the path for the abnormal situation, and the process returns to step S212.
[0243] When it is determined that there is no abnormality in step S214, the traveling robot 11 searches for the next checkpoint in step S216, and determines whether the next checkpoint exists in step S217. When the next checkpoint exists, the process returns to step S212, and the traveling robot 11 travels to the next checkpoint. On the other hand, when it is determined that there is no next checkpoint in step S217, the inspection action ends.
[0244] Fig.37 FIG. 1 is a flowchart showing an example of an inspection operation using the second method in the second embodiment.
[0245] The traveling robot 11 is set at a position to start the inspection action, and the inspection in the autonomous traveling mode is started. In step S221, the traveling robot 11 retrieves the inspection path, and in step S222, the traveling robot 11 travels to the inspection point. In step S223, the traveling robot 11 performs the inspection action at the inspection point. The inspection action is the shooting of an image of the inspection object, etc.
[0246] In step S224, the traveling robot 11 determines whether there is an abnormality in the inspection object based on, for example, the captured image and the determination criteria. For example, the inspection object is the reading value of the meter 1. When it is determined that there is an abnormality, in step S225, the traveling robot 11 retrieves the abnormality path and performs the inspection while traveling on the abnormality path. Then, the inspection action ends. When it is determined that there is no abnormality in step S224, the inspection action ends.
[0247] Fig.38 The flowchart of another example of the inspection action of the traveling robot 11 with the information prompting mechanism is shown. The traveling robot 11 is set at the position to start the inspection action, and the inspection in the autonomous traveling mode is started. In step S231, the traveling robot 11 retrieves the inspection path, and in step S232, the traveling robot 11 travels to the inspection point. In step S233, the traveling robot 11 determines whether gas is detected during the travel.
[0248] When gas is not detected, in step S234, the traveling robot 11 performs an inspection action at the inspection point. The inspection action is capturing an image of the inspection object, etc. On the other hand, when gas is detected, the process proceeds to step S236.
[0249] In step S235, the traveling robot 11 determines whether the inspection object has an abnormality based on, for example, the captured image and the determination criteria. For example, the inspection object is the reading value of the meter 1. When it is determined that there is an abnormality, in step S236, the traveling robot 11 searches for a path for abnormality.
[0250] When it is determined that there is no abnormality in step S235, in step S237, the traveling robot 11 searches for the next checkpoint and determines whether the next checkpoint is the last checkpoint. In the case where the next checkpoint is not the last checkpoint, the process returns to step S142. When the next checkpoint is the last checkpoint, in step S238, the traveling robot 11 travels to the end point and ends the inspection operation.
[0251] After determining that there is an abnormality, in step S240, the traveling robot 11 determines whether it needs to be shut down. When the traveling robot 11 determines that it needs to be shut down, in step S241, the traveling robot 11 performs information prompting. Then, in step S242, the traveling robot 11 shuts down. On the other hand, when it is determined in step S240 that it does not need to be shut down, the process returns to step S232, and the traveling robot 11 continues the inspection action.
[0252] As described above, according to the present disclosure, even when the cause of the abnormality exists at a position far from the monitored object, the cause of the abnormality can be identified. The travel of the traveling robot 11 is controlled by path data and position data of the checkpoint. The traveling robot 11 is controlled to sequentially track the passing point information of the path outside the inspection area. In the inspection area, the setting values of panning, tilting and zooming for taking an image of the inspection object are transmitted to the camera device 22, and the camera device 22 takes an image of the inspection object with the transmitted setting values. The captured image is used to determine the presence of an abnormality. If an abnormality exists, the path is switched to the path for the abnormal situation.
[0253] In the above, the inspection of a chemical plant is used as an example for explanation, but the monitoring using the traveling robot 11 is not only applicable to the inspection of a chemical plant, but also applicable to the inspection of other places, security, etc. In terms of security, when an abnormality such as a broken window is detected, the cause of the abnormality can be identified by taking images of the entrance and exit, etc. In addition, the monitoring using the traveling robot 11 can also be applied to the fields of medical care and nursing. For example, in the case where a person has nausea and an abnormality is detected, the cause of the abnormality can be identified by taking images of the food or drink that the person ate before falling.
[0254] Below, refer to Fig.39 , an example is described in which the inspection area to be inspected is a room of a user of a medical facility such as a hospital. Various sensors are provided in the room. Based on the detection results of such sensors, a presence / absence indication indicating whether there is someone in the room (whether the room is empty) is provided outside the room. At night, the traveling robot 11 patrols the facility to obtain images of presence / absence indications of each room. When it is determined from the captured image that the room is empty, the captured situation is abnormal. Then, the path is switched to a path for abnormal situations, and, for example, an infrared camera can be used to capture images of the corridor at fixed intervals. This is just an example. The path for abnormal situations can be, for example, a path for performing thermal imaging detection of specific positions corresponding to the users in each room.
[0255] Fig.39 The figure shows an example of an abnormality determination in which an item of "check point", "image comparison" and "action" (indicating an abnormal path ID) is stored in association with each abnormality ID. The image comparison indicates the degree of matching with an image obtained by shooting an image indicating that a room is empty with or without a person. When the degree of matching between the image shot at the time of the check and the image indicating that a room is empty with or without a person is high, the situation can be determined as abnormal.
[0256] In addition, there are cases where the occurrence of an abnormality can be predicted from the appearance change of the equipment before the abnormality is detected. Examples of predicting the occurrence of an abnormality from the appearance change include the occurrence of rust on the equipment and the displacement of components. The occurrence of an abnormality can be predicted not only from the change in appearance, but also from the change in sound, etc.
[0257] The same equipment may be installed in different locations or different areas. In this case, data such as status information (e.g., captured images) indicating changes in appearance or sound obtained by various sensors of the traveling robot 11 are collected and stored in the management server 12, and the collected data is analyzed. The analysis results are transmitted to an operator who uses the same equipment in another location or area. Then, the operator in another location or area can predict the occurrence of an abnormality and take preemptive measures for the abnormality. Preemptive measures are, for example, but not limited to, applying anti-rust treatment, correcting the displacement of components, and replacing components.
[0258] Fig.40 FIG. 2 is a block diagram of an information processing system according to a third embodiment. Fig.40 In the information processing system 10 shown, one or more traveling robots 11a to 11z are arranged in each of a plurality of locations 1 to N. The management server 12 collects information or data from each traveling robot 11a to 11z, analyzes the collected information, and notifies the operator of the management location of the analysis result.
[0259] exist Fig.40 In the example shown, two communication terminals 14a and 14b (which may be collectively referred to as "communication terminals 14") are shown in the management site, but the number of communication terminals installed in the management site is not limited to this, and the number of management sites is not limited to one.
[0260] The number of traveling robots 11a to 11z installed at each location is not limited to one, and a plurality of traveling robots 11a to 11z may be installed according to the number of regions at each location. Fig.40 In the example shown, two traveling robots 11a to 11z (hereinafter, they may be collectively referred to as "traveling robots 11") are arranged at each location 1 to N. Each traveling robot 11a to 11z has an imaging device 22 and the like, and captures an image of a designated monitoring object.
[0261] Fig.41 FIG. 1 is a block diagram showing an example of the functional configuration of the traveling robot, the management server, and the communication terminal according to the third embodiment. The management server 12 supports cloud computing services such as AMAZONWEBSERVICE. In the information processing system 10, the traveling robot 11 and one or more communication terminals 14 communicate with each other via the management server 12.
[0262] The management server 12 can improve the security of data such as manual operation commands from the communication terminal 14 and captured images from the traveling robot 11 by using authentication processing of the cloud computing service during communication. The authentication may be authentication using a user ID and password, biometric authentication using biometric information such as a fingerprint, or multi-factor authentication using a combination of two or more factors.
[0263] In addition, the management server 12 has the ability to generate and manage data, and the same data can be shared by multiple locations or regions. Therefore, the management server 12 not only handles point-to-point communication (one-to-one direct communication), but also flexibly handles communication in one-to-many locations. Therefore, the operator can not only operate an arbitrary traveling robot 11 in the same location or the same area from one communication terminal 14 via the management server 12, but also can operate multiple traveling robots 11 in the same location or the same area, and can also operate multiple traveling robots 11 in different locations or areas. In addition, in each of the multiple locations or areas, the traveling robot 11 and the communication terminal 14 can be used as a group, and each traveling robot 11 can be operated by any communication terminal 14.
[0264] because Fig.41 The traveling robot 11 includes Figure 5 The functional units shown in FIG. 1 are the same as the functional units shown in FIG. 1 , and thus the description of the same functional units is omitted. Figure 5 In addition to the transmitting and receiving unit 100 shown, Fig.41 The management server 12 also includes a degradation determination unit 131 and a degradation information DB 130 in the storage unit 106. The degradation information DB 130 stores degradation information. The degradation information is information on changes in appearance, but is not limited thereto, and may be information on changes in sound, etc.
[0265] The degradation determination unit 131 of the management server 12 refers to the degradation information stored in the degradation information DB 130 under the control of the camera control unit 82 of the traveling robot 11, and determines the degradation state based on the image captured by the camera 22. The degradation determination unit 131 compares the degradation information with the captured image. The degradation determination unit 131 determines whether rust, displacement of components, etc. have occurred based on the image comparison, and determines whether degradation has progressed even if no abnormality has occurred.
[0266] The degradation determination unit 131 transmits the degradation state as a determination result to the communication terminal 14 operated by the operator. The degradation state is information indicating which device in which location (or which area) is degraded.
[0267] The communication terminal 14 is Figure 5In addition to the transmission and reception unit 120 and the like shown, there is also a notification unit 140 that receives the degradation state as a result of the determination from the management server 12, and notifies the operator of the degradation state, for example, via the display control unit 122. When receiving the notification of the degradation state from the notification unit 140, the operator can predict the occurrence of an abnormality and can take countermeasures in advance.
[0268] Fig.42 1 is a diagram showing an example of degradation information managed in the degradation information management table in the degradation information DB 130. The traveling robot 11 transmits captured images to the management server 12 at predetermined traveling intervals and predetermined time intervals during inspection to obtain information for determining degradation.
[0269] The travel interval and time interval can be appropriately set according to the device to be photographed. The device may be, for example, Figure 6 The tank 200 or the pump 201 shown may be a single device, or may be a group of devices and pipes, valves, meters, etc. The time interval may be determined in consideration of the deterioration rate of the device, the replacement time, the magnitude of the risk caused by the downtime of the device, etc. The traveling robot 11 may capture an image of a specific location at a specific time and transmit the captured image to the management server 12.
[0270] The degradation information management table stores the image information of the captured image in association with the device information (device ID) used in the location or area where the image was captured and the date and time of the image capture. At this time, the column of the degradation information flag is blank. The device ID may be the product number when the device was purchased, or may be a unique ID assigned by the owner of the device for management purposes. The image information may be any information that identifies the image, such as the file name of the image and the storage location of the file of the image.
[0271] Next, the degradation information flag of the degradation information management table will be explained. The determination unit 101 of the management server 12 is used as a determination means, and determines whether there is an abnormality in the state of the monitored equipment based on the captured images obtained from the traveling robot 11 during the inspection, the sensor detection results, etc. Then, when the determination unit 101 determines that there is an abnormality, the management server 12 refers to the degradation information management table of the degradation information DB 130. Then, among the captured images associated with the same device ID as the device determined to be abnormal, the management server 12 sets a flag for the image captured at a specified time before the date and time determined to be abnormal as a degraded image. Fig.42In the example shown, the device with device ID PS1 is checked every day, and an image of the entire device is acquired at the inspection point D001. When the determination unit 101 determines that there is an abnormality in the device with device ID "PS1" on March 25, 2022, the degradation information flag "1" is stored in association with the information obtained at 12:00 on March 24, 2022, which is a specified time before the time when the abnormality was determined (one day before). Note that the function of the determination unit 101 as a determination means is the same as Figure 5 The date and time determined to be abnormal are, for example, the date and time determined to be executed Fig.12 or Fig.26 The abnormality determination of the determination unit 101 may be any abnormality determination related to the device having the device ID "PS1", and there is no limitation on means such as sensors used for determination. That is, as described above, determination may be made based on the detection results of a gas sensor or a sound recorder.
[0272] exist Fig.42 In the example shown, the prescribed time (before determination) is one day, but is not limited thereto, and may be, for example, several hours or several days according to the time interval of image capturing.
[0273] Reference Fig.43 , to illustrate the use of Fig.42 During inspection, the imaging device 22 of the traveling robot 11 captures an image of the inspection object device at a specified travel interval, at a specified time, or at a specified time interval under the control of the imaging control unit 82 (S11). The transmitting and receiving unit 80 of the traveling robot 11 transmits the device ID together with the image information of the captured image (S12).
[0274] The degradation information DB 130 stores the image information in association with the device ID, and the degradation determination unit 131 of the management server 12 acquires an image having the same device ID as the acquired device ID and having image information associated with the degradation information flag ( S13 ).
[0275] The degradation determination unit 131 of the management server 12 compares the captured image acquired from the traveling robot 11 with the image retrieved from the degradation information DB 130 and calculates a matching degree indicating the similarity between the captured image and the retrieved image. The matching degree can be calculated using any known method, such as pattern matching.
[0276] The degradation determination unit 131 of the management server 12 sets a threshold value for the degree of matching. When the degree of matching is equal to or higher than the threshold value, the degradation determination unit 131 determines that the degree of matching is high and the degradation of the equipment in the location or area where the traveling robot 11 is operating has progressed (S14). When it is determined that the degradation has progressed, the degradation determination unit 131 of the management server 12 sends an instruction to prompt a text message about the degradation to the communication terminal 14 via the sending and receiving unit 100 (S15).
[0277] The transmitting and receiving unit 120 of the communication terminal 14 receives the instruction from the management server 12, and the notifying unit 140 of the communication terminal 14 prompts a text message about the degradation via the display control unit 122 (S16). The text message about the degradation may include information about the device that has degraded and the location or area where the device is located.
[0278] When it is determined that the matching degree is lower than the threshold and the matching degree is low, the degradation determination unit 131 of the management server 12 may send an indication message to the communication terminal 14 via the sending and receiving unit 100 to prompt a text message indicating that the degradation has not progressed, or may not send any information to the communication terminal 14.
[0279] Even if the devices are located in different places or regions, the management server 12 compares images having the same device information. Therefore, before an abnormality occurs, the management server 12 can report the abnormality using the degradation state of different places or regions and prompt the user to take countermeasures in advance.
[0280] Fig.44 The figure shows an example of a text message about degradation displayed on the screen of the communication terminal 14. The communication terminal 14 receives an instruction from the management server 12 and displays a text message 300 about degradation on the display screen. The content of the text message 300 is, for example, "The image information shows degradation of the piping system PS1". In this example, the degradation notification is performed by displaying the text message, but in addition to displaying the text message, the notification can also be performed in the form of, for example, sound.
[0281] exist Fig.44 In the example, a text message 300 having the above content is displayed on an image captured by the traveling robot 11. The displayed screen may include a captured image 301 as an image of the surroundings of the traveling robot 11, an emergency stop button 302, an autonomous traveling end button 303, a home button 304, a traveling path map 305, and a status display 306. In addition, these buttons are only examples, and only a part of them may be provided, or other buttons may be provided.
[0282] The image 301 was captured at Figure 6The image is captured at a location near the check point D001 shown in FIG. 3. The emergency stop button 302 can be used to input an action such as temporarily stopping the traveling of the traveling robot 11.
[0283] The emergency stop button 302 is a visual representation of the reception unit 121 of the communication terminal 14 receiving an instruction of an emergency stop from the operator. When the emergency stop button 302 is selected again after being selected for an emergency stop, the emergency stop button 302 can receive a cancellation of a temporary stop for resuming autonomous travel. The autonomous travel end button 303 is used to switch the traveling robot 11 from the autonomous traveling mode to the manual traveling mode. The home button 304 is used to switch to the home screen. The traveling path diagram 305 displays the traveling path of the traveling robot 11 and the position of the traveling robot 11 on the traveling path. The status indication 306 displays the status of the traveling robot 11 in the autonomous traveling mode, such as autonomous traveling or temporary stop.
[0284] As described above, when the management server 12 determines that a specific monitored object has an abnormality, it obtains degradation information (captured images, etc.) related to the monitored object. The degradation information is information that is acquired before a specified time when it is determined that an abnormality exists and has the same attribute information (device ID, etc.) as the specific monitored object. At this time, the management server 12 is able to determine the degradation state based on the acquired information and the degradation information, and send an instruction to the communication terminal 14 to report the degradation state of the monitored object having the same attribute information as the specific monitored object. The communication terminal 14 then receives the instruction and notifies the operator by displaying a text message related to the degradation, etc. Therefore, before the abnormality is detected, the occurrence of the abnormality can be predicted, and appropriate measures can be taken before damage such as a dangerous gas leak occurs.
[0285] The above embodiments are illustrative and do not limit the present disclosure. The above embodiments can be modified within the scope that can be imagined by those skilled in the art. The modification includes adding another component and changing or deleting one of the above components. As long as the function and effect of the present disclosure are provided, such modification is within the scope of the present disclosure.
[0286] The present disclosure has the following aspects.
[0287] The first aspect is about a control system (control server) for controlling a moving body. The control system has an instruction unit to instruct the moving body to move on a first path, and obtain the status information of a first monitored object (specific monitored object) on the first path. When it is determined that the first monitored object is abnormal according to the obtained status information, the instruction unit instructs the moving body to move on a second path different from the first path, and obtains the status information of a second monitored object (a designated part related to the first monitored object) on the second path.
[0288] In a second aspect, the control system according to the first aspect further includes: a determination unit that determines whether there is an abnormality in the state of the first monitored object based on the acquired state information.
[0289] In a third aspect, the control system according to the first or second aspect further includes: a receiving unit that receives status information acquired by the traveling body, and the receiving unit receives one or both of an image captured by the traveling body and a sound recorded by the traveling body as the status information.
[0290] In a fourth aspect, in the control system according to the third aspect, the instruction unit instructs the traveling body to obtain status information of one or more second monitored objects different from the first monitored object and located in the monitored area where the first monitored object is located. The instruction unit instructs the traveling body to perform one or both of image capture of the status of the one or more second monitored objects and recording of the sounds of the one or more second monitored objects.
[0291] In a fifth aspect, in the control system according to any one of the first to fourth aspects, when there are a plurality of second monitored objects, the instructing unit instructs the traveling body to obtain status information of the plurality of second monitored objects one by one until the cause of the abnormality is determined.
[0292] In a sixth aspect, in the control system according to any one of the first to fourth aspects, in the case where there are a plurality of second monitored objects, the instructing unit instructs the traveling body to acquire the state information of all of the plurality of second monitored objects.
[0293] In a seventh aspect, in the control system according to the second aspect, a storage unit and a degradation determination unit are further included. The storage unit stores the state information obtained before the specified time when the determination unit determines that an abnormality exists as degradation information. When the state information of another monitored object having the same attribute information as the attribute information of the first monitored object is obtained, the degradation determination unit determines the degradation state of the monitored object having the same attribute information as the attribute information of the first monitored object based on the obtained state information and the degradation information stored in the storage unit.
[0294] An eighth aspect relates to an information processing system, comprising a control system according to any one of the first to seventh aspects, and one or more moving bodies controlled by the control system.
[0295] In the ninth aspect, in the information processing system according to the eighth aspect, the moving body includes a moving mechanism so that the moving body travels on a first path indicated by a control server or a second path different from the first path; a status information acquisition unit, which acquires status information of a first monitored object and status information of a second monitored object, and a sending unit, which sends the status information of the first monitored object or the status information of the second monitored object to the control system.
[0296] In a tenth aspect, in the information processing system according to the eighth or ninth aspect, a monitoring area monitored by the information processing system is divided into a plurality of areas, each area being monitored by one or more traveling bodies.
[0297] In an eleventh aspect, in the information processing system according to any one of the eighth to tenth aspects, the moving body includes: a state information acquisition unit that acquires surrounding state information indicating the state of the surrounding of the moving body; and a determination unit that determines whether there is an abnormality in the surrounding of the moving body based on the surrounding state information acquired by the state information acquisition unit. When the prompt unit determines that there is an abnormality in the state of the surrounding of the moving body, the prompt unit prompts information about the abnormality.
[0298] In a twelfth aspect, in the information processing system according to the eleventh aspect, the presenting unit presents the information related to the abnormality by raising a flag, raising a balloon, spraying powder, or a combination of two or more thereof.
[0299] In a thirteenth aspect, in the information processing system according to any one of the seventh to twelfth aspects, after the information is presented by the presentation unit, the traveling body performs an action responsive to the abnormality.
[0300] In a fourteenth aspect, in the information processing system according to the thirteenth aspect, the action in response to the abnormality includes turning off the power supply of the traveling body.
[0301] In a fifteenth aspect, in the information processing system according to any one of the eighth to fourteenth aspects, the traveling body travels in a factory as a monitoring area and acquires the status information.
[0302] In a sixteenth aspect, in the information processing system according to any one of the eighth to fourteenth aspects, the traveling body travels in a medical facility as a monitoring area and acquires the status information.
[0303] According to a seventeenth aspect, the information processing system according to the ninth aspect further includes a communication terminal that receives a notification instruction to notify an operator of a degradation state of another monitored object having attribute information identical to that of the first monitored object. The communication terminal includes a notification unit that prompts the operator with information related to degradation of the monitored object having attribute information identical to that of the first monitored object based on the received notification instruction.
[0304] The eighteenth aspect is about a moving body including a control system. The control system has an indication unit to indicate that the moving body moves on a first path, and obtains the status information of a first monitored object on the first path. When it is determined that the first monitored object is abnormal according to the obtained status information, the indication unit indicates that the moving body moves on a second path different from the first path, and obtains the status information of a second monitored object on the second path.
[0305] In the nineteenth aspect, the moving body according to the eighteenth aspect includes a moving mechanism to enable the moving body to travel on a first path indicated by a control server or a second path different from the first path; a status information acquisition unit, which acquires status information of a first monitored object and status information of a second monitored object, and a sending unit, which sends the status information of the first monitored object or the status information of the second monitored object to the control system.
[0306] In a twentieth aspect, the moving body according to the eighteenth or nineteenth aspect comprises: a state information acquisition unit that acquires surrounding state information indicating a state around the moving body; and a determination unit that determines whether there is an abnormality around the moving body based on the state information around the moving body acquired by the state information acquisition unit. When the prompt unit determines that there is an abnormality in the state around the moving body, the prompt unit prompts information about the abnormality.
[0307] A twenty-first aspect relates to a method for controlling a moving body using a computer. The method includes instructing the moving body to move on a first path and obtaining status information of a first monitored object on the first path. The method also includes, when it is determined that the first monitored object is abnormal based on the obtained status information, an instructing unit instructs the moving body to move on a second path different from the first path and obtains status information of a second monitored object on the second path.
[0308] A twenty-second aspect relates to a recording medium storing a plurality of program codes, and when a computer executes the program codes, the computer executes a method for controlling a moving body. The method includes instructing the moving body to move on a first path, and obtaining status information of a first monitored object on the first path. The method also includes, when it is determined that the first monitored object is abnormal according to the obtained status information, an instruction unit instructs the moving body to move on a second path different from the first path, and obtains status information of a second monitored object on the second path.
[0309] The above embodiments are illustrative and do not limit the present invention. Therefore, in view of the above teachings, many additional modifications and variations can be made. For example, within the scope of the present invention, elements and / or features of different illustrative embodiments can be combined with each other and / or replaced with each other. Any of the above actions can be performed in various other ways, for example, in an order different from the above order.
[0310] The present invention can be implemented in any convenient form, such as using dedicated hardware or a mixture of dedicated hardware and software. The present invention can be implemented by computer software executed by one or more networked process devices. The network can include any conventional terrestrial or wireless communication network, such as the Internet. The processing device includes any device suitable for programming, such as a general-purpose computer, a personal digital assistant, a wireless application protocol (WAP) or a third generation (3G) compatible mobile phone, etc. Since the present invention can be implemented by software, all aspects of the present invention include computer software that can be implemented on a programmable device. Computer software can be provided to a programmable device using any conventional carrier medium (carrier device). The carrier medium includes a transient carrier medium, such as an electrical, optical, microwave, acoustic or radio frequency signal carrying a computer code. An example of such a transient medium is a transmission control protocol / internet protocol (TCP / IP) signal carrying a computer code on an IP network such as the Internet. The carrier medium can also include a storage medium for storing processor readable code, such as a floppy disk, a hard disk, a compact disc read-only memory (CD-ROM), a magnetic tape device or a solid-state memory device. The computer software may be provided to a programmable device using any storage medium that stores processor readable code, such as a floppy disk, hard disk, CDROM, magnetic tape device, or solid state memory device.
[0311] The hardware platform includes any desired type of hardware resources, such as a central processing unit (CPU), a random access memory (RAM), and a hard disk drive (HDD). The CPU can be implemented by any desired number of processors of any desired type. The RAM can be implemented by any desired type of volatile or non-volatile memory. The HDD can be implemented by any desired type of non-volatile memory capable of storing a large amount of data. Depending on the type of device, the hardware resources may additionally include input devices, output devices, or network devices. Alternatively, the HDD may be disposed outside the device as long as the HDD is accessible. In this example, the CPU (such as a cache memory of the CPU) and the RAM can be used as physical memory or primary memory of the device, and the HDD can be used as secondary memory of the device.
[0312] The functions of the elements disclosed herein can be performed using circuits or processing circuits, which include general-purpose processors, special-purpose processors, integrated circuits, application-specific integrated circuits (ASICs), digital signal processors (DSPs), field programmable gate arrays (FPGAs), conventional circuits, and / or combinations thereof configured or programmed to perform the disclosed functions. The processor can be a processing circuit or circuit including transistors and other circuits therein. In the present disclosure, a circuit, unit, or device is hardware that performs or is programmed to perform the functions. Hardware can be any hardware that is disclosed herein or otherwise known to be programmed or configured to perform the functions. When hardware is a processor that can be considered a type of circuit, a circuit, device, or unit is a combination of hardware and software, and software is used to configure hardware and / or processors.
[0313] This patent application is based upon and claims the benefit of priority from Japanese Patent Application No. 2022-046195 filed with the Japan Patent Office on March 23, 2022, and Japanese Patent Application No. 2023-011242 filed with the Japan Patent Office on January 27, 2023, the disclosures of which are incorporated herein in their entirety by reference.
[0314] Reference numerals list
[0315] 10 Information Processing System
[0316] 11,11a~11z Mobile Robot
[0317] 12. Manage Server
[0318] 13 Communication Network
[0319] 14, 14a, 14b communication terminal
[0320] 20 Controllers
[0321] 21 Shell
[0322] 22Camera equipment
[0323] 23 Support components
[0324] 24 mobile mechanism
[0325] 25 Tips Agency
[0326] 30CPU
[0327] 31ROM
[0328] 32RAM
[0329] 33HDD controller
[0330] 34HD
[0331] 35Media I / F
[0332] 36 Input and Output I / F
[0333] 37 Sound input / output I / F
[0334] 38 Network I / F
[0335] 39 Short distance communication circuit
[0336] 40 antenna
[0337] 41 External device I / F
[0338] 42 bus lines
[0339] 43 Recording Media
[0340] 44 Display
[0341] 45 Microphones
[0342] 46 speakers
[0343] 47 drive motor
[0344] 48 acceleration and orientation sensors
[0345] 49GPS sensor
[0346] 50 batteries
[0347] 51 Sensors
[0348] 60CPU
[0349] 61ROM
[0350] 62RAM
[0351] 63HD
[0352] 64HDD Controller
[0353] 65 Display
[0354] 66 External device I / F
[0355] 67 Network I / F
[0356] 68 bus lines
[0357] 69 Keyboard
[0358] 70 fixed point equipment
[0359] 71 Sound input / output I / F
[0360] 72 Microphones
[0361] 73 speakers
[0362] 74 cameras
[0363] 75DVD-RW drive
[0364] 76Media I / F
[0365] 77DVD-RW
[0366] 78 Recording Media
[0367] 80 send and receive units
[0368] 81 judgment unit
[0369] 82 Camera control unit
[0370] 83 Status information acquisition unit
[0371] 84 Position information acquisition unit
[0372] 85 Candidate destination acquisition unit
[0373] 86 Path information generation unit
[0374] 87 Destination Setting Unit
[0375] 88 Travel Control Unit
[0376] 89 Image Recognition Unit
[0377] 90 mode setting unit
[0378] 91 Autonomous Travel Unit
[0379] 92 Manual Action Processing Unit
[0380] 93 Mission Execution Unit
[0381] 94 Image Processing Unit
[0382] 95 study units
[0383] 96 storage and reading units
[0384] 97 storage units
[0385] 100 sending and receiving units
[0386] 101 judgment unit
[0387] 102 Indicator Unit
[0388] 103 Map Information Management Unit
[0389] 104 Path Information Management Unit
[0390] 105 storage and reading unit
[0391] 106 storage units
[0392] 107 Candidate Destination Management DB
[0393] 108 Map Information Management DB
[0394] 109 Learning Data Management DB
[0395] 110 Route information management DB
[0396] 120 sending and receiving units
[0397] 121 receiving unit
[0398] 122 Display control unit
[0399] 123 judgment unit
[0400] 124 Manual action command generation unit
[0401] 125 Autonomous travel request information generation unit
[0402] 126 Image Processing Unit
[0403] 127 storage and reading units
[0404] 128 memory cells
[0405] 130 Degradation Information DB
[0406] 131 Degradation determination unit
[0407] 140 Notification Unit
[0408] 200 cans
[0409] 201 Pump
[0410] 202, 203 valve
[0411] 204, 205 flow meter
[0412] 300 SMS
[0413] Image captured by 301
[0414] 302 Emergency stop button
[0415] 303 Autonomous travel end button
[0416] 304 Home Button
[0417] 305 Travel Path Map
[0418] 306 status means
Claims
1. A control server for controlling a moving object, comprising: an instruction unit, which instructs the moving body to move on a first path and obtains state information of a first monitored object on the first path, in, Based on a determination that an abnormality has occurred in the state of the first monitored object from the acquired state information, the instructing unit instructs the moving body to move on a second path different from the first path, and acquires state information of a second monitored object different from the first monitored object and located on the second path.
2. The control server according to claim 1, further comprising: A determination unit determines whether there is an abnormality in the state of the first monitored object based on the acquired state information.
3. The control server according to claim 1 or 2, further comprising: a receiving unit configured to receive the state information acquired by the moving body, The status information is an image captured by the moving object, a sound recorded by the moving object, or a combination of the captured image and the recorded sound.
4. The control server according to claim 3, characterized in that: The second monitored object is located in the monitoring area where the first monitored object is located. The state information of the second monitored object is an image captured by the traveling object, a sound recorded by the traveling object, or a combination of the captured image and the recorded sound.
5. The control server according to any one of claims 1 to 4, characterized in that : In a case where the second monitored object includes a plurality of second monitored objects, the instructing unit instructs the traveling body to acquire the status information of the plurality of second monitored objects one by one until the cause of the abnormality is determined.
6. The control server according to any one of claims 1 to 4, characterized in that : In a case where the second monitored object includes a plurality of second monitored objects, the instructing unit instructs the traveling object to obtain the status information of all of the plurality of second monitored objects.
7. The control server according to claim 2, characterized in that Also includes: a storage unit that stores, as degradation information, the state information of the first monitored object obtained a predetermined time before the moment when the determination unit determines that an abnormality exists, and A degradation determination unit determines a degradation state of another monitored object having attribute information identical to that of the first monitored object based on state information of the other monitored object and the degradation information of the first monitored object stored in the storage unit.
8. An information processing system, characterized in that include: A control server according to any one of claims 1 to 7, and One or more moving bodies controlled by a control server.
9. The information processing system according to claim 8, characterized in that Each of the one or more moving bodies comprises: a travel control unit configured to control the traveling body to travel on the first path indicated by the control server or on the second path different from the first path; a state information acquisition unit, which acquires the state information of the first monitored object and the state information of the second monitored object, and A sending unit is configured to send the state information of the first monitored object and the state information of the second monitored object to the control server.
10. The information processing system according to claim 8 or 9, characterized in that: The one or more moving bodies are a plurality of moving bodies, and The monitoring area monitored by the information processing system is divided into a plurality of areas, each area being monitored by one or more of the plurality of traveling bodies.
11. The information processing system according to any one of claims 8 to 10, characterized in that Each of the one or more moving bodies comprises: a state information acquisition unit, which acquires surrounding state information indicating a state around the moving body; a determination unit that determines whether there is an abnormality around the moving body based on the surrounding state information acquired by the state information acquisition unit, and A prompting unit that, based on the determination by the determination unit that there is an abnormality in the state around the traveling body, presents information about the abnormality to the surroundings of the traveling body.
12. The information processing system according to claim 11, characterized in that: The prompting unit performs one or a combination of raising a flag, raising a balloon, and spraying powder to prompt the information about the abnormality.
13. The information processing system according to claim 11 or 12, characterized in that: The moving body performs an action in response to the abnormality after the prompting unit prompts the information.
14. The information processing system according to claim 13, characterized in that: The action in response to the abnormality includes shutting down power to the traveling body.
15. The information processing system according to any one of claims 8 to 14, characterized in that: The monitoring area monitored by the one or more traveling bodies is in a factory.
16. The information processing system according to any one of claims 8 to 14, characterized in that : The monitoring area monitored by the one or more traveling objects is in a medical facility.
17. The information processing system according to claim 9, characterized in that Also includes: a communication terminal receiving a notification instruction to notify an operator of a degradation state of another monitored object having the same attribute information as the first monitored object, The communication terminal includes a notification unit configured to present information related to degradation of the other monitored object to the operator based on the received notification instruction.
18. A moving body, characterized in that include: an instruction unit, which instructs the moving body to move on a first path and obtains state information of a first monitored object on the first path, Based on a determination that an abnormality has occurred in the state of the first monitored object from the acquired state information, the instructing unit instructs the moving body to move on a second path different from the first path, and acquires state information of a second monitored object different from the first monitored object and located on the second path.
19. The walking body according to claim 18, characterized in that: A walking control unit is also provided to control the walking body to walk on the first path or the second path.
20. The traveling body according to claim 18 or 19, characterized in that Also includes: a state information acquisition unit, which acquires surrounding state information indicating a state around the moving body; a determination unit that determines whether there is an abnormality in the state of the surroundings of the moving body based on the surrounding state information acquired by the state information acquisition unit, and A prompting unit that, based on the determination by the determination unit that there is an abnormality in the state around the traveling body, presents information about the abnormality to the surroundings of the traveling body.
21. A method for controlling a moving object, characterized in that include: instructing the moving body to move on a first path, and obtaining status information of a first monitored object on the first path, and Based on the determination that an abnormality has occurred in the state of the first monitored object from the acquired state information, the moving body is instructed to travel on a second path different from the first path, and state information of a second monitored object different from the first monitored object and located on the second path is acquired.
22. A recording medium storing a plurality of program codes, which, when executed by one or more processors, causes the processors to execute a method for controlling a moving body, the method comprising: instructing the moving body to move on a first path, and obtaining status information of a first monitored object on the first path, and Based on the determination that an abnormality has occurred in the state of the first monitored object from the acquired state information, the moving body is instructed to travel on a second path different from the first path, and state information of a second monitored object different from the first monitored object and located on the second path is acquired.
