Management device, management system, and management method

Abstract

To enable a moving body to automatically recover its own position when it loses sight of its own position. [Solution] A management device comprising: an image storage unit that stores multiple environmental images captured within a specified area; an image acquisition unit that acquires captured images captured by a mobile body that moves autonomously within the specified area; a judgment unit that, when the mobile body loses its own position, determines whether the captured image acquired by the image acquisition unit matches each of the multiple environmental images; and a remote control unit that controls the operation of the mobile body that has lost its own position based on the judgment result by the judgment unit.

Description

[Technical Field]

[0001] The present invention relates to a management device, a management system, and a management method. [Background technology]

[0002] In recent years, with the advancement of robot-related technologies, development is underway on systems that operate autonomously performing robots (hereinafter referred to as mobile objects) as a means of resolving labor shortages resulting from population decline.

[0003] For example, systems have been developed that use map data within or between buildings and technology such as artificial intelligence (AI) to allow a mobile object to move autonomously within an area indicated by the map data.

[0004] However, currently, autonomously moving mobile bodies may encounter events that are difficult for the mobile body to deal with or judge autonomously.

[0005] Specifically, an example of an event that is difficult for a mobile body to deal with or judge autonomously is when the mobile body loses track of its own position (gets lost). Possible causes of a mobile body losing its own position include, for example, when the mobile body moves outside of the map data it holds in an attempt to avoid a person, or when the mobile body is manually moved outside of the map data it holds.

[0006] In such cases, the mobile body can recover its own position by requesting remote operation from an operator in a remote location and having the operator remotely operate the mobile body to return it to a specified location (e.g., a charging location, a waiting location, or a home point).

[0007] In addition, Patent Document 1 listed below discloses a technology that connects a communication robot to the terminal of an operator who meets the conditions required by the conversation partner when the communication robot determines that it is difficult to respond through autonomous control. [Prior art documents] [Patent documents]

[0008] [Patent Document 1] Japanese Patent Application Laid-Open No. 2009-90420 Summary of the Invention [Problem to be solved by the invention]

[0009] However, there is a problem in that it is time-consuming and cumbersome for the operator to request remote control every time the mobile unit loses track of its own position.

[0010] Therefore, the present invention has been made in consideration of the above problems, and an object of the present invention is to provide a new and improved management device, management system, and management method that can automatically restore a moving body's own position when it loses track of its own position. [Means for solving the problem]

[0011] In order to solve the above problem, according to one aspect of the present invention, a management device is provided, comprising: an image storage unit that stores a plurality of environmental images captured within a specified area; an image acquisition unit that acquires captured images captured by a moving body that moves autonomously within the specified area; a determination unit that, when the moving body loses its own position, determines whether the captured image acquired by the image acquisition unit matches each of the plurality of environmental images; and a remote control unit that controls the operation of the moving body that has lost its own position based on the determination result by the determination unit.

[0012] If the determination unit determines that the captured image matches one of the multiple environmental images, the remote control unit may determine that the moving body is present at a coordinate position corresponding to the environmental image that matches the captured image, and cause the moving body to restore its own position.

[0013] If the determination unit determines that the captured image matches one of the multiple environmental images, the remote control unit may display a map of the specified area, with the coordinate position indicated, on an operation terminal that monitors the moving object.

[0014] When the determination unit determines that the captured image does not match any of the plurality of environmental images, the remote control unit may move the moving object.

[0015] The remote control unit may move the moving object in a direction that reduces a difference between the captured image and an environmental image that most closely matches the captured image.

[0016] The remote control unit may move the moving object in a direction that avoids an obstacle present around the moving object.

[0017] When the moving object is moved by the remote control unit, the determination unit may determine whether an image captured by the moving object after the movement matches each of the plurality of environmental images.

[0018] If, after the moving body has been moved a predetermined number of times, the determination unit determines that the captured image does not match each of the multiple environmental images, the remote control unit may send a notification to an operating terminal monitoring the moving body requesting remote operation of the moving body.

[0019] When the moving object loses track of its own position, an alert indicating that the moving object has lost track of its own position may be transmitted to an operation terminal that monitors the moving object.

[0020] In addition, in order to solve the above problem, according to another aspect of the present invention, there is provided a management system including a management device having: a mobile body that can autonomously move within a specified area and acquire captured images of the surrounding environment; an image storage unit that stores a plurality of environmental images captured within the specified area; a determination unit that, when the mobile body loses its own position, determines whether the captured image captured by the mobile body matches each of the plurality of environmental images; and a remote control unit that controls the operation of the mobile body that has lost its own position based on the determination result by the determination unit.

[0021] In addition, in order to solve the above problem, according to another aspect of the present invention, a management method is provided which includes storing a plurality of environmental images taken within a specified area, and when a moving body moving autonomously within the specified area loses its own position, determining whether the image taken by the moving body matches each of the plurality of environmental images, and controlling the operation of the moving body that has lost its own position based on the result of the determination.

[0022] With the above configuration, the position coordinates of the moving body are identified by determining whether an image captured by the moving body matches each of a plurality of environmental images captured in advance within a predetermined area. [Effects of the Invention]

[0023] As described above, according to the present invention, when a moving body loses track of its own position, it is possible to automatically recover its own position. [Brief explanation of the drawings]

[0024] [Figure 1] 1 is a block diagram showing a functional configuration of a management system according to an embodiment of the present invention; [Figure 2] FIG. 2 is an explanatory diagram illustrating a moving object managed by a management system. [Figure 3] FIG. 10 is a flowchart illustrating the operation flow of the management system according to the embodiment. [Figure 4]FIG. 10 is a flowchart illustrating the flow of operations of a management system according to a modified example of the embodiment. [Figure 5] FIG. 2 is a block diagram showing an example of a hardware configuration of an information processing device that embodies the management device according to the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0025] Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In this specification and drawings, components having substantially the same functional configurations are designated by the same reference numerals, and redundant explanations will be omitted.

[0026] <1. Configuration example> First, a configuration example of a management system 1 according to an embodiment of the present invention will be described with reference to Fig. 1 and Fig. 2. Fig. 1 is a block diagram showing the functional configuration of the management system 1 according to this embodiment. Fig. 2 is an explanatory diagram showing a moving object 100 managed by the management system 1.

[0027] 1, the management system 1 according to this embodiment includes a mobile object 100, an operation terminal 210, and a management device 200. There may be multiple mobile objects 100 and multiple operation terminals 210.

[0028] 2, the mobile object 100 is an autonomous mobile robot that performs tasks such as monitoring, inspection, security, or transportation by patrolling a predetermined area M indicated by map data acquired in advance along a predetermined route R. The mobile object 100 can also move under control (so-called remote control) from an operation terminal 210.

[0029] The operation terminal 210 is a terminal operated by an operator who manages the moving body 100. The operator can monitor or remotely control the moving body 100 by operating the operation terminal 210.

[0030] The management device 200 is a device that manages the entire management system 1 including the mobile object 100. The management device 200 may, for example, manage information on the mobile object 100, the operation terminal 210, and the operator who operates the operation terminal 210, manage the operation status of the mobile object 100, manage the connection between the mobile object 100 and the operation terminal 210, and manage operation information of the mobile object 100 by the operator.

[0031] (Mobile unit 100) A more detailed description will be given of each component of the mobile object 100. The mobile object 100 includes a camera unit 101, a sensor unit 102, a display unit 103, a speaker 104, a microphone 105, a mobile object communication unit 106, a battery unit 107, a movement unit 108, and a mobile object control unit 110.

[0032] The camera unit 101 is at least one imaging device capable of capturing images of the environment surrounding the moving object 100. For example, the camera unit 101 may be an imaging device that captures images of the environment in front of the moving object 100, or may be a plurality of imaging devices that capture images of the environment in each direction of the moving object 100, including the front, back, left, and right. The camera unit 101 may be, for example, an RGB camera, a stereo camera, a 360-degree camera, a thermography camera, or an infrared camera. The camera unit 101 may also be, for example, a plurality of fisheye cameras provided in each direction of the moving object 100. In such a case, the camera unit 101 can generate an image of the environment surrounding the moving object 100 as viewed from above by seamlessly combining images captured by the fisheye cameras in all directions.

[0033] The sensor unit 102 is at least one type of sensor capable of sensing parameters of the environment surrounding the mobile object 100. The sensor unit 102 may be, for example, one or more optical sensors (such as a ToF sensor, LiDAR, or millimeter-wave radar) capable of simultaneously estimating the self-position of the mobile object 100 and creating a map. Note that the term "sensing" in the above context includes not only detecting various parameters of the environment surrounding the mobile object 100, but also further measuring, determining, and analyzing the detected parameters.

[0034] The sensor unit 102 may further include a sensor other than a sensor that uses light. The sensor unit 102 may further include a sensor that can sense, for example, temperature, humidity, illuminance, air pressure, vibration, the distance from the mobile object 100 to surrounding obstacles, or a change in the inclination of the mobile object 100, and may further include a sensor that can sense the generation of smoke, chemical substances, static electricity, or the like.

[0035] The display unit 103 is a display device that displays various information under the control of the mobile object control unit 110. The display unit 103 may be, for example, a liquid crystal display (LCD) device capable of displaying images or videos, an organic light emitting diode (OLED) display device, or an LED lamp. The display unit 103 may also be a touch panel display device that has an image display function and an input function via touch operation.

[0036] The speaker 104 is a sound generating device that outputs sound around the moving object 100 under the control of the moving object control unit 110. The speaker 104 may, for example, output sound effects or synthesized voices corresponding to the movement of the moving object 100 around the moving object 100, or may output voices input by an operator operating the operation terminal 210 around the moving object 100.

[0037] The microphone 105 is a sound collection device that collects sounds around the moving body 100. The microphone 105 may collect, for example, the sounds of people around the moving body 100 or environmental sounds around the moving body 100.

[0038] The mobile communication unit 106 is a communication interface for connecting to the network 10. The mobile communication unit 106 may be, for example, a communication interface that can be connected wirelessly or wired to the network 10 or a base station that can be connected to the network 10, or a communication interface that can be connected to the network 10 via a network such as a mobile phone communication network or a wireless LAN.

[0039] The battery unit 107 is a power source for the mobile object 100, and supplies power to each component of the mobile object 100. The battery unit 107 may be, for example, a battery pack including a secondary battery such as a lithium-ion secondary battery.

[0040] The moving unit 108 is a moving mechanism that moves the moving body 100 under the control of the moving body control unit 110. The moving unit 108 may be a moving mechanism of various types, such as a wheel type, a leg type, a caterpillar type, or an air cushion type.

[0041] The mobile object control unit 110 is a control device that controls the overall operation of the mobile object 100. Specifically, the mobile object control unit 110 may determine the situation based on images and sensing data acquired by the camera unit 101 and the sensor unit 102, and control the operation of the mobile object 100 based on the determined situation. For example, the mobile object control unit 110 may control the movement of the mobile object 100 based on map data generated in advance and its own position estimated from the images and sensing data acquired by the camera unit 101 and the sensor unit 102.

[0042] The mobile object control unit 110 may also include an AI unit 111. The AI ​​unit 111 may perform various types of recognition, judgment, and control using a learning model constructed by machine learning. For example, the AI ​​unit 111 may use the learning model to recognize the environment around the mobile object 100 from images and sensing data acquired by the camera unit 101 and the sensor unit 102. The AI ​​unit 111 may also use the learning model to determine the situation of the mobile object 100 and control the operation of the mobile object 100 based on the determined situation.

[0043] (Operation terminal 210) A more detailed description will be given of each component included in the operation terminal 210. The operation terminal 210 includes a communication unit 211, an input unit 212, and a display unit 213, for example.

[0044] The communication unit 211 is a communication interface for connecting to the service management unit 220 provided in the management device 200 and the mobile object 100. The communication unit 211 can receive notifications from the service management unit 220 of the management device 200 and transmit and receive data to and from the mobile object 100.

[0045] The input unit 212 is an input device that accepts operation input from an operator, and may be, for example, an input device such as a keyboard, a mouse, a touch panel, a track pad, a cross key, an operation lever, or a microphone.

[0046] The display unit 213 is a display device that displays notifications sent from the management device 200, information about the mobile object 100, or images captured by the camera unit 101 of the mobile object 100. The display unit 213 may be a display device such as a liquid crystal display (LCD) device, an organic light emitting diode (OLED) display device, or a touch panel display device.

[0047] (Management device 200) The following provides a more detailed description of each component of the management device 200. The management device 200 includes a communication unit 240, a service management unit 220, and a management database 230.

[0048] The communication unit 240 is a communication interface for communication between the mobile object 100 and the management device 200. The communication unit 240 may receive images or sensing data from the mobile object 100, or may relay remote control from the operation terminal 210 to the mobile object 100. The communication unit 240 may be connected to the mobile object 100 using MQTT (Message Queue Telemetry Transport) communication, WebRTC, or the like. The communication unit 240 is one embodiment of the image acquisition unit of the present invention.

[0049] The management database 230 is a database that manages data related to the operation of the mobile body 100. For example, the management database 230 may store various tasks performed by the mobile body 100, an operation plan for the mobile body 100, and sensing data acquired by the mobile body 100.

[0050] The management database 230 further includes an image storage unit 231. The image storage unit 231 is provided so that the moving object 100 that has lost its own position can automatically recover its own position.

[0051] The image storage unit 231 stores a plurality of environmental images captured within a predetermined area M. The predetermined area M is an area including a predetermined route R along which the mobile object 100 autonomously moves, and the plurality of environmental images are environmental images captured in advance by the camera unit 101 of the mobile object 100 located at various coordinates within the predetermined area M. The environmental images captured by the camera unit 101 are stored in the image storage unit 231 in association with the position coordinates at which they were captured. This allows the management device 200 to compare the environmental images stored in the image storage unit 231 with other images, thereby estimating the position coordinates at which the other images were captured from the position coordinates associated with the environmental images. Note that the image storage unit 231 may also store environmental images captured by the camera unit 101 of the mobile object 100 located at various coordinates outside the predetermined area M in association with the position coordinates at which they were captured.

[0052] The service management unit 220 remotely manages various services performed by the mobile object 100 and sensing data acquired by the mobile object 100. For example, the service management unit 220 may instruct an operation plan for the mobile object 100 based on images or sensing data acquired by the camera unit 101 or sensor unit 102 of the mobile object 100 and an operation by an operator input via the operation terminal 210. The service management unit 220 may instruct an operation plan for the mobile object 100 based on information stored in the management database 230.

[0053] The service management unit 220 further includes a determination unit 221 and a remote control unit 222. The determination unit 221 and the remote control unit 222 are provided so that the moving object 100 that has lost its own position can automatically recover its own position.

[0054] When the moving object 100 loses sight of its own position, the determination unit 221 compares the image captured by the camera unit 101 of the moving object 100 with each of the multiple environmental images stored in the image storage unit 231, and determines whether the images match. For example, the determination unit 221 may determine whether the image captured by the camera unit 101 of the moving object 100 matches each of the multiple environmental images stored in the image storage unit 231 by using a known algorithm that determines the degree of match between images.

[0055] The captured image used by the determination unit 221 to determine whether the captured image matches each of the plurality of environmental images is, for example, an image captured by the camera unit 101 at the current position of the moving object 100 that has lost sight of its own position. However, the captured image used by the determination unit 221 to determine whether the captured image matches each of the plurality of environmental images does not have to be an image captured at the current position of the moving object 100, as long as it is an image captured by the camera unit 101 after the moving object 100 has lost sight of its own position. Furthermore, the captured image used by the determination unit 221 to determine whether the captured image matches each of the plurality of environmental images may be an image captured by the camera unit 101 immediately before the moving object 100 loses sight of its own position.

[0056] Based on the determination result by the determination unit 221, the remote control unit 222 controls the operation of the moving object 100 that has lost track of its own position.

[0057] For example, the remote control unit 222 may estimate the position coordinates associated with an environmental image determined to match an image captured by the camera unit 101 of the mobile object 100 as the position coordinates of the mobile object 100 that captured the image. In this way, the remote control unit 222 can cause the mobile object 100 to restore its own position based on the estimated position coordinates of the mobile object 100.

[0058] Furthermore, the remote control unit 222 may move the moving object 100 when it is determined that the image captured by the camera unit 101 of the moving object 100 does not match each of the multiple environmental images. In this way, the determination unit 221 can compare the image captured by the camera unit 101 of the moving object 100 after the movement with each of the multiple environmental images again to determine whether the images match. Therefore, when it is determined that the captured image does not match each of the multiple environmental images, the remote control unit 222 can move the moving object 100 little by little, thereby causing the moving object 100 to search for a position where the self-position of the moving object 100 can be estimated.

[0059] Furthermore, when it is determined that the image captured by the camera unit 101 of the moving object 100 does not match each of the plurality of environmental images, the remote control unit 222 may connect the moving object 100 to the operation terminal 210 so that the moving object 100 can be remotely controlled from the operation terminal 210. In this way, the remote control unit 222 can cause the operator to remotely control the moving object 100 via the operation terminal 210, thereby causing the moving object 100 to recover its own position.

[0060] The management system 1 having the above configuration is applied to a mobile object 100 that autonomously moves along a predetermined route R in a predetermined area M indicated by map data acquired in advance. The management system 1 enables the mobile object 100 to automatically recover its own position when it loses its own position during autonomous movement.

[0061] Specifically, when the mobile object 100 loses its own position, the management system 1 compares an image captured by the mobile object 100 with each of a plurality of environmental images captured within a predetermined area M indicated by the map data. If it is determined that the image captured by the mobile object 100 matches the environmental image, the management system 1 estimates the position coordinates at which the environmental image determined to match the captured image was captured as the position coordinates of the mobile object 100. This allows the management system 1 to restore the mobile object 100 to its own position based on the estimated position coordinates of the mobile object 100. On the other hand, if it is determined that the image captured by the mobile object 100 does not match the environmental image, the management system 1 allows the mobile object 100 to restore its own position by autonomously moving the mobile object 100 or by remotely moving the mobile object 100 by an operator.

[0062] Therefore, when the moving body 100 loses track of its own position, the management system 1 can automatically restore its own position, thereby reducing the effort and man-hours required of the operator.

[0063] <2. Example of operation> Next, the flow of operations of the management system 1 according to this embodiment will be described with reference to Fig. 3. Fig. 3 is a flowchart showing the flow of operations of the management system 1 according to this embodiment.

[0064] 3, first, the moving body 100 detects that its own position has been lost (S101). For example, the moving body 100 may detect that its own position has been lost when it becomes unable to determine the route it is traveling, when map data held by the moving body 100 and sensing data around the moving body 100 become inconsistent, or when the continuity of the moving position of the moving body 100 is interrupted.

[0065] When the loss of the mobile object 100's own location is detected, the mobile object 100 notifies the management device 200 of the loss of the mobile object 100's own location (S102). As a result, the management device 200 notifies the operation terminal 210 of the fact that the mobile object 100 has lost its own location as an alert, and causes the operation terminal 210 to display the alert (S103).

[0066] Next, the management device 200 acquires an image captured by the camera unit 101 of the moving object 100 that has lost its own position (S104).

[0067] Next, the management device 200 compares the acquired captured image with each of a plurality of environmental images captured in advance within a predetermined area in which the mobile object 100 autonomously moves, which are stored in the image storage unit 231 (S105). Specifically, the management device 200 may compare the acquired captured image with each of a plurality of environmental images captured in advance within a predetermined area, based on a known algorithm for determining the degree of match between images, to determine the degree of match between the two.

[0068] According to this, the management device 200 can estimate the position coordinates corresponding to the environmental image that most closely matches the acquired captured image (i.e., the position coordinates at which the environmental image was captured) as the position coordinates of the mobile object 100 (S106). Thereafter, the management device 200 determines, based on the estimated position coordinates, whether or not the mobile object 100 is within a predetermined area (map) indicated in the map data held by the mobile object 100 (S107).

[0069] If it is determined that the moving object 100 is not present within the predetermined area (map) (S107 / NO), the management device 200 causes the moving object 100 to move autonomously (S110). Thereafter, the management device 200 again acquires an image captured by the moving object 100 (S104), and compares the acquired image with each of the multiple environmental images (S105).

[0070] The direction of the autonomous movement of the moving body 100 in step S110 may be, for example, a direction that avoids an obstacle detected based on sensing data from the sensor unit 102. This allows the moving body 100 to move in an open direction with fewer obstacles without colliding with any obstacles. Alternatively, the direction of the autonomous movement of the moving body 100 in step S110 may be a direction in which the degree of match between the captured image and the environmental image that most closely matches the captured image is higher (i.e., the difference between the captured image and the environmental image is reduced). This allows the moving body 100 to move in a direction in which the captured image is more likely to match each of multiple environmental images captured in advance within a predetermined area. Alternatively, the direction of the autonomous movement of the moving body 100 in step S110 may be a random direction.

[0071] On the other hand, if it is determined that the moving body 100 is within a predetermined area (map) indicated by the map data (S107 / YES), the management device 200 displays a map showing the estimated location coordinates of the moving body 100 on the display unit 213 of the operation terminal 210 (S108).

[0072] Next, the management device 200 checks whether the mobile object 100 has recovered its own position (S109). When the mobile object 100 returns to a predetermined area (map) indicated by the map data held by the mobile object 100, the mobile object 100 can recover its own position by comparing the sensing data sensed by the sensor unit 102 with the map data. Therefore, when the mobile object 100 has returned to the predetermined area (map) indicated by the map data as determined in step S107, the mobile object 100 can recover its own position.

[0073] When the moving object 100 has recovered its own position (S109 / YES), the management device 200 stops the alert that has been notified to the operation terminal 210 (S111). After that, the moving object 100 that has recovered its own position can resume its work such as monitoring, inspection, security, or transportation (S112).

[0074] If the moving object 100 has not recovered its own position (S109 / NO), the management device 200 autonomously moves the moving object 100 (S110), as in the case where step S107 is determined to be NO. Thereafter, the management device 200 again acquires a captured image captured by the moving object 100 (S104) and compares the acquired captured image with each of the multiple environmental images (S105). Note that even after the moving object 100 has moved a predetermined number of times, if the determination in step S107 or step S109 is NO, the management device 200 may have an operator remotely operate the moving object 100 via the operation terminal 210. In this way, the management device 200 can reliably cause the moving object 100 to recover its own position through remote operation by the operator.

[0075] According to the above operation, the management system 1 can estimate the position coordinates of the moving object 100 that has lost its own position by comparing images with each other, and can move the moving object 100 to a predetermined area (map) where the moving object 100 can recover its own position. Therefore, when the moving object 100 loses its own position, the management system 1 can automatically recover its own position, thereby reducing the effort and man-hours of the operator.

[0076] <3. Modifications> Next, the flow of operation of the management system 1 according to a modified example of this embodiment will be described with reference to Fig. 4. Fig. 4 is a flowchart showing the flow of operation of the management system 1 according to a modified example of this embodiment. The management system 1 according to this modified example is a modified example in which, when the mobile object 100 loses its own position, the operator remotely controls the mobile object 100 via the operation terminal 210 to move the mobile object 100, thereby causing the mobile object 100 to recover its own position.

[0077] 4, first, the moving body 100 detects that its own position has been lost (S101). The moving body 100 may detect that its own position has been lost, for example, when the moving body 100 loses track of the route it is traveling, when map data held by the moving body 100 and sensing data around the moving body 100 become inconsistent, or when the continuity of the moving position of the moving body 100 is broken.

[0078] When the loss of its own location is detected, the mobile object 100 notifies the management device 200 of the loss of its own location (S102). As a result, the management device 200 notifies the operation terminal 210 of the fact that the mobile object 100 has lost its own location as an alert, and causes the operation terminal 210 to display the alert (S103). At this time, the management device 200 connects the operation terminal 210 that notified the alert to the mobile object 100, thereby enabling the operation terminal 210 to remotely control the mobile object 100.

[0079] Thereafter, the operator of the operation terminal 210 who has been notified of the alert operates the operation terminal 210 to remotely control the moving object 100, thereby moving the moving object 100 (S120).

[0080] Thereafter, the management device 200 determines whether the mobile object 100 is within a predetermined area (map) indicated by the map data held in the mobile object 100 (S107). For example, the management device 200 may compare the captured image captured by the camera unit 101 of the mobile object 100 with each of a plurality of environmental images captured in advance within the predetermined area, and determine whether the mobile object 100 is within the predetermined area (map) based on whether the two images match.

[0081] If it is determined that the moving object 100 is not within a predetermined area (map) indicated by the map data (S107 / NO), the management device 200 causes the operator of the operation terminal 210 to further remotely operate the moving object 100 (S120).

[0082] On the other hand, if it is determined that the moving body 100 is within a predetermined area (map) indicated by the map data (S107 / YES), the management device 200 displays a map showing the estimated location coordinates of the moving body 100 on the display unit 213 of the operation terminal 210 (S108).

[0083] Next, the management device 200 checks whether the mobile object 100 has recovered its own position (S109). When the mobile object 100 returns to a predetermined area (map) indicated by the map data held by the mobile object 100, the mobile object 100 can recover its own position by comparing the sensing data sensed by the sensor unit 102 with the map data. Therefore, when the mobile object 100 has returned to the predetermined area (map) indicated by the map data as determined in step S107, the mobile object 100 can recover its own position.

[0084] When the moving object 100 has recovered its own position (S109 / YES), the management device 200 stops the alert that has been notified to the operation terminal 210 (S111). After that, the moving object 100 that has recovered its own position can resume its work such as monitoring, inspection, security, or transportation (S112).

[0085] If the mobile object 100 has not recovered its own position (S109 / NO), the management device 200 causes the operator of the operation terminal 210 to further remotely operate the mobile object 100 (S120), as if step S107 was judged to be NO.

[0086] According to the above operation, the management system 1 allows an operator to remotely operate the moving object 100 that has lost its position, thereby allowing the moving object 100 to move into a predetermined area (map) where the moving object 100 can recover its position. Therefore, when the moving object 100 has lost its position, the management system 1 can recover the position of the moving object 100 without going to the site, thereby reducing the operator's workload.

[0087] <4. Hardware configuration example> The above has described an embodiment of the present invention. The various information processes performed by the management device 200 described above are realized by cooperation between software and the hardware of the management device 200 described below.

[0088] FIG. 5 is a block diagram showing an example of the hardware configuration of an information processing device 900 that embodies the management device 200 according to this embodiment.

[0089] As shown in FIG. 5, the information processing device 900 includes a CPU (Central Processing Unit) 901, a ROM (Read Only Memory) 902, a RAM (Random Access Memory) 903, a host bus 904, a bridge 905, an external bus 906, an interface 907, an input device 908, an output device 909, a storage device 910, a drive 911, and a communication device 913.

[0090] The CPU 901 functions as an arithmetic processing unit and control unit, and controls the overall operation of the information processing device 900 in accordance with various programs. The CPU 901 may be a microprocessor. The ROM 902 stores programs and calculation parameters used by the CPU 901. The RAM 903 temporarily stores programs used in the execution of the CPU 901, as well as parameters that change as appropriate during program execution. The CPU 901, ROM 902, and RAM 903 are interconnected by a host bus 904 that is composed of a CPU bus and the like. The functions of the above-mentioned business management unit 220 are realized by the cooperation of the CPU 901, ROM 902, and RAM 903.

[0091] The host bus 904 is connected to an external bus 906 such as a PCI (Peripheral Component Interconnect / Interface) bus via a bridge 905. The host bus 904, bridge 905, and external bus 906 do not necessarily have to be separate, and these functions may be implemented on a single bus.

[0092] The input device 908 is composed of an input means such as a mouse, keyboard, touch panel, button, switch, or microphone that allows the user to input information, and an input control circuit that generates an input signal based on the user's input and outputs it to the CPU 901. By operating the input device 908, the user operating the information processing device 900 can input various data to the information processing device 900 and instruct processing operations.

[0093] The output device 909 includes, for example, a display device such as a CRT (Cathode Ray Tube) display device, a Liquid Crystal Display (LCD) device, an OLED (Organic Light Emitting Diode) device, or a lamp, or an audio output device such as a speaker.

[0094] The storage device 910 is a device for storing data. The storage device 910 may include a storage medium, a recording device for recording data on the storage medium, a reading device for reading data from the storage medium, and a deleting device for deleting data recorded on the storage medium. The storage device 910 may implement the functions of the management database 230, for example.

[0095] The drive 911 is a reader / writer for a storage medium, and is externally attached to the information processing device 900. The drive 911 reads information recorded on a removable storage medium 912, such as an attached magnetic disk, optical disk, magneto-optical disk, or semiconductor memory, and outputs the information to the RAM 903. The drive 911 can also write information to the removable storage medium 912.

[0096] The communication device 913 is a communication interface configured with a communication device for performing communication, etc. The communication device 913 may be a wireless LAN (Local Area Network) compatible communication device, or may be a wired communication device that performs wired communication.

[0097] The hardware configuration of the information processing device 900 is not limited to the configuration shown in Fig. 5. For example, the information processing device 900 may not include the input device 908 or the output device 909. Furthermore, part or all of the configuration shown in Fig. 5 may be realized by one or more integrated circuits (ICs).

[0098] For example, the processes described herein using flowcharts do not necessarily have to be performed in the order shown, some process steps may be performed in parallel, additional process steps may be employed, and some process steps may be omitted.

[0099] Although the preferred embodiments of the present invention have been described in detail above with reference to the accompanying drawings, the present invention is not limited to these examples. It is clear that a person skilled in the art to which the present invention pertains can conceive of various modifications and alterations within the scope of the technical ideas set forth in the claims, and it is understood that these also naturally fall within the technical scope of the present invention. [Explanation of symbols]

[0100] 1 Management System 10 Network 100 Mobile 200 Management device 210 Operation terminal 211 Communications Department 212 Input section 213 Display section 220 Business Management Department 221 Judgment section 222 Remote control unit 230 Management Database 231 Image storage unit 240 Communications Department

Claims

1. an image storage unit that stores a plurality of environmental images captured within a predetermined area; an image acquisition unit that acquires a captured image captured by a moving object that autonomously moves within the predetermined area; a determination unit that determines whether the captured image acquired by the image acquisition unit matches each of the plurality of environmental images when the moving object loses track of its own position; a remote control unit that controls an operation of the moving object that has lost its own position based on the determination result by the determination unit; A management device comprising:

2. The management device described in claim 1, wherein when the determination unit determines that the captured image matches one of the multiple environmental images, the remote control unit determines that the moving body is present at a coordinate position corresponding to the environmental image that matches the captured image, and causes the moving body to restore its own position.

3. The management device described in claim 2, wherein when the determination unit determines that the captured image matches one of the multiple environmental images, the remote control unit causes an operation terminal monitoring the mobile object to display a map of the specified area with the coordinate position indicated.

4. A management device described in any one of claims 1 to 3, wherein if the determination unit determines that the captured image does not match each of the multiple environmental images, the remote control unit moves the moving object.

5. The management device according to claim 4 , wherein the remote control unit moves the moving object in a direction that reduces a difference between the captured image and an environmental image that most closely matches the captured image.

6. The management device according to claim 4 , wherein the remote control unit moves the mobile object in a direction that avoids obstacles present around the mobile object.

7. The management device according to claim 4 , wherein when the moving object is moved by the remote control unit, the determination unit determines whether an image captured by the moving object after the movement matches each of the plurality of environmental images.

8. The management device described in claim 7, wherein if the determination unit determines that the captured image does not match each of the multiple environmental images after the moving body has been moved a predetermined number of times, the remote control unit sends a notification to an operation terminal monitoring the moving body requesting remote operation of the moving body.

9. The management device according to claim 1 , wherein, when the mobile object loses track of its own position, an alert indicating that the mobile object has lost track of its own position is transmitted to an operation terminal that monitors the mobile object.

10. a mobile object that can autonomously move within a predetermined area and acquire captured images of the surrounding environment; a management device including an image storage unit that stores a plurality of environmental images captured within the predetermined area; a determination unit that, when the moving body loses its own position, determines whether the captured image captured by the moving body matches each of the plurality of environmental images; and a remote control unit that controls the operation of the moving body that has lost its own position based on the determination result by the determination unit; a management system, including

11. storing a plurality of environmental images captured within a predetermined area; When a moving body that autonomously moves within the predetermined area loses track of its own position, determining whether an image captured by the moving body matches each of the plurality of environmental images; Controlling the operation of the moving body that has lost track of its own position based on the result of the determination; Management methods, including:

Images