Driving operation information processing system and autonomous driving operation device

Abstract

To effectively use an autonomously traveling work device taking advantage of the remote control function of the autonomously traveling work device.SOLUTION: A traveling work information processing system 1 comprises an autonomously traveling work device 2 which is capable of executing automatically traveling cleaning, an operation terminal 4 with which a user operates the autonomously traveling work device 2, and a management server 3 that manages the autonomously traveling work device 2. The autonomously traveling work device 2 includes an image capturing unit, and transmits captured data having been captured by the image capturing unit to the management server 3. The management server 3 acquires, as captured data, first captured data 60 in which the traveling state of the autonomously traveling work device 2 is captured, second captured data 61 in which the work state of the autonomously traveling work device 2 is captured, and third other captured data, causing the operation terminal 4 to display, in list form, at least one or more of the first captured data 60, the second captured data 61, and the third captured data.SELECTED DRAWING: Figure 22

Description

Technical Field

[0001] The present invention relates to a traveling work information processing system including an autonomous traveling work device capable of executing an autonomous traveling work that autonomously travels according to a pre-input program and automatically performs work, and the autonomous traveling work device.

Background Art

[0002] Conventionally, an autonomous traveling work device creates or sets and stores an environmental map, a traveling route, traveling settings, and work settings (cleaning settings) according to a pre-input program, and autonomously travels and automatically performs work (cleaning) based on the stored results. It has a function capable of executing an autonomous traveling work (autonomous traveling cleaning). For example, the autonomous traveling work device is applied to an industrial (business-use) autonomous traveling work device (autonomous traveling cleaning device) that performs a cleaning work on the floor surface of a work area such as a commercial facility, an office, a hotel, a hospital, etc.

[0003] In a commercial facility such as an office having a plurality of work areas or a vast work area, in order to perform an autonomous traveling work in each work area by operating a plurality of cleaning robots configured by autonomous traveling work devices, a traveling work information processing system that manages a plurality of autonomous traveling work devices via a network may be applied. In such a traveling work information processing system, in the autonomous traveling work of each autonomous traveling work device, when an unknown obstacle that does not exist in the environmental map is detected or an emergency state occurs, the autonomous traveling work becomes impossible to continue. Therefore, as a means for avoiding an emergency state, for example, there is one that uses a remote operation function.

[0004] By the way, according to Patent Document 1, a system that provides a notification of a wet floor includes a scrubbing robot that scrubs the floor and transports liquid, and a sensor arranged to collect data around the area of the floor. The sensor is, for example, an imaging device that captures a first image of the floor. Further, this system includes an application that determines that liquid is on the floor by analyzing the data.

Prior Art Documents

[0005] [Patent Document 1] Special Publication No. 2020-534899 [Overview of the project] [Problems that the invention aims to solve]

[0006] While it is possible to detect residual water on the floor surface using the technology described in Patent Document 1, this type of system cannot clearly inform the operator remotely controlling the autonomous mobile work device of the residual water situation. Furthermore, even if residual water on the floor surface is detected, this type of system cannot eliminate the cause of the residual water generation, nor can it stop subsequent residual water generation. In this type of system, only a single operator can use the remote control function of the autonomous mobile work device, and multiple users cannot share this function. Moreover, in information processing systems using this type of device, it is desirable to apply the remote control function of the autonomous mobile work device to purposes other than avoiding emergency situations. For example, it is desirable to share the remote control function not only with workers such as cleaning companies, but also with other users such as security guards and managers. Thus, in recent information processing systems using autonomous mobile work devices, it is expected that the autonomous mobile work device will be utilized more effectively by making use of its remote control function.

[0007] The present invention has been made in view of the above-mentioned problems, and the object of the present invention is to provide a driving work information processing system and an autonomous driving work device that can effectively utilize an autonomous driving work device by utilizing the remote control function of the autonomous driving work device. [Means for solving the problem]

[0008] To solve the above problems, the first driving work information processing system of the present invention comprises an autonomous driving work device capable of autonomously driving and performing automated driving work, an operation terminal for a user to operate the autonomous driving work device, and a management server for managing the autonomous driving work device, wherein the autonomous driving work device is equipped with a camera unit, transmits captured data to the management server, and the management server acquires, as captured data, first captured data capturing the driving status of the autonomous driving work device, second captured data capturing the work status of the autonomous driving work device, and other third captured data, and, according to a predetermined determination, displays at least one of the first, second, and third captured data on the operation terminal. When the autonomous driving work device detects an error related to the automated driving work, it sends an error notification to the management server. If the management server has not received the error notification from the autonomous driving work device, it displays one of the first, second, and third captured data on the operation terminal. If the management server has received the error notification from the autonomous driving work device, it displays at least two of the first, second, and third captured data on the operation terminal in parallel, depending on whether the error notification is related to the driving situation or the work situation. It is characterized by the following:

[0009] According to the first driving work information processing system of the present invention, users of the autonomous driving work device can view captured data of various situations of the autonomous driving work device on an operating terminal. Furthermore, since the captured data is routed through a management server for viewing and display, the load on the autonomous driving work device and the operating terminal can be reduced. Furthermore, according to the first driving operation information processing system of the present invention, when an error related to automated driving operation is detected, the user can simultaneously check the first, second, and third image data displayed in parallel. This allows the user to understand the error situation in more detail.

[0010] To solve the above problems, the second driving work information processing system of the present invention is characterized in that the third photographic data is an image captured by a surveillance camera that monitors the work area in which the autonomous driving work device is traveling, or an image captured by a camera mounted on the side of the cleaning unit provided on the autonomous driving work device.

[0011] According to the second driving work information processing system of the present invention, not only the driving status and work status of the autonomous driving work device can be obtained as third-party image data, including images from a surveillance camera that provide an overhead view of the area around the autonomous driving work device and images that allow for detailed viewing of the cleaning section of the autonomous driving work device. Users can refer to such third-party image data as needed.

[0014] Furthermore, the present invention 3 In the driving operation information processing system, if the management server has not received the error notification from the autonomous driving operation device, of the first image data, the second image data, and the third image data, 2 The operation terminal is configured to display multiple images in a larger size and in parallel, and when an error notification is received from the autonomous driving work device, it is configured to display different images from those displayed in a larger size before the error notification on the operation terminal in a larger size and in parallel.

[0015] This invention 3 According to the vehicle operation information processing system, when an error related to automated driving operations is detected, users can gain a more detailed understanding of the error by viewing the large-displayed photographic data.

[0016] To solve the above problems, the present invention 4 In the driving operation information processing system, the management server stores viewing settings for viewing captured data, and, in accordance with the viewing settings, causes at least one of the first, second, and third captured data to be displayed on the operation terminal for viewing.

[0017] This invention 4 According to the driving operation information processing system, the management server can display appropriate photographic data on the operation terminal according to each user's role, even if the roles of users using the autonomous driving operation device differ for each user, such as operators, administrators, and security guards. For example, by displaying first photographic data capturing the driving situation to the operator in charge of driving, and second photographic data capturing the cleaning situation to the operator in charge of cleaning, it is possible to accurately determine whether or not there is a problem with the automated cleaning operation.

[0018] To solve the above problems, the present invention 5The traveling work information processing system includes a plurality of operation terminals as the operation terminals. When each of the plurality of operation terminals logs in to the management server, it transmits a selection request to the management server for selecting the autonomous traveling work device to be remotely operated. The management server receives the selection requests for one autonomous traveling work device from two or more logged-in operation terminals, and causes the two or more operation terminals to view and display the captured data transmitted from the one autonomous traveling work device.

[0019] According to the traveling work information processing system of the 5 present invention, captured data captured by one autonomous traveling work device can be confirmed by a plurality of users on their respective operation terminals. Therefore, since a plurality of users can share the captured data, the remote operation function of the autonomous traveling work device can be utilized to effectively use the autonomous traveling work device.

[0020] In order to solve the above problems, in the traveling work information processing system of the 6 present invention, when the management server receives the error notification from the autonomous traveling work device, The aforementioned it causes the captured data to be displayed Ta and accepts remote operation of the autonomous traveling work device by the operation terminal.

[0021] According to the traveling work information processing system of the 6 present invention, a user can remotely operate the autonomous traveling work device while confirming the first captured data, the second captured data, and the third captured data.

[0022] In order to solve the above problems, in the traveling work information processing system of the 7 present invention, when the autonomous traveling work device detects an error related to the traveling status of the autonomous traveling work device, it transmits the error notification to the management server, and when the management server Regarding the aforementioned driving conditions receives the error notification, it causes the operation terminal to view and display the first captured data obtained by capturing the traveling status, which is characterized in that.

[0023] According to the traveling work information processing system of the 7 present invention, the user can perform avoidance driving of an error related to the traveling situation by remote operation while checking the first captured data obtained by capturing the traveling situation, and the avoidance of the error becomes accurate and easy.

[0024] In order to solve the above problems, in the traveling work information processing system of the 8 present invention, the autonomous traveling work device is characterized by including a cleaning unit that performs a cleaning operation on the floor surface.

[0025] Also, in the traveling work information processing system of the 9 [[ID=1*]]present invention, the imaging unit As for the aforementioned work status captures an image including the cleaning work situation behind the cleaning unit to obtain the captured data, and the management server determines the cleaning work situation of the cleaning unit based on the captured data obtained by capturing the cleaning work situation, and detects an error related to the cleaning work of the cleaning unit on the floor surface.

[0026] According to the traveling work information processing system of the 8 to the <* 9 present invention, it is possible to perform a cleaning operation on the floor surface as an automatic traveling work, and by using the captured data obtained by appropriately capturing the cleaning work situation of the cleaning unit, it is possible to detect an abnormal operation of the cleaning unit, so that an error related to the cleaning work can be accurately determined.

[0027] [[ID=SO]] In order to solve the above problems, in the traveling work information processing system of the 10 present invention, the cleaning unit is configured to clean the floor surface using a cleaning liquid, and the imaging unit As for the aforementioned work status captures the cleaning work situation of the cleaning unit from above to obtain the second captured data, and the management server determines the cleaning work situation of the cleaning unit based on the second captured data obtained by capturing the cleaning work situation, and detects residual water on the floor surface.

[0028] Alternatively, in the11 In the vehicle operation information processing system, the cleaning unit is configured to clean the floor surface using a cleaning solution, the imaging unit captures details of the cleaning operation status of the cleaning unit to acquire third imaging data, and the management server determines the cleaning operation status of the cleaning unit based on the third imaging data capturing the cleaning operation status.

[0029] Alternatively, the present invention 12 In the driving work information processing system, the cleaning unit is configured to clean the floor surface using a cleaning solution, the imaging unit captures images of the cleaning work status of the cleaning unit and acquires the image data while the autonomous driving work device is turning, and the management server determines the cleaning work status of the cleaning unit based on the image data of the cleaning work status and detects residual water on the floor surface.

[0030] This invention 10 ~ 12 According to the vehicle operation information processing system, by using photographic data taken at appropriate times to capture the cleaning status of the cleaning unit, abnormal operation of the cleaning unit can be detected, allowing for accurate identification of residual cleaning solution or wastewater on the floor surface and other errors related to the cleaning operation.

[0031] In the 13th driving operation information processing system of the present invention , The imaging unit is the The first imaging unit acquires the first imaging data, The aforementioned The system is characterized by comprising a second imaging unit for acquiring second imaging data.

[0032] This invention The 13th Driving Operation Information Processing System According to the report, users of the autonomous driving work device will be able to use the first set of captured data, which shows the driving situation, and the second set of captured data, which shows the work situation, as needed. [Effects of the Invention]

[0033] According to the present invention, the driving work information processing system and the autonomous driving work device can effectively utilize the autonomous driving work device by taking advantage of the remote control function of the autonomous driving work device. [Brief explanation of the drawing]

[0034] [Figure 1] This is a block diagram showing the configuration of a driving operation information processing system according to the first embodiment of the present invention. [Figure 2] This is a schematic diagram showing the configuration of an autonomous driving device in a driving work information processing system according to the first embodiment of the present invention. [Figure 3] This is a plan view showing an example of a cleaning area in a driving work information processing system according to the first embodiment of the present invention, in which an autonomous driving work device performs automatic cleaning work. [Figure 4] This is a block diagram showing the configuration of an autonomous driving device in a driving work information processing system according to the first embodiment of the present invention. [Figure 5] This is a block diagram showing the configuration of the management server in the driving operation information processing system according to the first embodiment of the present invention. [Figure 6] This is a block diagram showing the configuration of an operating terminal in a driving operation information processing system according to the first embodiment of the present invention. [Figure 7] This is a plan view showing an example of first captured data of the driving status of an autonomous driving work device in a driving work information processing system according to the first embodiment of the present invention. [Figure 8] This is a plan view showing an example of second captured data of the cleaning work status of an autonomous driving work device in a driving work information processing system according to the first embodiment of the present invention. [Figure 9] This table shows examples of personal registration information for each user in the driving work information processing system according to the first embodiment of the present invention. [Figure 10] This table shows examples of device registration information for each autonomous driving device in the driving work information processing system according to the first embodiment of the present invention. [Figure 11] This table shows examples of user browsing settings in the driving work information processing system according to the first embodiment of the present invention. [Figure 12]This is a plan view showing an example of a remote control screen that displays first captured data in a driving work information processing system according to the first embodiment of the present invention. [Figure 13] This is a plan view showing an example of a remote control screen displaying second captured data in a driving work information processing system according to the first embodiment of the present invention. [Figure 14] This is a plan view showing an example of a remote control screen that displays first and second image data in parallel in a driving work information processing system according to the first embodiment of the present invention. [Figure 15] This is a plan view showing an example of a remote control screen in a driving work information processing system according to the first embodiment of the present invention, which displays first and second captured data in parallel when an unknown obstacle is present. [Figure 16] This is a plan view showing an example of a remote control screen in a driving work information processing system according to the first embodiment of the present invention, which displays first and second captured data in parallel when an unknown obstacle is present. [Figure 17] This is a plan view showing an example of a remote control screen in a driving work information processing system according to the first embodiment of the present invention, which displays first and second captured data in parallel when an unknown obstacle is present. [Figure 18] This is a plan view showing an example of a remote control screen that displays first captured data in a driving work information processing system according to the first embodiment of the present invention, when an unknown obstacle is present. [Figure 19] This is a plan view showing an example of a remote control screen that displays first captured data in a driving work information processing system according to the first embodiment of the present invention, when an unknown obstacle is present. [Figure 20] This is a plan view showing an example of a remote control screen that displays first captured data in a driving work information processing system according to the first embodiment of the present invention, when an unknown obstacle is present. [Figure 21] This is a plan view showing an example of a login screen for an operating terminal in a driving operation information processing system according to the first embodiment of the present invention. [Figure 22]This flowchart shows an example of an operation where a remote control screen is displayed on an operation terminal in a driving operation information processing system according to the first embodiment of the present invention. [Figure 23] This flowchart shows an example of the operation of an autonomous driving device in a driving work information processing system according to the first embodiment of the present invention. [Figure 24] This flowchart shows an example of the operation of the management server in the driving operation information processing system according to the first embodiment of the present invention. [Figure 25] This flowchart shows an example of the operation of an operating terminal in a driving operation information processing system according to the first embodiment of the present invention. [Figure 26] This is a block diagram showing the configuration of the management server in a driving operation information processing system according to a second embodiment of the present invention. [Figure 27] This is a plan view showing an example of a remote control screen displaying first captured data in a driving work information processing system according to a second embodiment of the present invention. [Figure 28] This is a plan view showing an example of a remote control screen displaying second image data of a cleaning unit photographed from above, in a driving work information processing system according to a second embodiment of the present invention. [Figure 29] This is a plan view showing an example of a remote control screen displaying second image data of a cleaning unit photographed from above, in a driving work information processing system according to a second embodiment of the present invention. [Figure 30] This is a plan view showing an example of a remote control screen displaying second image data of a cleaning unit photographed from the side, in a driving work information processing system according to a second embodiment of the present invention. [Figure 31] This is a plan view showing an example of a remote control screen displaying second image data of a cleaning unit photographed from the side, in a driving work information processing system according to a second embodiment of the present invention. [Figure 32] This is a plan view showing an example of turning movement of an autonomous mobile work device when the cleaning unit is operating normally in a mobile work information processing system according to a second embodiment of the present invention. [Figure 33]This is a plan view showing an example of turning movement of an autonomous mobile work device when the cleaning unit malfunctions in a mobile work information processing system according to a second embodiment of the present invention. [Figure 34] This is a plan view showing an example of a remote control screen in a driving operation information processing system according to a second embodiment of the present invention, which displays in parallel second image data of the cleaning unit taken from above and the side. [Figure 35] This is a plan view showing an example of a remote control screen in a driving operation information processing system according to a second embodiment of the present invention, which displays in parallel second image data of the cleaning unit taken from above and the side. [Figure 36] This is a plan view showing an example of a remote control screen displaying multiple users associated with an autonomous driving device in a driving work information processing system according to another embodiment of the present invention. [Modes for carrying out the invention]

[0035] Embodiments of the present invention will be described below with reference to the drawings. The following embodiments are preferred examples of the present invention and disclose various preferred techniques, but the technical scope of the present invention is not limited to these embodiments.

[0036] A first embodiment of the present invention, a driving work information processing system 1, will now be described. As shown in Figure 1, the driving work information processing system 1 comprises an autonomous driving work device 2, a management server 3, and a plurality of operation terminals 4. The autonomous driving work device 2 is configured to perform automated driving work, which involves autonomously driving and performing work automatically. The management server 3 is configured to manage the autonomous driving work device 2, and each operation terminal 4 is configured to operate the autonomous driving work device 2. The autonomous driving work device 2 is connected to the management server 3 via a predetermined network 5 so as to be able to communicate with it, and each operation terminal 4 is connected to the management server 3 via the predetermined network 5 so as to be able to communicate with it.

[0037] The driving operation information processing system 1 may be equipped with multiple autonomous driving devices 2 having similar functions connected to a single management server 3, but this embodiment describes an example in which one autonomous driving device 2 is equipped. Furthermore, the driving operation information processing system 1 may be equipped with multiple operating terminals 4 having similar functions connected to a single management server 3, but this embodiment describes an example in which two operating terminals 4 are equipped. The multiple operating terminals 4 are used not only by operators such as cleaning workers, but also by multiple users including security guards and administrators.

[0038] First, let's describe the autonomous mobile work device 2. As shown in Figure 2, the autonomous mobile work device 2 comprises a main body 10 for housing various parts and a mobile unit 11 for moving the main body 10. The autonomous mobile work device 2 functions as an autonomous mobile cleaning device, equipped with a cleaning unit 12 as a work unit for performing predetermined tasks, for example, cleaning the floor surface FL below the main body 10. The autonomous mobile work device 2 uses all or part of an area such as a shopping mall, office, hotel, hospital, school, or factory as the cleaning area (work area), and as shown in Figure 3, the floor surface FL of the cleaning area is the target of cleaning (work target).

[0039] Furthermore, the autonomous mobile work device 2 includes a measurement unit 13 that measures the positional relationship between the device body 10 and non-working objects such as surrounding walls and obstacles (for example, ornaments), and an obstacle detection unit 14 that detects walls and obstacles within a predetermined distance. The autonomous mobile work device 2 also includes a first imaging unit 15 that photographs the area in front of the device body 10 and a second imaging unit 16 that photographs the area behind the device body 10. The autonomous mobile work device 2 also includes an operation and display unit 17 consisting of a touch panel or the like for operating and displaying various functions of the autonomous mobile work device 2, and a power supply unit 18 that supplies power to each part of the autonomous mobile work device 2 and controls the remaining battery level (not shown) and charging.

[0040] Furthermore, the autonomous mobile work device 2 includes a device-side control unit 20 that comprehensively controls each part and various functions of the autonomous mobile work device 2 (such as driving by the driving unit 11, cleaning by the cleaning unit 12, measurement by the measurement unit 13, and photography by the first imaging unit 15 and the second imaging unit 16), a device-side storage unit 21 that stores a cleaning plan (work plan) consisting of driving data from the driving unit 11 and cleaning data (work data) from the cleaning unit 12, and a device-side communication unit 22 for communicating with external devices such as the management server 3 and each operation terminal 4.

[0041] Next, we will explain the general operation of the autonomous driving work device 2.

[0042] The autonomous driving work device 2 is a vehicle capable of both manual and automatic driving, and operates by switching between one of three operating modes: learning mode, reproduction (automatic) mode, and manual mode. The autonomous driving work device 2 can switch operating modes in response to operations on the operation terminal 4 and the operation display unit 17.

[0043] In learning mode and manual mode, the autonomous driving work device 2 performs learning driving cleaning (learning driving work) and manual driving cleaning (manual driving work), respectively, in response to manual operations on the operation terminal 4 and operation display unit 17 by workers such as cleaning staff, operators, or managers. In addition, during learning driving cleaning, it creates and stores an environmental map of the cleaning area in conjunction with the learning driving, and acquires driving data showing the driving route and driving status during the learning driving, as well as cleaning data showing the cleaning status during the learning cleaning, and stores a cleaning plan including the driving data and cleaning data.

[0044] In reproduction mode, the autonomous driving work device 2 performs automatic driving and cleaning (automatic driving work) in accordance with the automatic operation by the device-side control unit 20, based on the selected cleaning plan and the corresponding environmental map, in order to reproduce the cleaning plan selected by the worker from among the cleaning plans stored in the learned driving cleaning.

[0045] The cleaning plan consists of multiple steps along the travel route, with travel data and cleaning data associated with each step. The interval between steps may be set to a predetermined time interval (e.g., 25 ms) when performing a learning-type cleaning run, or it may be set to a predetermined travel distance of the autonomous work device 2 (e.g., 0.5 m). In addition to the above, the cleaning plan may also store the elapsed time from the start of the learning-type cleaning run associated with each step.

[0046] The driving data includes, for example, the self-position data of the driving unit 11 on its driving path (X and Y coordinates indicating its position on the environmental map, and the angle relative to the orientation of the starting position), steering flags (straight, left turn, right turn), driving speed [m / s] in the direction of travel, and turning speed [deg / s], and the driving path can be visualized based on the driving data. Alternatively, the driving speed may be switched to any of several speed levels, and the driving data may be stored by converting the speed levels into numbers (for example, 8 levels from 0 to 7).

[0047] The cleaning data includes, for example, the pad pressure of the cleaning member 24, the amount of water supplied by the cleaning fluid supply unit 25, and the suction amount of the suction unit 26. The pad pressure is the force with which the cleaning member 24 is pressed against the floor surface FL, the amount of water supplied is the amount of cleaning fluid (working fluid) supplied to the floor surface FL by the cleaning fluid supply unit 25, and the suction amount is the operating strength of the suction blower (not shown) when the suction unit 26 sucks up the wastewater after cleaning from the floor surface FL. The pad pressure, water supply amount, and suction amount may be switched to any of several strength levels, and the cleaning data may be stored by converting the stepped strengths of the pad pressure, water supply amount, and suction amount into numbers (for example, 3 levels from 0 to 2, or 5 levels from 0 to 4). The cleaning data may also include the operation / stop and rotation speed of the cleaning member 24.

[0048] Next, we will explain the various parts of the autonomous driving work device 2.

[0049] The running unit 11 is located at the bottom of the main body 10 and is equipped with one front wheel 11a as a drive wheel and a pair of rear wheels 11b as auxiliary wheels. Tires are attached to both the front wheel 11a and the pair of rear wheels 11b. The front wheel 11a is located at the front of the device in the width direction and is centered in the device width direction. It is equipped with a driving motor (not shown) and a front wheel rotation encoder (not shown). The running unit 11 moves the main body 10 forward by driving the driving motor to rotate the front wheel 11a, and stops the main body 10 by stopping the rotation of the front wheel 11a. The speed of the autonomous mobile work device 2 (running unit 11) is adjusted (accelerated / decelerated) by controlling the drive of the driving motor. The running unit 11 may also move the main body 10 backward by having the driving motor reverse the rotation of the front wheel 11a.

[0050] Furthermore, the front wheel 11a is equipped with a steering shaft (not shown), a steering motor (not shown), and a steering rotation encoder (not shown). The driving unit 11 steers the device body 10 by changing the direction of the front wheel 11a by driving the steering motor to rotate the steering shaft, and by moving the device body 10 forward while changing the direction of the front wheel 11a, the autonomous driving work device 2 (driving unit 11) turns left (left turn) or right (right turn). The steering angle of the autonomous driving work device 2 (driving unit 11) is adjusted by controlling the drive of the steering motor. For example, by rotating the steering shaft and controlling the steering motor to tilt the steering angle 90 degrees to the left or right with respect to the direction of travel, while turning the device body 10 forward or backward, the device body 10 makes a pivot turn to the left or right.

[0051] A pair of rear wheels 11b are provided at the rear in the direction of travel, spaced apart in the width direction (left-right direction) of the device, and each is equipped with an encoder (not shown) for determining the travel distance from the amount of rotation. The pair of rear wheels 11b rotate in response to the movement of the device body 10 driven by the front wheels 11a. The adjustment (acceleration / deceleration) of the travel speed (travel unit 11) and steering of the autonomous travel work device 2 (travel unit 11) may be performed by controlling the travel drive motor and the steering motor while feeding back the amount of rotation of each rear wheel 11b using the encoders provided on the rear wheels 11b. In this embodiment, an example in which the device is equipped with drive front wheels 11a and auxiliary rear wheels 11b has been described, but the present invention is not limited to this example, and in other embodiments, for example, the device may be equipped with auxiliary front wheels 11a and a pair of drive rear wheels 11b.

[0052] When the operating mode is set to learning mode or manual mode, the travel unit 11 operates in response to manual operation by the operator or worker on the operation terminal 4 or operation display unit 17. When the operating mode is set to reproduction mode, the travel unit 11 operates in response to control by the device-side control unit 20 (reproduction control unit 34) based on the travel data and environmental map of the cleaning plan selected by the operator or worker.

[0053] The cleaning unit 12 is located at the bottom of the main unit 10 and is configured to clean the floor surface FL below the main unit 10. When the operating mode is set to learning mode or manual mode, the cleaning unit 12 operates in response to manual operation by the operator or worker on the operation terminal 4 or operation display unit 17. When the operating mode is set to reproduction mode, the cleaning unit 12 operates in response to control (automatic operation) by the device-side control unit 20 (reproduction control unit 34) based on the cleaning data of the cleaning plan selected by the operator or worker.

[0054] The cleaning unit 12 is configured as a wet cleaning mechanism that cleans the floor surface FL using a cleaning solution, and includes a cleaning member 24 (working member) that contacts and cleans the floor surface FL, a cleaning solution supply unit 25 that supplies cleaning solution to the floor surface FL, and a suction unit 26 that sucks up the used cleaning solution, i.e., the wastewater, after cleaning the floor surface FL. The cleaning unit 12 includes a motor for the cleaning member (not shown) that rotates the cleaning member 24 on the floor surface FL, and a actuator for the cleaning member (not shown) that moves the cleaning member 24 vertically relative to the floor surface FL and varies the pad pressure. The cleaning unit 12 also includes a wastewater collection unit (not shown) that collects the wastewater sucked up by the suction unit 26. Note that if the floor surface FL is carpeted, the cleaning unit 12 may be configured to perform dry cleaning without using a cleaning solution. Alternatively, the autonomous mobile work device 2 may be a dry floor cleaning machine that sucks up and removes dust from the floor surface FL without the cleaning solution supply unit 25.

[0055] The cleaning component 24 consists of a pair of cleaning pads or a pair of cleaning brushes mounted side by side in the width direction (left-right direction) of the device approximately in the center of the direction of travel. The left cleaning pad or cleaning brush rotates clockwise when viewed from above, and the right cleaning pad or cleaning brush rotates counterclockwise when viewed from above, rotating from front to back on the center side in the width direction. This collects the dirty water and dust in front of the pair of cleaning pads or the pair of cleaning brushes towards the center side in the width direction and discharges them to the rear. Note that the cleaning pads or cleaning brushes may consist of a single unit instead of a pair.

[0056] The cleaning solution supply unit 25 is configured to supply and spray cleaning solution from a cleaning solution tank (not shown) onto the floor surface FL using a supply pump (not shown).

[0057] The suction unit 26 consists of a suction blower and the like, and collects the sucked-up wastewater into a wastewater collection unit (not shown). The wastewater collection unit uses a squeegee 27, which is set in contact with the floor surface FL behind the cleaning member 24, to receive and collect the wastewater discharged from the cleaning member 24 to the rear. The collected wastewater is then sucked up using the suction force of the suction unit 26 by a wastewater duct (not shown) connected to the suction unit 26, and the sucked-up wastewater is collected into a wastewater tank connected to the wastewater duct.

[0058] The measurement unit 13 includes a laser range finder (LRF) 13a that measures positional information (for example, the angle and distance with respect to the direction of travel of the device body 10) between the device body 10 and non-working objects such as surrounding walls and obstacles. The measurement unit 13 measures positional information with respect to non-working objects at predetermined intervals (for example, every 25 ms) while the device body 10 is moving. The LRF 13a is configured to have a detection range that extends to the front and both left and right sides of the device body 10.

[0059] The obstacle detection unit 14 includes multiple proximity sensors 14a (detection units), such as ultrasonic sensors, that detect the presence or absence of walls or obstacles within a predetermined distance from the main unit 10; step sensors, such as infrared sensors, that detect steps in the floor surface FL near the main unit 10; and bumper sensors, which are attached to the front lower bumper (not shown) of the main unit 10 and detect contact with walls or obstacles. The obstacle detection unit 14 may operate continuously while the main unit 10 is in motion.

[0060] The first imaging unit 15 is used to photograph the driving status of the autonomous mobile work device 2 and includes a first imaging device 15a such as a CCD (Charge Coupled Device) camera. The first imaging device 15a is mounted, for example, on the front upper side of the main body 10 and photographs the area in front of the main body 10 to photograph the driving status of the autonomous mobile work device 2 and acquire first photographic data 60 as shown in Figure 7. The first imaging unit 15 is not limited to photographing in one direction and acquiring one first photographic data 60, but may also photograph in multiple directions and acquire two or more first photographic data 60.

[0061] The second imaging unit 16 is for capturing images of the cleaning operation status (operation status) of the autonomous mobile work device 2, and includes a second imaging device 16a such as a CCD camera. The second imaging device 16a is equipped with a CCD camera or the like that captures the rear of the device body 10, and the second imaging device 16a is, for example, mounted on the rear lower side of the device body 10, and captures images of the cleaning operation status of the autonomous mobile work device 2 by capturing images of the rear lower side of the device body 10 from above and from the side, thereby acquiring second image data 61 as shown in Figure 8. Alternatively, the second imaging unit 16 may consist of a wide-angle camera mounted on the rear of the device body 10, and capture images from the rear upper side in a diagonal rear lower direction. The second imaging device 16a is preferably arranged to capture images including the cleaning unit 12 (e.g., squeegee 27). Furthermore, the second imaging unit 16 is not limited to capturing images in one direction and acquiring one second image data 61, but may capture images in multiple directions and acquire two or more second image data 61.

[0062] The operation display unit 17 is located on the upper rear side of the main body 10 of the device and is configured to include a touch panel or the like. Each part of the operation display unit 17 is connected to the device-side control unit 20. The operation display unit 17 includes, for example, a key switch (not shown) for operating the autonomous mobile work device 2 and an emergency stop button (not shown) for operating the autonomous mobile work device 2 in an emergency.

[0063] The touch panel of the operation display unit 17 displays various screens in response to control signals from the device-side control unit 20, and transmits operation signals to the device-side control unit 20 based on touch operations on each screen. For example, the touch panel of the operation display unit 17 may display a mode selection screen (not shown) on which the operating mode can be selected, or a setting screen (not shown) on which, in learning mode and manual mode, the travel speed of the travel unit 11, the operation / stop and pad pressure of the cleaning member 24, the operation / stop and water supply volume of the cleaning fluid supply unit 25, the operation / stop and suction volume of the suction unit 26, etc.

[0064] The power supply unit 18 includes a battery (power source) and a charging circuit mounted inside the main unit 10 of the device. When connected to an external power source, the battery is charged, and power is supplied to each part of the autonomous driving work device 2. The power supply unit 18 may also output a signal indicating the remaining battery level to the device-side control unit 20.

[0065] The device-side control unit 20 is composed of a computer such as a CPU (Central Processing Unit) and is connected to the device-side storage unit 21 and the device-side communication unit 22, which include ROM (Read Only Memory), RAM (Random Access Memory), hard disk, and flash memory, as shown in Figures 2 and 4. The device-side control unit 20 is also connected to each part of the autonomous driving work device 2, such as the driving unit 11, cleaning unit 12, measurement unit 13, obstacle detection unit 14, first imaging unit 15, second imaging unit 16, operation display unit 17, and power supply unit 18.

[0066] Furthermore, the device-side control unit 20 is connected to external devices via the device-side communication unit 22. The device-side communication unit 22 communicates wirelessly with external devices such as the management server 3 and multiple operation terminals 4 using a wireless LAN communication standard such as Wi-Fi. For example, while the autonomous mobile work device 2 is performing automatic mobile cleaning, the device-side communication unit 22 transmits to the management server 3 first image data 60 captured by the first image unit 15 showing the driving status of the autonomous mobile work device 2, second image data 61 captured by the second image unit 16 showing the cleaning status of the autonomous mobile work device 2, and measurement results from the measurement unit 13. Alternatively, the first image data 60, second image data 61, and environmental map data as measurement results from the measurement unit 13 may be stored in the device-side storage unit 21 and transmitted to the management server 3 according to the timing.

[0067] The device-side storage unit 21 stores programs and data for controlling each part and various functions of the autonomous driving work device 2, and the device-side control unit 20 performs calculation processing based on the programs and data stored in the device-side storage unit 21 to comprehensively control each part and various functions. For example, by executing a program stored in the device-side storage unit 21, the device-side control unit 20 operates as a mode switching unit 30, a learning control unit 31, a map creation unit 32, a cleaning plan creation unit 33, a reproduction control unit 34, and a remote operation control unit 35, as shown in Figure 4. As a result, the autonomous driving work device 2 can autonomously drive and perform work automatically according to a pre-stored program. In addition, the device-side storage unit 21 stores one or more cleaning plans, as well as environmental maps corresponding to each cleaning plan.

[0068] The mode switching unit 30 switches the operating mode to one of the following: learning mode, reproduction mode, or manual mode, depending on the operation of each operating terminal 4 or the operation display unit 17.

[0069] In learning mode, the learning control unit 31 acquires travel data from the travel unit 11 and cleaning data from the cleaning unit 12 at predetermined step intervals while learning travel cleaning is performed, and stores them in the device-side storage unit 21.

[0070] The learning control unit 31 acquires, for example, self-position data (X and Y coordinates, angle) based on position information measured by the measurement unit 13 as driving data. It also detects the number of rotations of the front wheels 11a using the front wheel rotation encoder of the front wheels 11a of the driving unit 11 and acquires the driving speed of the driving unit 11 based on the detection result. Furthermore, it detects the number of steering rotations of the front wheels 11a using the steering rotation encoder of the front wheels 11a of the driving unit 11 and acquires the turning speed of the driving unit 11 based on the detection result. The learning control unit 31 acquires, for example, cleaning data, including the operation / stop and pad pressure set for the cleaning member 24, the operation / stop and water supply amount set for the cleaning fluid supply unit 25, and the operation / stop and suction amount set for the suction unit 26.

[0071] The map creation unit 32, while learning-based driving and cleaning is performed in learning mode, uses technologies such as SLAM (Simultaneous Localization and Mapping) to estimate its own position and create an environmental map in real time.

[0072] Specifically, during the learning-type cleaning run in a predetermined cleaning area, the map creation unit 32 acquires positional information of the device body 10 and non-working objects such as the walls surrounding the device body 10 as measurement results from the measurement unit 13, and creates a local map of the area around the device body 10 at predetermined time intervals or predetermined distance intervals based on these measurement results from the measurement unit 13. Furthermore, the map creation unit 32 estimates the self-position (coordinates) of the autonomous driving work device 2 in the local map based on the local map and the detection results from each encoder of the driving unit 11 (amount of movement of the driving unit 11).

[0073] Then, when the learning run cleaning is completed, the map creation unit 32 creates an environmental map of the cleaning area by connecting (combining) the local maps. The map creation unit 32 also creates a running route by connecting (combining) its own position (the location information measured by the measurement unit 13) in the local maps. By using this environmental map, the state of the running route within the work area (such as the presence or absence of unknown obstacles) can be detected based on the self-position in the local maps and the location information of non-work targets measured by the measurement unit 13.

[0074] The cleaning plan creation unit 33 associates the running data and cleaning data that the learning control unit 31 has stored in the device-side storage unit 21 while the learning running cleaning is being performed, step by step, when the learning running cleaning is completed, to create a cleaning plan (work plan).

[0075] The cleaning plan creation unit 33 accepts the input of a plan name for the created cleaning plan on a predetermined input screen (not shown) on the touch panel of the operation display unit 17, and adds the plan name entered by the operator or the like to the created cleaning plan. The cleaning plan creation unit 33 stores the created cleaning plan in the device-side storage unit 21, associating it with the environmental map created by the map creation unit 32 in the same learning-run cleaning run. In addition, the cleaning plan creation unit 33 may create a cleaning plan for environmental maps, running data, and cleaning data that have been created in advance by using the autonomous driving work device 2 as a computer, or by external devices such as the management server 3 or each operation terminal 4, or by automatically creating a driving route according to a pre-entered program, without performing a learning-run cleaning run.

[0076] When the operating mode is in reproduction mode, the reproduction control unit 34 accepts the selection of a cleaning plan on a predetermined cleaning plan selection screen (not shown) on the touch panel of the operation display unit 17, reads the selected cleaning plan from the device-side storage unit 21, and controls the driving unit 11 and the cleaning unit 12 to perform automatic driving cleaning based on the driving data and cleaning data of this cleaning plan and the environmental map corresponding to the cleaning plan.

[0077] At this time, the reproduction control unit 34 performs automated cleaning while estimating the self-position of the autonomous driving work device 2 on the environmental map corresponding to the cleaning plan. For example, the reproduction control unit 34 uses the map creation unit 32 to create a local map using technologies such as SLAM, estimates the self-position of the autonomous driving work device 2 in the local map, and matches the local map to the environmental map to estimate its self-position on the environmental map.

[0078] The reproduction control unit 34 then controls the driving unit 11 by combining the self-position data for each step of the driving data of the cleaning plan with the self-position estimated on the environmental map, and controls the cleaning work step by step based on the cleaning data of the cleaning unit 12.

[0079] Furthermore, in automatic driving cleaning, the reproduction control unit 34 detects non-working objects such as obstacles using the laser rangefinder 13a of the measurement unit 13 and the proximity sensor 14a of the obstacle detection unit 14, and controls the driving unit 11 to decelerate when the distance to the non-working object approaches a predetermined safety distance, and controls the driving unit 11 to stop when the distance to the non-working object falls below the predetermined safety distance.

[0080] In the automated driving cleaning process, the reproduction control unit 34 transmits to the management server 3 via the device-side communication unit 22 the first image data 60 captured by the first image unit 15 showing the driving status of the autonomous driving work device 2, the second image data 61 captured by the second image unit 16 showing the cleaning work status of the autonomous driving work device 2, and the measurement results from the measurement unit 13.

[0081] When the reproduction control unit 34 detects an error during automatic cleaning that causes the autonomous driving work device 2 to be unable to continue driving, it sends an error notification indicating the inability to drive to the management server 3 via the device-side communication unit 22. For example, if the reproduction control unit 34 detects an unknown obstacle that is not present on the environmental map in the direction of travel based on the measurement results of the measurement unit 13, it will stop the automatic cleaning when it approaches the obstacle until the distance from the autonomous driving work device 2 to the obstacle reaches a predetermined stopping distance, and will enter a state where it is waiting for manual or remote operation. When the reproduction control unit 34 detects an unknown obstacle, it sends an error notification indicating the inability to drive due to the obstacle to the management server 3 via the device-side communication unit 22.

[0082] Furthermore, if the reproduction control unit 34 detects an error during automatic cleaning that prevents the autonomous work device 2 from continuing the cleaning work, it sends an error notification indicating the inability to work to the management server 3 via the device-side communication unit 22. For example, the reproduction control unit 34 monitors the operation of each part of the cleaning unit 12, such as the cleaning member 24, the cleaning liquid supply unit 25, and the suction unit 26, using sensors, and if it detects a malfunction in the cleaning work (for example, residual water on the floor surface FL due to a problem with the suction of dirty water) based on the monitoring results, it stops the automatic cleaning and enters a state where it is waiting for manual or remote operation. If the reproduction control unit 34 detects a malfunction in the cleaning unit 12, it sends an error notification indicating the inability to work due to the malfunction to the management server 3 via the device-side communication unit 22.

[0083] When the operating mode is in reproduction mode, if the device-side communication unit 22 receives a remote control command from an external device such as the management server 3 or each operation terminal 4 while the autonomous mobile work device 2 is performing automatic mobile cleaning, the remote control control unit 35 will perform remote mobile driving instead of automatic mobile cleaning. During remote mobile driving, the remote control control unit 35 controls the mobile unit 11 and the cleaning unit 12 so that the autonomous mobile work device 2 moves according to the remote control commands. When the remote control control unit 35 performs remote mobile driving, it temporarily suspends automatic mobile cleaning, and when it finishes remote mobile driving, the reproduction control unit 34 restarts automatic mobile cleaning. The reproduction control unit 34 may restart automatic mobile cleaning automatically in response to the end of remote mobile driving, or in response to a restart operation using the operation terminal 4 (for example, operation of the restart button). At this time, the reproduction control unit 34 restarts automatic mobile cleaning from the position on the travel path closest to the position of the autonomous mobile work device 2.

[0084] For example, if the remote control instructs the autonomous mobile work device 2 to move forward, stop, turn right, or turn left, the remote control unit 35 controls the mobile unit 11 so that the autonomous mobile work device 2 moves forward, stops, turns right, or turns left, and may also accept the setting of the mobile speed of the mobile unit 11 via remote control. The remote control unit 35 may accept the setting of execution / stop of cleaning work by the cleaning unit 12 via remote control, and may also accept individual settings such as operation / stop of cleaning of cleaning member 24 and pad pressure, operation / stop of cleaning fluid supply of cleaning fluid supply unit 25 and supplied water volume, and operation / stop of suction of suction unit 26 and suction volume. In addition, the remote control unit 35 may control the cleaning unit 12 to always stop cleaning work when remotely controlled travel is performed, regardless of remote control.

[0085] Furthermore, if the remote control unit 35 instructs the remote control to avoid an obstacle, in addition to controlling the driving unit 11 according to the remote control, it may also control the driving unit 11 to maintain a predetermined safe distance from the obstacle based on the measurement results of the measurement unit 13. At this time, the remote control unit 35 may continue remote driving while detecting an obstacle in the direction of travel of the autonomous driving work device 2, and terminate remote driving when it no longer detects an obstacle. Alternatively, the remote control unit 35 may terminate remote driving in response to the remote control. In addition, if the remote control unit 35 instructs automatic obstacle avoidance, it may control the driving unit 11 to avoid the obstacle by automatic driving, and terminate remote driving by automatic driving when it no longer detects an obstacle. When the remote control unit 35 terminates remote driving, it sends a notification of termination of remote driving to the management server 3 via the device-side communication unit 22.

[0086] Furthermore, when remote control instructs the first shooting unit 15 or the second shooting unit 16 to perform shooting operations, the remote control control unit 35 controls the shooting of the first shooting unit 15 or the second shooting unit 16, and the transmission of the first shooting data 60 captured by the first shooting unit 15 and the second shooting data 61 captured by the second shooting unit 16. The remote control control unit 35 may also control the shooting direction and zoom for the shooting of the first shooting unit 15 or the second shooting unit 16. The remote control control unit 35 is not limited to the remote control described above, and may accept various remote controls for each part of the autonomous driving work device 2, enabling control of each part.

[0087] Next, the management server 3 will be described. The management server 3 manages the autonomous driving work device 2 and also manages the operating rights of users such as operators and administrators who use each operating terminal 4. As shown in Figure 5, the management server 3 comprises a server-side control unit 40, a server-side storage unit 41, and a server-side communication unit 42.

[0088] The server-side control unit 40 is responsible for the overall control of each part and various functions of the management server 3. It is composed of a computer such as a CPU and is connected to the server-side storage unit 41, which includes ROM, RAM, hard disk, flash memory, etc., and the server-side communication unit 42.

[0089] Furthermore, the server-side control unit 40 is connected to external devices via the server-side communication unit 42 to enable communication. The server-side communication unit 42 communicates wirelessly with external devices such as the autonomous driving work device 2 and each operation terminal 4 held by users such as operators and administrators, using a wireless LAN communication standard such as Wi-Fi. For example, the server-side communication unit 42 receives from the autonomous driving work device 2 first image data 60 captured by the first image unit 15 of the autonomous driving work device 2 capturing the driving status of the autonomous driving work device 2, second image data 61 captured by the second image unit 16 capturing the cleaning work status of the autonomous driving work device 2, and measurement results from the measurement unit 13 of the autonomous driving work device 2. It also displays the first image data 60 and the second image data 61 on a remote operation screen 70 (see Figures 12 to 20, etc.) on each operation terminal 4. The server-side communication unit 42 receives remote operations from each operation terminal 4 and transmits remote operation commands corresponding to those remote operations to the autonomous driving work device 2.

[0090] The server-side storage unit 41 stores programs and data for controlling each part and various functions of the management server 3, and the server-side control unit 40 performs calculations based on the programs and data stored in the server-side storage unit 41 to comprehensively control each part and various functions. For example, the server-side control unit 40 operates as the device terminal linking unit 45, the image viewing control unit 46, and the remote control command unit 47 by executing the programs stored in the server-side storage unit 41.

[0091] The management server 3 manages users such as operators and administrators of each operation terminal 4. As shown in Figure 9, the server-side storage unit 41 stores personal registration information for each user, including personal identification information (personal code), name, affiliation, user type, and password. The management server 3 also manages the autonomous driving work devices 2. As shown in Figure 10, the server-side storage unit 41 stores device registration information for each autonomous driving work device 2, including device identification information (robot identification code) and name. Furthermore, the management server 3 receives cleaning plans and environmental maps created by the autonomous driving work devices 2 during learning-based cleaning, as well as the operation details of the operation terminal 4 by the worker, via the server-side communication unit 42, and stores the history in the server-side storage unit 41.

[0092] Furthermore, the management server 3 allows for the registration of viewing settings for screens such as the remote control screen 70 (see Figures 12 to 20, etc.) which are displayed on each operation terminal 4 for remote control of the autonomous mobile work device 2 that performs automatic mobile cleaning, for each user whose personal registration information has been registered. For example, as shown in Figure 11, the server-side storage unit 41 stores personal identification information (personal code), image viewing settings, etc., as viewing settings for each user's remote control screen 70. The viewing settings may be registered in advance when the user's personal registration information is registered, and may also be registered and changed in response to any operation using the operation terminal 4 by an authorized administrator or other person.

[0093] The remote control screen 70 can display photographic data such as the first photographic data 60, which is a photograph of the autonomous mobile work device 2's driving status taken by the first photographic unit 15 of the autonomous mobile work device 2, and the second photographic data 61, which is a photograph of the autonomous mobile work device 2's cleaning status taken by the second photographic unit 16. The image viewing setting indicates the display type, which of the photographic data to display. Display types include "Camera 1," which displays the first photographic data 60 alone; "Camera 2," which displays the second photographic data 61 alone; and "Camera 1 / Camera 2," which displays the first photographic data 60 and the second photographic data 61 in parallel. Note that the display type for displaying photographic data in parallel is not limited to two photographic data sets, but may display three or more photographic data sets in parallel. Furthermore, for display types that display two or more photographic data sets in parallel, the arrangement of the photographic data may be set in advance, or it may be set according to any operation by the user using the operation terminal 4.

[0094] On the management server 3, at least one display type is set as an option in each user's image viewing settings, and on the remote control screen 70, the captured data is displayed according to the one display type selected from the one or more display types. The one or more display types set in each user's image viewing settings may be set automatically according to each user's user type and affiliation, or they may be set according to any operation performed by the user using the operation terminal 4. For example, if the user type is administrator, the image viewing settings may be set to display types "Camera 1", "Camera 2", and "Camera 1 / Camera 2"; if the user type is an operator responsible for both driving and cleaning, the image viewing settings may be set to display types "Camera 1" and "Camera 2"; if the user type is an operator responsible only for driving, the image viewing settings may be set to display type "Camera 1"; if the user type is an operator responsible only for cleaning, the image viewing settings may be set to display type "Camera 2"; and if the user type is a security guard, the image viewing settings may be set to display type "Camera 1".

[0095] Furthermore, in each user's image viewing settings, one of one or more display types is set as the priority display, and on the remote control screen 70, the priority display type is selected by default, or the user may be able to select a display type other than the priority display in response to any operation using the operation terminal 4. The priority display may be set automatically according to each user's user type and affiliation, or it may be set in response to any operation by the user using the operation terminal 4. In Figure 11, the user types for priority display are indicated by white ovals. For example, if the user type is administrator, the priority display is set to "Camera 1 / Camera 2", if the user type is an operator responsible for both driving and cleaning, the priority display is set to "Camera 1", if the user type is an operator responsible only for driving, the priority display is set to "Camera 1", if the user type is an operator responsible only for cleaning, the priority display is set to "Camera 2", and if the user type is a security guard, the priority display is set to "Camera 1".

[0096] The device terminal linking unit 45 links the autonomous driving work device 2 and the operation terminal 4 in order to enable remote control of the autonomous driving work device 2 by the operation terminal 4. For example, the device terminal linking unit 45 receives a selection request for an autonomous driving work device 2 that is desired for remote control from an operation terminal 4 logged into the management server 3, and links the autonomous driving work device 2 with the operation terminal 4. The device terminal linking unit 45 can, of course, link one operation terminal 4 to one autonomous driving work device 2, but if there are selection requests from two or more operation terminals 4 for one autonomous driving work device 2, it can also link two or more operation terminals 4 to one autonomous driving work device 2.

[0097] The device-terminal linking unit 45 stores linking information between the operating terminal 4 and the autonomous driving work device 2 linked to it in the server-side storage unit 41 while the operating terminal 4 is logged into the management server 3. When the device-terminal linking unit 45 links the operating terminal 4 and the autonomous driving work device 2, it sends a success notification to the operating terminal 4 for the selection request. The device-terminal linking unit 45 also deletes the linking information between the operating terminal 4 and the autonomous driving work device 2 from the server-side storage unit 41 when the operating terminal 4 logs out of the management server 3, cancels the selection request for the autonomous driving work device 2, or makes a selection request to another autonomous driving work device 2.

[0098] The image viewing control unit 46 selects the first image data 60 captured by the first image unit 15 and the second image data 61 captured by the second image unit 16 in the autonomous driving work device 2, and displays them for viewing on the operation terminal 4. For example, the image viewing control unit 46 creates a remote control screen 70 that displays the first image data 60 and the second image data 61, and presents the remote control screen 70 to the operation terminal 4. The image viewing control unit 46 basically displays the video of the first image data 60 and the second image data 61 on the remote control screen 70, but may also display still images extracted from the first image data 60 and the second image data 61 in response to any operation performed by the user using the operation terminal 4. It is preferable that the image viewing control unit 46 creates the remote control screen 70 as a screen created on a website managed by the management server 3, and make the remote control screen 70 corresponding to each operation terminal 4 accessible only to that operation terminal 4.

[0099] Specifically, the image viewing control unit 46 reads the user's personal registration information from the server-side storage unit 41 when each of the one or more operating terminals 4 linked to each autonomous driving work device 2 that performs automatic driving cleaning, in order to create a remote control screen 70 for each of the operating terminals 4 linked to the management server 3. The image viewing control unit 46 reads the viewing settings corresponding to the personal identification information of each personal registration information read from the server-side storage unit 41, and selects the display type (captured data) to be viewed by each user based on the image viewing settings of the viewing settings. The image viewing control unit 46 creates a remote control screen 70 that displays the captured data of the selected display type for each user, and displays it for viewing on the operating terminal 4 corresponding to that user. Note that the image viewing control unit 46 does not need to display the captured data at all times on the remote control screen 70 displayed on the operating terminal 4. For example, the captured data may be displayed when the management server 3 receives an error notification, or it may be configured so that, after automatic cleaning, the captured data stored in the device-side storage unit 21 is read and displayed in response to instructions entered by each user on the remote control screen 70.

[0100] The image viewing control unit 46 displays, on the remote operation screen 70 displayed on the operation terminal 4, the login name 71 indicating the user logged in on the operation terminal 4 and the device name 72 indicating the autonomous driving work device 2 associated with the operation terminal 4, as shown in Figures 12 to 20. The image viewing control unit 46 also displays, on the remote operation screen 70, an image display area 73 for displaying the selected shooting data for each user and a camera operation field 74 for performing shooting operations on the first shooting unit 15 and the second shooting unit 16 that capture the shooting data. In the camera operation field 74, the image viewing control unit 46 displays the zoom button 75 for operating the zoom of the first shooting unit 15 and the second shooting unit 16 and the shooting direction button 76 for operating the shooting direction of the first shooting unit 15 and the second shooting unit 16 in an operable manner. When displaying shooting data captured by a wide-angle camera as the first shooting unit 15 and the second shooting unit 16 in the image display area 73, the image may be displayed with the area to be focused on as the center of the field of view, or it may be displayed in a magnified view.

[0101] Furthermore, the image viewing control unit 46 may display a travel operation section (not shown) on the remote operation screen 70 for operating the travel unit 11, and in the travel operation section, remote operation of the travel unit 11 such as forward movement, stopping, right turn, or left turn may be enabled. In addition, the image viewing control unit 46 may display a cleaning operation section (not shown) on the remote operation screen 70 for operating the cleaning unit 12, and in the cleaning operation section, remote operation of the cleaning member 24 of the cleaning unit 12, pad pressure, operation of the cleaning liquid supply unit 25 of the cleaning unit 12, water supply volume, operation of the suction unit 26 of the cleaning unit 12, suction volume, etc. may be enabled.

[0102] If the display type "Camera 1" is selected from the image viewing settings (for example, if "Camera 1" is selected as the priority display in the image viewing settings for operators or security guards who are only responsible for driving), the image viewing control unit 46 displays only the first captured data 60 in the image display area 73, as shown in Figure 12. If the display type "Camera 2" is selected from the image viewing settings (for example, if "Camera 2" is selected as the priority display in the image viewing settings for operators who are only responsible for cleaning), the image viewing control unit 46 displays only the second captured data 61 in the image display area 73, as shown in Figure 13. If the display type "Camera 1 / Camera 2" is selected from the image viewing settings (for example, if "Camera 1 / Camera 2" is selected as the priority display in the image viewing settings for operators or administrators who are responsible for both driving and cleaning), the image viewing control unit 46 displays the first captured data 60 and the second captured data 61 side by side in the image display area 73, as shown in Figure 14.

[0103] When displaying shooting data side-by-side, one set of shooting data may be displayed larger than the other. For example, the first shooting data 60 may be displayed in a large window, and the second shooting data 61 may be displayed in a smaller window within or outside the range of the first shooting data 60. It is recommended to set which shooting data to display larger in the viewing settings beforehand. Alternatively, when displaying shooting data side-by-side, the data may initially be displayed at the same size, and then switch to displaying one set of shooting data larger and the other smaller depending on the selection operation of one set of shooting data.

[0104] Furthermore, when the image viewing control unit 46 receives an error notification from the autonomous driving work device 2, it changes the captured data displayed in the image display area 73 based on the user type of the user logged in on the operation terminal 4, regardless of the display type selected from the image viewing settings. For example, if the autonomous driving work device 2 detects an unknown obstacle that does not exist on the environmental map in the direction of travel, the autonomous driving work device 2 will send a notification to the management server 3 indicating a warning that it may become unable to move because it cannot avoid the unknown obstacle, or an error notification indicating that it is unable to move. At this time, as shown in Figure 15, the image viewing control unit 46 will display the first captured data 60 of the unknown obstacle in the image display area 73. If the user type is an administrator or security guard, the image viewing control unit 46 does not change the captured data displayed in the image display area 73. However, if the user type is an operator, it displays the first captured data 60 and the second captured data 61 in parallel in the image display area 73, similar to the administrator's remote control screen 70 shown in Figure 14. Furthermore, when the autonomous driving device 2 travels around an unknown obstacle, as shown in Figure 16, the unknown obstacle may not be captured in the first image data 60 but may be captured in the second image data 61. Also, after the autonomous driving device 2 has passed around the unknown obstacle, as shown in Figure 17, the unknown obstacle will no longer be captured in either the first image data 60 or the second image data 61.

[0105] Alternatively, the image viewing control unit 46 may change the image data displayed in the image display area 73 based not only on the user type but also on the content of the error notification. For example, if the user type is an operator responsible only for driving, and the error notification is due to the inability to drive because the operator was unable to avoid an obstacle while maintaining a predetermined distance (safe distance) from the obstacle, the image viewing control unit 46 may simply display only the first image data 60 in the image display area 73. However, if the error notification is due to the inability to perform the work, the image viewing control unit 46 will change the display to show both the first image data 60 and the second image data 61 in parallel in the image display area 73. Similarly, if the user type is an operator responsible only for cleaning, and the error notification is due to the inability to perform the work, the image viewing control unit 46 may simply display only the second image data 61 in the image display area 73. However, if the error notification is due to the inability to drive, the image viewing control unit 46 will change the display to show both the first image data 60 and the second image data 61 in parallel in the image display area 73.

[0106] When an error notification is received, the parallel display of the captured data should be such that a different captured data is displayed larger than the one displayed larger before the error notification. For example, if the first captured data 60 was displayed larger and the second captured data 61 was displayed smaller before the error notification, when an error notification is received, the first captured data 60 should be displayed smaller and the second captured data 61 should be displayed larger. Alternatively, when an error notification is received, the parallel display of the captured data should be such that if the error is due to driving, the first captured data 60 is displayed larger, and if the error is due to cleaning work, the second captured data 61 is displayed larger.

[0107] Furthermore, if the error notification is due to an inability to perform the operation, the image viewing control unit 46 may control each unit to display the second captured data 61 centered on the point of focus in the image display area 73, with the cleaning unit 12 (e.g., the squeegee 27) as the point of focus. For example, the image viewing control unit 46 may detect the point of focus based on the second captured data 61, and based on the detection result, control the shooting direction of the second imaging device 16a so that the point of focus becomes the center of the image in the second captured data 61, in order to capture an image centered on the point of focus using the second imaging unit 16. In addition, when the remote control command unit 47 controls the second imaging unit 16 in response to remote operation by the operation terminal 4, the image viewing control unit 46 may display the second captured data 61 captured by the second imaging device 16a with its shooting direction controlled in the image display area 73.

[0108] Furthermore, if the error notification is due to the vehicle becoming inoperable, the image viewing control unit 46 may control its components to display the captured data of the unknown obstacle causing the inoperability as a designated point of focus in the image display area 73.

[0109] For example, the image viewing control unit 46 acquires the environmental map obtained from the autonomous driving work device 2 and the measurement results from the measurement unit 13, and based on these, it grasps the location information of unknown obstacles on the environmental map. The image viewing control unit 46 also acquires the coordinate system of the first captured data 60 using visual SLAM or the like. The image viewing control unit 46 then acquires the feature points of the environmental map and the feature points of the first captured data 60, and by matching and relating them, it associates the coordinates of the environmental map and the first captured data 60. Based on this association, the image viewing control unit 46 grasps the location information of unknown obstacles on the environmental map in the first captured data 60. The image viewing control unit 46 then corrects the first captured data 60 so that the image is centered on the point of focus, based on the location information of the unknown obstacle (i.e., the point of focus) in the first captured data 60, or controls the shooting direction of the first imaging device 15a so that the point of focus is at the center of the image of the first captured data 60 in order to capture an image centered on the point of focus with the first imaging unit 15.

[0110] Alternatively, a 360° camera may be used as the first imaging device 15a to capture first image data 60 by centering the field of view on the direction in which an unknown obstacle was detected. The image viewing control unit 46 may then display this first image data 60 in the image display area 73. In this case, since distortion occurs around the edges of the image, a conceptual image with distortion removed and the focus on the point of focus may be displayed as a corrected image for easier understanding by the user.

[0111] For example, as shown in Figure 18, the image viewing control unit 46 may display in the image display area 73 the first image data 60a, which is a normal image taken of the area in front of the autonomous driving work device 2 by the first image capture unit 15, and the first image data 60b, which has been corrected so that an unknown obstacle is at the center of the image as the point of focus.

[0112] Furthermore, when the autonomous driving work device 2 is driving around an unknown obstacle, the image viewing control unit 46 may display in the image display area 73, as shown in Figure 19, first image data 60 taken by the first imaging unit 15 in the normal manner in front of the autonomous driving work device 2, and second image data 61 taken by the second imaging unit 16 with the unknown obstacle as the point of focus and centered in the image.

[0113] In addition, the image viewing control unit 46 basically displays the first shooting data 60 and the second shooting data 61 on the remote operation screen 70. However, if the driving work information processing system 1 is in cooperation with a shooting unit such as a surveillance camera (not shown) that monitors the cleaning area (work area), the image viewing control unit 46 may acquire images taken by the surveillance camera as third shooting data from the surveillance camera and display them on the remote operation screen 70. For example, when shooting data taken with an unknown obstacle as the point of focus is to be displayed exclusively in the image display area 73, the image viewing control unit 46 may acquire images taken by the surveillance camera of the area including the unknown obstacle from the surveillance camera and display them in the image display area 73.

[0114] For example, as shown in Figure 20, the image viewing control unit 46 may display in the image display area 73 first image data 60, which is obtained by the first imaging unit 15 by normally capturing the area in front of the autonomous driving work device 2, and monitoring data 63 (third image data), which is obtained by the monitoring camera by capturing an area containing an unknown obstacle.

[0115] When the remote control command unit 47 receives an error notification from the autonomous driving work device 2, it sends a remote control request to the operation terminal 4 via the server-side communication unit 42. When the remote control command unit 47 receives a remote control request from the operation terminal 4 via the server-side communication unit 42, it generates a remote control command corresponding to that remote control request.

[0116] For example, when the remote control command unit 47 receives an error notification from the autonomous driving work device 2 that has detected an unknown obstacle and is unable to move due to the obstacle, it sends a remote control request to the operation terminal 4 via the server-side communication unit 42. Also, when the remote control command unit 47 receives a remote control from the operation terminal 4 via the server-side communication unit 42, it generates a remote control command corresponding to that remote control. Specifically, if the autonomous driving work device 2 detects an obstacle in the direction of travel based on the first image data 60 captured by the first image unit 15 and the measurement data measured by the measurement unit 13, and the remote control is an instruction to avoid the obstacle, the remote control command unit 47 generates a remote control command instructing the autonomous driving work device 2 to avoid the obstacle by driving within a predetermined safe distance in the automated driving cleaning.

[0117] Furthermore, if the remote control is a driving instruction such as moving forward, stopping, turning right, or turning left, the remote control command unit 47 generates a remote control command for the driving instruction. If the remote control is a driving setting instruction such as the driving speed of the driving unit 11, the remote control command unit 47 generates a remote control command for the driving setting instruction. If the remote control is a work instruction such as executing / stopping cleaning work of the cleaning unit 12, the remote control command unit 47 generates a remote control command for the work instruction. If the remote control is an individual setting instruction such as operating / stopping cleaning of the cleaning member 24 and pad pressure, operating / stopping cleaning fluid supply of the cleaning fluid supply unit 25 and the amount of water supplied, or operating / stopping suction of the suction unit 26 and the amount of suction, the remote control command unit 47 generates a remote control command for the individual setting instruction.

[0118] If the remote control is a shooting operation instruction for the first shooting unit 15 or the second shooting unit 16, the remote control command unit 47 generates a remote control command for that shooting operation instruction. For example, the remote control command unit 47 receives a shooting operation instruction indicating the operation of the shooting direction or zoom in or zoom out for the first shooting unit 15 or the second shooting unit 16 and generates a remote control command. The remote control command unit 47 may receive a shooting operation instruction and generate a remote control command only when the image display area 73 of the remote control screen 70 is displaying only the first shooting data 60 or only the second shooting data 61, based on each user's image viewing settings.

[0119] Furthermore, the remote control command unit 47 is not limited to the remote control described above, but may also accept various remote control requests for each part of the autonomous driving work device 2 and generate remote control commands for controlling each part.

[0120] Furthermore, when the autonomous driving work device 2 has finished remotely controlled driving, the remote control command unit 47 receives a notification of the end of remotely controlled driving from the autonomous driving work device 2 via the server-side communication unit 42, and then transmits the notification of the end of remotely controlled driving to the operation terminal 4 via the server-side communication unit 42.

[0121] Next, the operation terminal 4 will be described. The operation terminal 4 consists of a personal computer or tablet terminal owned by a user such as an operator, administrator, or security guard who uses the autonomous driving work device 2. As shown in Figure 6, the operation terminal 4 comprises a terminal-side control unit 50, a terminal-side storage unit 51, a terminal-side communication unit 52, and a display unit 53 consisting of a touch panel or the like.

[0122] The terminal-side control unit 50 provides overall control over all parts and functions of the operating terminal 4. It is composed of a computer such as a CPU and is connected to the terminal-side storage unit 51, which includes ROM, RAM, hard disk, and flash memory, as well as the terminal-side communication unit 52 and the display unit 53.

[0123] Furthermore, the terminal-side control unit 50 is connected to external devices via the terminal-side communication unit 52, enabling communication with external devices. The terminal-side communication unit 52 communicates wirelessly with external devices such as the autonomous driving work device 2 and the management server 3 using a wireless LAN communication standard such as Wi-Fi. For example, the terminal-side communication unit 52 sends a selection instruction for the autonomous driving work device 2 to the management server 3 in order to link it with the autonomous driving work device 2 that is to be remotely controlled. The terminal-side communication unit 52 accesses a website managed by the management server 3 to obtain a remote control screen 70 for operating the autonomous driving work device 2 linked to the operation terminal 4, and also sends remote control using the remote control screen 70 to the management server 3.

[0124] The terminal-side storage unit 51 stores programs and data for controlling each part and various functions of the operating terminal 4, and the terminal-side control unit 50 performs calculation processing based on the programs and data stored in the terminal-side storage unit 51 to comprehensively control each part and various functions. For example, the terminal-side control unit 50 operates as a login processing unit 55, a device selection unit 56, a display control unit 57, and an operation input unit 58 by executing a program stored in the terminal-side storage unit 51.

[0125] The login processing unit 55 accepts login requests to the management server 3 from operators or administrators using the operation terminal 4. As shown in Figure 21, the login processing unit 55 displays a login screen 65 on the display unit 53 for performing the login process, and accepts input of personal identification information (personal code) and passwords from each user, such as operators or administrators, on the login screen 65. The login screen 65 may consist of a screen created on a website managed by the management server 3. Personal identification information and passwords are managed by the management server 3. The login processing unit 55 transmits the login request, including the entered personal identification information and password, to the management server 3 via the terminal-side communication unit 52.

[0126] When the management server 3 authenticates the personal identification information and password based on the login request and sends a successful login notification to the operation terminal 4, the device selection unit 56 displays a device selection screen (not shown) on the display unit 53 for selecting the autonomous driving work device 2 to be remotely controlled, and accepts the selection of the autonomous driving work device 2 on the device selection screen. The device selection screen may consist of a screen created on a website managed by the management server 3. The device selection unit 56 sends the selection instruction for the selected autonomous driving work device 2 to the management server 3 via the terminal-side communication unit 52. As a result, the management server 3 links the autonomous driving work device 2 and the operation terminal 4.

[0127] When the operating terminal 4 logs into the management server 3 and selects the autonomous mobile work device 2, and when the autonomous mobile work device 2 linked to the operating terminal 4 performs automatic mobile cleaning, the display control unit 57 acquires a remote control screen 70 for remotely controlling the autonomous mobile work device 2 via the terminal-side communication unit 52 and displays it on the display unit 53, as shown in Figures 12 to 20. As described above, the remote control screen 70 displays the first image data 60 captured by the first image unit 15 and the second image data 61 captured by the second image unit 16 of the autonomous mobile work device 2 selected by the operating terminal 4, based on the user's viewing settings on the operating terminal 4.

[0128] Furthermore, when the autonomous driving work device 2 detects an unknown obstacle, the display control unit 57 receives a remote operation request from the management server 3 via the terminal-side communication unit 52, and displays a notification recommending remote operation of the autonomous driving work device 2 on the remote operation screen 70. In addition, when the autonomous driving work device 2 finishes remote operation, the display control unit 57 receives a notification of completion of remote operation from the management server 3 via the terminal-side communication unit 52, and displays the notification of completion of remote operation on the remote operation screen 70.

[0129] The operation input unit 58 accepts input operations from various screens, such as the remote control screen 70, and transmits the remote operations entered using the remote control screen 70 to the management server 3 via the terminal-side communication unit 52. For example, if the autonomous driving work device 2 performing automatic driving cleaning detects an error such as being unable to drive or unable to work and sends an error notification, and the management server 3 sends a remote operation request to the operation terminal 4, the operation input unit 58 makes it possible to accept remote operations on the remote control screen 70 to avoid the error.

[0130] The remote operations that the operation input unit 58 receives on the remote operation screen 70 can be operations of the driving unit 11 or cleaning unit 12 in the driving operation section or cleaning operation section of the remote operation screen 70. This allows the operation input unit 58 to receive remote operations such as avoiding unknown obstacles or re-executing cleaning. The operation input unit 58 instructs the autonomous driving work device 2 via the management server 3 to avoid errors by driving while avoiding unknown obstacles or by re-executing cleaning for residual water on the floor surface FL or remaining dirt on the floor surface FL caused by a wastewater suction malfunction. In addition, remote operations can be operations of the first shooting unit 15 or second shooting unit 16 in the camera operation section 74 of the remote operation screen 70. This allows the operation input unit 58 to receive remote operations to take pictures centered on the point of focus that is causing the error. This remote operation is instructed to the autonomous driving work device 2 via the management server 3, and the captured data centered on the point of focus is acquired and displayed in detail in the image display area 73.

[0131] Next, an example of how the remote control screen 70 of the autonomous driving work device 2 is displayed on the operation terminal 4 in the driving work information processing system 1 will be explained with reference to the flowchart in Figure 22.

[0132] First, in the driving work information processing system 1, in order to perform automated driving cleaning of the cleaning area, a worker such as a cleaning company places the autonomous driving work device 2 at the cleaning start position (start position) of the cleaning area (step S1).

[0133] The operation terminal 4 is operated by the operator of the autonomous driving work device 2 to access the management server 3 and display the login screen 65 (step S2). When the operator enters personal identification information and a password on the login screen 65 of the operation terminal 4 and completes the login process (step S3), the entered login information is sent to the management server 3, and the management server 3 accepts the login from the operation terminal 4 by authenticating the personal identification information and password (step S4).

[0134] When the operating terminal 4 successfully logs in to the management server 3, it accesses the management server 3 and displays a device selection screen (not shown). At this time, the management server 3 switches from the login screen 65 to the device selection screen and presents it to the operating terminal 4 (allowing it to be viewed). When the operating terminal 4 selects the autonomous mobile work device 2 on the device selection screen (step S5), a request to select the autonomous mobile work device 2 is sent to the management server 3, and the management server 3 manages the operating terminal 4 and the autonomous mobile work device 2 in association (step S6).

[0135] When the operating terminal 4 receives a success notification for a selection request from the management server 3, it sends a screen request regarding the remote control screen 70 to the management server 3 in order to remotely control the autonomous driving work device 2 (step S7). In response to such a screen request, the management server 3 sends a data request regarding various data, such as the image data required to create the remote control screen 70, to the autonomous driving work device 2 linked to the operating terminal 4 (step S8). Note that the operating terminal 4 and the autonomous driving work device 2 may be linked in advance before the operation.

[0136] In response to such data requests, the autonomous driving work device 2 acquires various data, such as the first image data 60 captured by the first imaging unit 15 and the second image data 61 captured by the second imaging unit 16, and transmits them to the management server 3 (step S9). Based on the various data such as the first image data 60 and the second image data 61 received from the autonomous driving work device 2 and the viewing settings of the user logged in on the operation terminal 4, the management server 3 creates a remote operation screen 70 (step S10) and makes it accessible from the operation terminal 4 corresponding to that user.

[0137] At this time, in response to the screen request, the management server 3 switches from the device selection screen to the remote control screen 70 and presents it to the operation terminal 4 (allows viewing and display) (step S11). The management server 3 may also present the remote control screen 70 of the autonomous driving work device 2 to the operation terminal 4 while the operation terminal 4 has selected the autonomous driving work device 2.

[0138] Next, an example of the operation of the autonomous driving work device 2 in the driving work information processing system 1 for automatic driving cleaning will be explained with reference to the flowcharts in Figures 23 to 25. Figure 23 shows the operation of the autonomous driving work device 2, Figure 24 shows the operation of the management server 3, and Figure 25 shows the operation of the operation terminal 4.

[0139] First, when the autonomous mobile work device 2 starts automatic mobile cleaning (step S21 in Figure 23), it acquires various data such as the first image data 60 captured by the first image unit 15 and the second image data 61 captured by the second image unit 16 while performing automatic mobile cleaning, and transmits them to the management server 3 (step S22 in Figure 23).

[0140] In this example, the process proceeds to step A in Figure 24(a), where the management server 3 receives the first image data 60 and the second image data 61 from the autonomous mobile work device 2 (step S31 in Figure 24(a)). Based on the first image data 60 and the second image data 61, and the user's viewing settings on the operation terminal 4 associated with the autonomous mobile work device 2, the management server 3 creates a remote control screen 70 corresponding to the operation terminal 4 (step S32 in Figure 24(a)). The remote control screen 70 displays the first image data 60, which shows the driving status of the autonomous mobile work device 2, and the second image data 61, which shows the cleaning status of the autonomous mobile work device 2, based on the user's viewing settings. The management server 3 makes the created remote control screen 70 accessible to the corresponding operation terminal 4 for viewing (step S33 in Figure 24(a)).

[0141] At this point, in the example operation, the process moves to step B in Figure 25, and the operation terminal 4 accesses the management server 3 to view and display the remote control screen 70 (step S41 in Figure 25). As a result, while the autonomous mobile work device 2 is performing automatic mobile cleaning, the operation terminal 4 can check the first captured data 60, which captures the driving status of the autonomous mobile work device 2, and the second captured data 61, which captures the cleaning status of the autonomous mobile work device 2, on the remote control screen 70.

[0142] Incidentally, if the autonomous driving work device 2 does not detect an error that would cause it to become unable to move or perform work (for example, an unknown obstacle not present on the environmental map) during the execution of automatic driving cleaning (step S23 in Figure 23: No), it will continue automatic driving cleaning. When it finishes automatic driving cleaning (step S24 in Figure 23: Yes), it will terminate the operation of the driving work information processing system 1. On the other hand, if the autonomous driving work device 2 detects an error that would cause it to become unable to move or perform work during the execution of automatic driving cleaning (step S23 in Figure 23: Yes), it will send an error notification to the management server 3 (step S25 in Figure 23), stop automatic driving cleaning, and wait (step S26 in Figure 23).

[0143] In this example, the process proceeds to step C in Figure 24(b), where the management server 3 receives an error notification from the autonomous driving work device 2 (step S34 in Figure 24(b)) and sends a remote operation request to the operation terminal 4 (step S35 in Figure 24(b)). The management server 3 also changes the image data displayed on the remote operation screen 70 based on the user type of the user on the operation terminal 4, in response to the error notification. For example, if the user type is an administrator or security guard, the management server 3 does not change the image data, but if the user type is an operator, it displays both the first image data 60 and the second image data 61 in parallel.

[0144] In the example operation, the process moves to step D in Figure 25, where the operation terminal 4 receives a remote operation request from the management server 3 (step S42 in Figure 25), and displays a notification recommending remote operation of the autonomous driving work device 2 (step S43 in Figure 25).

[0145] On the operating terminal 4, when an permission response is given to the remote operation recommendation notification on the remote operation screen 70 (step S44: Yes in Figure 25), the remote operation is sent to the management server 3 (step S45 in Figure 25).

[0146] In this example, the process proceeds to step E in Figure 24(c), where the management server 3 receives a remote operation from the operation terminal 4 (step S36 in Figure 24(c)) and transmits a remote operation command corresponding to that remote operation to the autonomous driving work device 2 (step S37 in Figure 24(c)).

[0147] In the example operation, the process moves to step F in Figure 23, where the autonomous driving work device 2 stops automatic driving cleaning and is in standby mode. Upon receiving a remote operation command from the management server 3 (step S27 in Figure 23: Yes), it operates in accordance with the manual's remote operation instructions to avoid errors, for example, by driving while avoiding obstacles (step S28 in Figure 23).

[0148] When remote control by the operation terminal 4 is terminated, the autonomous mobile work device 2 will cease remotely controlled operation. Subsequently, when a restart operation is performed to resume automatic mobile cleaning, the operation terminal 4 sends a restart command to the management server 3, and the management server 3 then sends such a restart command to the autonomous mobile work device 2. In response to the restart command, the autonomous mobile work device 2 resumes automatic mobile cleaning and proceeds to step S22 in Figure 23 in the example operation.

[0149] As described above, according to this embodiment, the driving work information processing system 1 comprises an autonomous driving work device 2 capable of performing autonomous driving cleaning (autonomous driving work) that drives autonomously and performs (cleaning) work automatically, multiple operation terminals 4 for multiple users to operate the autonomous driving work device 2, and a management server 3 for managing the autonomous driving work device 2. The autonomous driving work device 2 is equipped with an imaging unit such as a first imaging unit 15 and a second imaging unit 16, and transmits imaging data such as first imaging data 60 and second imaging data 61 captured by the imaging unit to the management server 3. Each of the multiple operation terminals 4 sends a selection request to the management server 3 to select the autonomous driving work device 2 to be remotely operated when logging in to the management server 3. The management server 3 receives selection requests for one autonomous driving work device 2 from two or more logged-in operation terminals 4 and displays the imaging data transmitted from one autonomous driving work device 2 on two or more operation terminals 4.

[0150] With this configuration, the driving work information processing system 1 allows multiple users to view the captured data from a single autonomous driving work device 2 on their respective operating terminals 4. Therefore, since multiple users can share the captured data, the autonomous driving work device 2 can be effectively utilized by taking advantage of its remote control function.

[0151] Furthermore, in this embodiment, the driving operation information processing system 1 stores viewing settings for each user regarding the viewing of captured data, and displays the captured data on the operation terminal 4 corresponding to a user according to the viewing settings corresponding to that user.

[0152] With this configuration, the driving operation information processing system 1 allows the management server 3 to display appropriate captured data on the operation terminal 4 according to each user's role, even when the roles of users utilizing the autonomous driving operation device 2 differ for each user, such as operators, administrators, and security guards. For example, by displaying the first captured data 60, which captures the driving situation, to the operator in charge of driving, and the second captured data 61, which captures the cleaning situation, to the operator in charge of cleaning, it is possible to accurately determine whether or not there is a problem with the automated driving cleaning.

[0153] Furthermore, in this embodiment, the driving work information processing system 1 acquires, as photographic data, first photographic data 60, which is a photograph of the driving status of the autonomous driving work device 2, and second photographic data 61, which is a photograph of the work status of the autonomous driving work device 2, from the first photographic unit 15 and the second photographic unit 16, respectively. When the autonomous driving work device 2 detects an error related to automated driving work, it sends an error notification to the management server 3. If the management server 3 has not received an error notification from the autonomous driving work device 2, it displays either the first photographic data 60 or the second photographic data 61 on the operation terminal 4 for viewing. If the management server 3 has received an error notification from the autonomous driving work device 2, it displays the first photographic data 60 and the second photographic data 61 on the operation terminal 4 in parallel for viewing.

[0154] Specifically, the autonomous mobile work device 2 is equipped with a cleaning unit 12 that performs cleaning work on the floor surface FL, and the second imaging unit 16 photographs the cleaning work status of the cleaning unit 12 and acquires second imaging data 61. When the autonomous mobile work device 2 detects an error related to the cleaning work of the cleaning unit 12, it sends an error notification to the management server 3. When the management server 3 receives an error notification from the autonomous mobile work device 2, it displays the second imaging data 61, which photographs the cleaning work status, on one of the operation terminals 4 and accepts remote operation of the autonomous mobile work device 2 from one of the operation terminals 4.

[0155] Alternatively, the first imaging unit 15 photographs the driving status of the autonomous driving work device 2 and acquires first imaging data 60. When the autonomous driving work device 2 detects an error related to its driving status, it sends an error notification to the management server 3. When the management server 3 receives an error notification from the autonomous driving work device 2, it displays the first imaging data 60, which captures the driving status, on one of the operation terminals 4 and accepts remote control of the autonomous driving work device 2 from the operation terminal 4.

[0156] With this configuration, the driving operation information processing system 1 can simultaneously check the first photographic data 60, which captures the driving situation, and the second photographic data 61, which captures the cleaning situation. This allows for accurate and easy operation of the cleaning work while remotely controlling the vehicle to avoid errors, thus enabling accurate and easy error avoidance.

[0157] Next, the driving operation information processing system 1 of the second embodiment will be described. In the following description of the second embodiment, components similar to those in the driving operation information processing system 1 of the first embodiment will be denoted by the same reference numerals, and detailed explanations will be omitted.

[0158] In the second embodiment in particular, the driving work information processing system 1 is configured to determine the cleaning work status based on first photographic data 60 captured by the first photographic unit 15 of the autonomous driving work device 2 capturing the driving situation and second photographic data 61 captured by the second photographic unit 16 capturing the cleaning work status.

[0159] In the second embodiment, the server-side control unit 40 of the management server 3 operates as a work status determination unit 48 in addition to the device terminal linking unit 45, image viewing control unit 46, and remote operation command unit 47, as shown in Figure 26, by executing a program stored in the server-side storage unit 41. The work status determination unit 48 determines the cleaning work status based on first photographic data 60 that captures the driving situation and second photographic data 61 that captures the cleaning work situation.

[0160] When the autonomous mobile work device 2 is performing automatic mobile cleaning, the work status determination unit 48 receives the first image data 60 and the second image data 61 from the autonomous mobile work device 2 via the server-side communication unit 42, and determines the cleaning status of the floor surface FL (for example, the degree of dirt on the floor surface FL after cleaning, and the remaining cleaning solution and wastewater) based on the first image data 60 and the second image data 61.

[0161] Specifically, the work status determination unit 48 extracts floor surface images of the floor surface FL from the first captured data 60 and the second captured data 61, and compares them with a reference image showing a clean floor surface FL to determine whether the floor surface FL is dirty as a cleaning work status. For example, as shown in Figure 27, the work status determination unit 48 extracts floor surface images from the first captured data 60, divides the floor surface images into multiple regions, and compares the floor surface images of each region with the reference image to determine whether each region of the floor surface FL is dirty as a cleaning work status. The work status determination unit 48 can similarly determine whether each region of the floor surface FL is dirty using the second captured data 61. Reference images can be obtained by the management server 3 when the autonomous driving work device 2 cleans each cleaning area for the first time, for example, when performing learning-type cleaning or when the administrator determines that the floor surface FL is clean. This can be done by the management server 3 extracting floor surface images from the first image data 60 captured by the first image unit 15 and the second image data 61 captured by the second image unit 16, and storing them in the server-side storage unit 41 of the management server 3.

[0162] Furthermore, the work status determination unit 48 may extract a posture image of the cleaning unit 12 from the second image data 61 and compare it with a reference image showing the posture of the cleaning unit 12 in its normal state to determine whether the cleaning unit 12 is operating normally as a cleaning operation. For example, as shown in Figures 28 and 29, the work status determination unit 48 obtains a posture image of the cleaning unit 12 (squeegee 27) taken from above using the second image data 61, and compares the posture image with a reference image to determine whether the cleaning unit 12 (squeegee 27) is operating normally as a cleaning operation. The squeegee 27 of the cleaning unit 12 is rotatable left and right and is normally maintained in the normal position, but in the event of an abnormality such as foreign matter getting caught or movable parts sticking, it may become fixed in a position rotated from the normal position. Figure 28 shows an example where the squeegee 27 of the cleaning unit 12 is maintained in the normal position and is properly collecting cleaning fluid and wastewater, thus operating normally. Figure 29 shows an example where the squeegee 27 of the cleaning unit 12 is rotated from its normal position, and therefore cannot collect all the cleaning fluid and wastewater, resulting in abnormal operation. The reference image should be obtained by the management server 3 when the autonomous mobile work device 2 is stopped before starting cleaning. The management server 3 should acquire the posture image from the second image data 61 captured by the second image capture unit 16 and store it in the server-side storage unit 41 of the management server 3.

[0163] Alternatively, the work status determination unit 48 may determine the cleaning status of the cleaning unit 12 based on the analysis results of the second captured data 61, the measurement results of the vibration meter installed in the cleaning unit 12 (squeegee 27), the measurement results of the rotation sensor of the cleaning member motor of the cleaning member 24 and the current sensor of the cleaning member motor for controlling the appropriate pressing pressure of the cleaning member 24 on the floor surface FL, the detection results of the position sensor of the cleaning member actuator of the cleaning member 24, the detection results of the rotation sensor of the squeegee 27, etc. For example, if the work status determination unit 48 detects an abnormality such as vibration of the cleaning unit 12 (squeegee 27) when the measured value of the vibration meter is above a predetermined abnormal value, it may determine that the cleaning unit 12 (squeegee 27) is operating abnormally as the cleaning status. Furthermore, the image viewing control unit 46 may display the analysis results, measurement results, and detection results for evaluating such abnormal operation of the cleaning unit 12 on the remote operation screen 70 and make them viewable on the operation terminal 4.

[0164] Furthermore, the work status determination unit 48 may, for example, acquire photographic data from the autonomous mobile work device 2, taken by a camera or other imaging unit mounted on the side of the cleaning unit 12 (squeegee 27) (for example, at the side tip), as shown in Figures 30 and 31, and obtain a posture image of the cleaning unit 12 from the photographic data. By comparing the posture image with a reference image, the unit may determine whether the cleaning unit 12 (squeegee 27) is operating normally as part of the cleaning work status. This photographic data may be stored in the server-side storage unit 41 as detailed cleaning photographic data (third photographic data), similar to other photographic data such as the first photographic data 60 and the second photographic data 61. Figure 30 shows an example where the squeegee 27 of the cleaning unit 12 is maintained in the normal position and is properly collecting cleaning fluid and wastewater, thus operating normally. Figure 31 shows an example where the squeegee 27 of the cleaning unit 12 is jammed with foreign matter and is lifted above the floor surface FL, so it is unable to collect all the cleaning fluid and wastewater and is not operating normally (it is operating abnormally).

[0165] Furthermore, the work status determination unit 48 may acquire from the autonomous mobile work device 2 machine image data (third image data) taken by a camera or other imaging unit mounted on the side of the device body 10, capturing the condition of the side of the device body 10 (such as water damage, dirtiness, and unevenness), and detailed cleaning image data (third image data) taken by capturing the cleaning status of details such as corners along the walls of the cleaning area. The work status determination unit 48 may then determine the cleaning status of the cleaning unit 12 based on this third image data capturing the cleaning status of these detailed cleaning areas.

[0166] Furthermore, the image viewing control unit 46 may display the above-mentioned aircraft body photography data and detailed cleaning photography data as third-party photography data on the remote control screen 70 for viewing on the operation terminal 4. This allows the user of the operation terminal 4 to check for damage to the main unit 10 and the cleaning status of the details of the cleaning area. Accordingly, the image viewing control unit 46 can display at least one of the above-mentioned third-party photography data on the remote control screen 70 for viewing on the operation terminal 4, including the first-party photography data 60 captured by the first-party photography unit 15 of the autonomous mobile work device 2 capturing the driving situation, the second-party photography data 61 captured by the second-party photography unit 16 capturing the cleaning work situation, and the third-party photography data. Similar to the first embodiment, the remote control screen 70 can accept remote control of the autonomous mobile work device 2.

[0167] This allows users of the autonomous mobile work device 2 to view captured data of various situations of the autonomous mobile work device 2 on the operation terminal 4. Furthermore, because the captured data is routed through the management server 3 for viewing and display, the load on the autonomous mobile work device 2 and the operation terminal 4 can be reduced. In addition to the driving and work status of the autonomous mobile work device 2, images from a surveillance camera that provides an overhead view of the area around the autonomous mobile work device 2, and images that allow detailed viewing of the cleaning unit 12 of the autonomous mobile work device 2 can be acquired as third-party captured data, and users can refer to this third-party captured data as needed. Furthermore, users can remotely operate the autonomous mobile work device 2 while viewing the first-party captured data 60, second-party captured data 61, and third-party captured data.

[0168] In this case, the image viewing settings in the remote control screen 70 include the first image data 60 captured by the first shooting unit 15 of the autonomous driving work device 2 capturing the driving situation, the second image data 61 captured by the second shooting unit 16 capturing the cleaning work situation, and the third image data mentioned above. For example, the display types in the image viewing settings include "Camera 1" which displays the first image data 60 alone, "Camera 2" which displays the second image data 61 alone, and "Camera 1 / Camera 2" which displays the first image data 60 and the second image data 61 in parallel, as well as "Camera 3" which displays the third image data alone, "Camera 1 / Camera 3" which displays the first image data 60 and the third image data in parallel, "Camera 2 / Camera 3" which displays the second image data 61 and the third image data in parallel, and "Camera 1 / Camera 2 / Camera 3" which displays the first image data 60, the second image data 61 and the third image data in parallel, and at least one display type is set as an option in each user's image viewing settings.

[0169] By using these viewing settings, when multiple shooting data such as the first shooting data 60, the second shooting data 61, and the third shooting data are available, the appropriate shooting data can be viewed and displayed on the operation terminal 4 according to each user's role.

[0170] Furthermore, similar to the first embodiment, the image viewing control unit 46 can, if it has not received an error notification from the autonomous driving work device 2, display one of the first, second, and third captured data on the remote control screen 70 for viewing on the operation terminal 4. On the other hand, if it has received an error notification from the autonomous driving work device 2, it can display at least two of the first, second, and third captured data on the remote control screen 70 in parallel for viewing on the operation terminal 4. In this case, it is advisable to pre-configure the display type for when no error notification is received and the display type for when an error notification is received in the viewing settings.

[0171] This allows users to simultaneously view the first, second, and third image data, which are displayed in parallel, in the event of an error related to automated driving operations. This enables users to gain a more detailed understanding of the error situation.

[0172] Furthermore, similar to the first embodiment, the image viewing control unit 46, when it has not received an error notification from the autonomous driving work device 2, displays one or more of the first, second, and third captured data in a larger size on the remote control screen 70 and displays them in parallel for viewing on the operation terminal 4. On the other hand, when it has received an error notification from the autonomous driving work device 2, it can also display different captured data from the one displayed in a larger size on the remote control screen 70 and displays them in parallel for viewing on the operation terminal 4. In this case, it is advisable to pre-set the display type when no error notification is received and the type of captured data to be displayed in a larger size in the viewing settings.

[0173] This allows users to gain a more detailed understanding of the error situation by viewing the large-screen image data when an error related to automated driving is detected.

[0174] By the way, when the autonomous mobile work device 2 is moving while turning, the squeegee 27 of the cleaning unit 12 is likely to fail to collect cleaning fluid and wastewater, resulting in residual water. Therefore, the work status determination unit 48 may acquire a series of posture images of the cleaning unit 12 (squeegee 27) based on a series of second image data 61 taken while the autonomous mobile work device 2 is moving while turning, and determine whether the cleaning unit 12 (squeegee 27) is operating normally as a cleaning work status by comparing the series of posture images with their respective reference images.

[0175] In other words, the work status determination unit 48 determines whether the cleaning unit 12 (squeegee 27) is operating normally during the turning maneuver of the autonomous mobile work device 2. In this case as well, the reference image is obtained when the autonomous mobile work device 2 first travels through the cleaning area, for example, when performing learning-type cleaning, and when the autonomous mobile work device 2 is turning, by the management server 3 extracting multiple posture images from a series of second image data 61 captured by the second imaging unit 16, and storing them in the server-side storage unit 41 of the management server 3. Figure 32 shows an example where the squeegee 27 of the cleaning unit 12 is operating normally during the turning maneuver of the autonomous mobile work device 2, and Figure 33 shows an example where the squeegee 27 of the cleaning unit 12 is not operating normally (is operating abnormally) during the turning maneuver of the autonomous mobile work device 2. In Figures 32 and 33, the imaging range in which the second imaging unit 16 captures the second image data 61 is indicated by a dashed line. Alternatively, the normal operation of the squeegee 27 can be determined by pre-defining the normal attitude range of the squeegee 27 corresponding to the turning state.

[0176] The work status determination unit 48 determines the cleaning work status and, if it detects abnormal operation of the cleaning unit 12 (squeegee 27), it sends an abnormal cleaning work status notification to the operation terminal 4 via the server-side communication unit 42.

[0177] In the second embodiment, the image viewing control unit 46 may display one second image data 61 of the cleaning unit 12 (squeegee 27) taken from above or the side on the remote control screen 70, as shown in Figures 28 to 31. Alternatively, it may display two or more second image data 61 of the cleaning unit 12 (squeegee 27) on the remote control screen 70 to specifically display the cleaning unit 12 (squeegee 27) as a point of focus. For example, as shown in Figure 34, the image viewing control unit 46 may display a second image data 61a of the cleaning unit 12 (squeegee 27) taken from above and a second image data 61b taken from the side side-by-side on the remote control screen 70. Furthermore, as shown in Figure 35, the image viewing control unit 46 may display the first captured data 60 of the driving situation on the remote control screen 70, and may also display the second captured data 61a of the cleaning unit 12 (squeegee 27) taken from above and the second captured data 61b taken from the side side side by side on the remote control screen 70.

[0178] Furthermore, parallel display including two or more second image data 61 may be performed according to the display type of the pre-set image viewing settings, but may also be performed regardless of the image viewing settings when the autonomous mobile work device 2 detects an error. For example, when the management server 3 receives an error notification from the autonomous mobile work device 2 due to an inability to perform work, the image viewing control unit 46 may set the cleaning unit 12 (squeegee 27) that is causing the inability to perform work as the point of focus and display the image data of the point of focus exclusively on the remote control screen 70.

[0179] In the second embodiment, when the operating terminal 4 receives a notification of an abnormality in the cleaning work status from the management server 3 via the terminal-side communication unit 52, it notifies the user of the abnormality in the cleaning work status. For example, the operating terminal 4 may notify the user of the abnormality in the cleaning work status by displaying text on the screen, sounding a warning, or lighting or flashing a warning lamp.

[0180] As described above, according to the second embodiment, in the driving work information processing system 1, the cleaning unit 12 is configured to clean the floor surface FL using a cleaning solution, the second imaging unit 16 photographs the cleaning work status of the cleaning unit 12 from above and acquires second imaging data 61, and the management server 3 determines the cleaning work status of the cleaning unit 12 based on the second imaging data 61 that photographs the cleaning work status and detects residual cleaning solution and wastewater on the floor surface FL.

[0181] Alternatively, in the driving operation information processing system 1, the cleaning unit 12 is configured to clean the floor surface FL using a cleaning solution, the second imaging unit 16 photographs the cleaning operation status of the cleaning unit 12 from the side and acquires second imaging data 61, and the management server 3 determines the cleaning operation status of the cleaning unit 12 based on the second imaging data 61 that photographs the cleaning operation status and detects the remaining cleaning solution and wastewater on the floor surface FL.

[0182] Alternatively, in the mobile operation information processing system 1, the cleaning unit 12 is configured to clean the floor surface FL using a cleaning solution, the second imaging unit 16 continuously photographs the cleaning operation status of the cleaning unit 12 from above while the autonomous mobile operation device 2 is turning, and acquires second imaging data 61, and the management server 3 determines the cleaning operation status of the cleaning unit 12 based on the second imaging data 61 that photographs the cleaning operation status and detects any remaining cleaning solution or wastewater on the floor surface FL.

[0183] With this configuration, the driving operation information processing system 1 can detect abnormal operation of the cleaning unit 12 by taking appropriate photographs of the cleaning operation status of the cleaning unit 12 and utilizing the second photographed data 61, thereby accurately determining errors related to the cleaning operation, such as residual cleaning solution or wastewater on the floor surface FL.

[0184] In the above-described embodiment, an example was explained in which the remote control screen 70 of each operation terminal 4 displays a login name 71 indicating only the user logged in on that operation terminal 4. However, the present invention is not limited to this example. In other embodiments, if multiple operation terminals 4 are associated with a single autonomous driving work device 2 in the management server 3, the image viewing control unit 46 may, as shown in Figure 36, display the login name 71 indicating each user of the multiple operation terminals 4 associated with the same autonomous driving work device 2 on the remote control screen 70 of each operation terminal 4. In this case, the image viewing control unit 46 may highlight the login name 71 of the user of that operation terminal 4 on the remote control screen 70 of each operation terminal 4.

[0185] Furthermore, in the above-described embodiment, an example was explained in which the autonomous mobile work device 2 acquires first image data 60 capturing the driving situation and second image data 61 capturing the cleaning work situation, but the present invention is not limited to this example. In other embodiments, in addition to the first image data 60 capturing the driving situation and the second image data 61 capturing the cleaning work situation, the autonomous mobile work device 2 may be configured to acquire security image data capturing a security area in order to secure the cleaning area on which the autonomous mobile work device 2 is driving as a security area. For example, the autonomous mobile work device 2 may be equipped with a security imaging device (not shown) for capturing security image data of the security area, or the first imaging unit 15 may be controlled so that the first imaging device 15a captures security image data at a security camera angle directed towards the security area.

[0186] In this case, the image viewing control unit 46 of the management server 3 creates a remote operation screen 70 for the operation terminal 4, whose user type in the personal registration information is a security guard, to display the security photography data in the image display area 73. As shown in Figure 36, the image viewing control unit 46 accepts a selection operation on the image of the security guard who has detected an anomaly on the security photography data displayed on the remote operation screen 70 of the operation terminal 4, and the operation terminal 4 transmits the location information of the selected operation location in the security photography data to the management server 3. In addition, the image viewing control unit 46 grasps the coordinate system in the environmental map based on the environmental map and the measurement results of the measurement unit 13, and grasps the coordinate system of the security photography data using visual SLAM, etc., and associates the feature points of the environmental map and the feature points of the security photography data by matching them, thereby associating each coordinate between the environmental map and the security photography data. Based on the location information of the selected operation location in the security photography data and the above-mentioned association, the image viewing control unit 46 obtains the location information of the selected operation location in the environmental map, grasps the location information of the selected operation location in the cleaning area based on this location information, and transmits it to the operation terminal 4. The control terminal 4 notifies the security guard of the location information of the selected operation area within the cleaning area, such as through text display on the screen.

[0187] Furthermore, in the above-described embodiment, the device terminal linking unit 45 of the management server 3 receives a request from the operation terminal 4 to select one autonomous driving work device 2 and links that one autonomous driving work device 2 with the operation terminal 4. However, the present invention is not limited to this example. In other embodiments, the device terminal linking unit 45 may receive requests from the operation terminal 4 to select multiple autonomous driving work devices 2 and link those multiple autonomous driving work devices 2 with the operation terminal 4. In this case, the device terminal linking unit 45 may accept requests to select multiple autonomous driving work devices 2 when the user type in the user's personal registration information of the user of the operation terminal 4 is a security guard or the like. In addition, the image viewing control unit 46 of the management server 3 may create a remote operation screen 70 to display the captured data of one or more of the multiple autonomous driving work devices 2 in the image display area 73 for the operation terminal 4 linked to the multiple autonomous driving work devices 2. Furthermore, it is preferable that one or more autonomous mobile work devices 2 displaying the captured data be selectable by the user according to their arbitrary operation using the control terminal 4. With this configuration, the user can refer to each of the captured data acquired by multiple autonomous mobile work devices 2 according to their selection, making it easy to grasp changes in the layout and the presence or absence of obstacles in each cleaning area of ​​the facility.

[0188] Furthermore, in the above-described embodiment, an example was explained in which the image viewing control unit 46 of the management server 3 creates a remote control screen 70 and presents it as a screen on a website, and the remote control screen 70 is displayed for viewing on the operation terminal 4 that has logged into the management server 3. However, the present invention is not limited to this example. In other embodiments, the image viewing control unit 46 may be configured to transmit shooting data such as first shooting data 60 and second shooting data 61, and each user's viewing settings to the operation terminal 4, and the operation terminal 4 may create and display the remote control screen 70 based on these.

[0189] Furthermore, the present invention can be modified as appropriate, as long as it does not contradict the gist or idea of ​​the invention as can be read from the claims and the specification as a whole, and a driving work information processing system and an autonomous driving work device with such modifications are also included in the technical concept of the present invention. [Industrial applicability]

[0190] The present invention can be suitably used in a mobile work information processing system equipped with an industrial (commercial) work robot, such as an automatic floor cleaning and scrubbing device or a security device that uses cameras for monitoring. This device is capable of autonomously driving and performing automatic work tasks, such as cleaning the floors of commercial facilities like shopping malls, or performing automatic work in work areas like factories and railway terminals. [Explanation of Symbols]

[0191] 1. Driving Operation Information Processing System 2. Autonomous Driving Work Device 3. Management Server 4. Operating terminal 11. Running section 12 Cleaning Department (Working Department) 13 Measurement Unit 15. First Photography Department 20 Device-side control unit 22 Device-side communication unit 34 Reproduction Control Unit 35 Remote Control Unit 40 Server-side control unit 42 Server-side communication unit 45 Inter-device terminal linking section 46 Image Viewing Control Unit 47 Remote Control Command Unit 48 Work Status Determination Unit 50 Terminal-side control unit 52 Terminal-side communication unit 53 Display section 57 Display Control Unit 58 Operation Input Section 60 First shooting data 61. Second shooting data 70 Remote control screen

Claims

1. In a driving work information processing system comprising an autonomous driving work device capable of autonomously driving and performing automated work, an operating terminal for a user to operate the autonomous driving work device, and a management server for managing the autonomous driving work device, The autonomous driving work device is equipped with a camera unit, and transmits the captured data from the camera unit to the management server. The aforementioned management server, as the image data, First photographic data capturing the driving status of the autonomous driving work device, The second set of photographic data captures the working status of the autonomous driving work device, Other third-party imaging data is acquired, In accordance with a predetermined determination, at least one of the first, second, and third shooting data is displayed on the operation terminal for viewing. When the autonomous driving device detects an error related to the automated driving operation, it sends an error notification to the management server. If the management server has not received the error notification from the autonomous driving work device, it will display one of the first, second, and third captured data on the operation terminal. If the management server has received the error notification from the autonomous driving work device, it will display at least two of the first, second, and third captured data on the operation terminal in parallel, depending on whether the error notification is related to the driving situation or the work situation. A driving operation information processing system characterized by the following:

2. The third type of captured data is an image taken by a surveillance camera that monitors the work area in which the autonomous mobile work device travels, or an image taken by a camera mounted on the side of the cleaning unit of the autonomous mobile work device. The driving operation information processing system according to feature 1.

3. If the management server has not received the error notification from the autonomous driving work device, it will display two or more of the first, second, and third captured data in a larger size and in parallel on the operation terminal for viewing. If the management server has received the error notification from the autonomous driving work device, it will display different captured data from the data that was displayed in a larger size before the error notification was received, in a larger size and in parallel on the operation terminal for viewing. The driving operation information processing system according to feature 1.

4. The management server stores viewing settings for the viewing of the captured data, and, according to these viewing settings, causes at least one of the first, second, and third captured data to be displayed for viewing on the operation terminal. The driving operation information processing system according to claim 1 or 2, characterized in that it is the same as described in claim 1 or 2.

5. The aforementioned operating terminal is equipped with multiple operating terminals, Each of the aforementioned operating terminals, when logging into the management server, sends a selection request to the management server to select the autonomous driving work device to be remotely controlled. The management server receives the selection request for one of the autonomous driving work devices from two or more logged-in operation terminals, and displays the captured data transmitted from one of the autonomous driving work devices on the two or more operation terminals. The driving operation information processing system according to claim 1 or 2, characterized in that it is the same as described in claim 1 or 2.

6. When the management server receives the error notification from the autonomous driving work device, it accepts remote operation of the autonomous driving work device by the operation terminal displaying the captured data. The driving operation information processing system according to claim 1 or 2, characterized in that it is the same as described in claim 1 or 2.

7. When the autonomous driving device detects an error related to the driving status of the autonomous driving device, it sends the error notification to the management server. When the management server receives the error notification regarding the driving status from the autonomous driving work device, it causes the first captured data of the driving status to be displayed on the operation terminal for viewing. The driving operation information processing system according to feature 6.

8. The autonomous mobile work device includes a cleaning unit that performs floor cleaning. The driving operation information processing system according to claim 1 or 2, characterized in that it is the same as described in claim 1 or 2.

9. The aforementioned imaging unit captures images including the cleaning work status behind the cleaning unit as part of the work status, and acquires the aforementioned imaging data. The management server determines the cleaning status of the cleaning unit based on the captured data of the cleaning work, and detects errors related to the cleaning work of the cleaning unit on the floor surface. The driving operation information processing system according to feature 8.

10. The cleaning unit is configured to clean the floor surface using a cleaning solution. The aforementioned imaging unit captures the cleaning work status of the cleaning unit from above as the work status and acquires the second imaging data. Based on the second set of photographic data capturing the cleaning process, the management server determines the cleaning status of the cleaning unit and detects any remaining water on the floor surface. The driving operation information processing system according to feature 8.

11. The cleaning unit is configured to clean the floor surface using a cleaning solution. The aforementioned imaging unit captures detailed images of the cleaning work in the cleaning unit to acquire the third imaging data. The management server determines the cleaning status of the cleaning unit based on the third image data, which captures the cleaning work in progress. The driving operation information processing system according to feature 8.

12. The cleaning unit is configured to clean the floor surface using a cleaning solution. The aforementioned imaging unit, while the autonomous mobile work device is rotating, photographs the cleaning work status of the cleaning unit and acquires the photographic data. The management server determines the cleaning status of the cleaning unit based on the captured data of the cleaning work, and detects any remaining water on the floor surface. The driving operation information processing system according to feature 8.

13. The imaging unit is A first imaging unit that acquires the first imaging data, A second imaging unit that acquires the second imaging data, The driving operation information processing system according to claim 1 or 2, characterized by comprising the above.

Images