Driving operation information processing system and driving operation information processing method

Abstract

To enable responses to obstacles to be easily determined, and safely remote control an autonomously traveling work device.SOLUTION: An information processing system 1 comprises an autonomously traveling work device 2 which is capable of executing automatically traveling cleaning, and an operation terminal 4 for operating the autonomously traveling work device 2. The autonomously traveling work device 2 includes a travel unit 11, a first imaging unit 15, and a measurement unit 13. The operation terminal 4 includes a display unit 53. In the autonomously traveling work device 2, the first imaging unit 15 captures an image of the front of the autonomously traveling work device 2 and acquires first captured data 60, and the measurement unit 13 measures the distance from the autonomously traveling work device 2 to a non-work object, thereby measuring a first distance from the autonomously traveling work device 2 to an obstacle and a second distance from a wall to the obstacle so as to acquire measured data. In the operation terminal 4, the display unit 53 displays the image of display data that is generated associating the obstacle in the image of the first captured data 60 with the written distance of the measured data.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present invention relates to a travel work information processing system and a travel work information processing method including an autonomous travel work device capable of executing an autonomous travel work that autonomously travels according to a pre-input program and automatically performs work.

Background Art

[0002] Conventionally, an autonomous travel work device creates or sets and stores an environmental map, a travel route, travel settings, and work settings (cleaning settings) according to a pre-input program, and autonomously travels and automatically performs work (cleaning) based on the stored results. It has a function capable of executing an autonomous travel work (autonomous travel cleaning). For example, the autonomous travel work device is applied to an industrial (business use) autonomous travel work device (autonomous travel cleaning device) that performs cleaning work on the floor surface of a work area such as a commercial facility, an office, a hotel, or a hospital.

[0003] In addition, the autonomous travel work device may perform work during the usage time when a third party uses or travels through the work area. Therefore, there is an autonomous travel work device that determines whether to avoid an obstacle and autonomously travel when encountering an obstacle that does not exist in the pre-stored environmental map or the like in the work area. At this time, the autonomous travel work device may determine that it cannot avoid the obstacle and stop autonomous travel. Even in such a case, it is required to continue the work without being affected by the obstacle, such as avoiding the obstacle and traveling according to a manual operation by the user (operator).

[0004] For example, the autonomous traveling body of Patent Document 1 includes a travel command generation unit that generates a travel command to decelerate or stop when an obstacle exists in the traveling direction in the reproduction travel mode, and a notification unit that notifies that it is approaching an obstacle when the distance between the main body and the obstacle is less than or equal to a predetermined value in the teaching travel mode.

Prior Art Documents

Patent Documents

[0005] [Patent Document 1] Japanese Patent Publication No. 2014-219723 [Overview of the Initiative] [Problems that the invention aims to solve]

[0006] Conventional autonomous work devices issue commands to slow down or stop if an obstacle is present in the direction of travel, or notify the user that they are approaching an obstacle if the distance between the device and the obstacle is below a predetermined value. However, the work area of ​​an autonomous work device may contain not only obstacles but also non-work targets such as walls and steps, and when remotely controlling an autonomous work device, it is not easy for the user (operator) to determine whether the autonomous work device can avoid obstacles, walls, steps, etc.

[0007] The present invention has been made in view of the above-mentioned problems, and the object of the present invention is to provide a driving work information processing system and a driving work information processing method that can easily determine how to deal with obstacles in the work area and can safely remotely operate an autonomous driving work device. [Means for solving the problem]

[0008] To solve the above problems, the first driving operation information processing system of the present invention is: For cleaning work It drives autonomously and automatically The aforementioned cleaning work A driving work information processing system comprising an autonomous driving work device capable of performing the following, and an operating terminal for operating the autonomous driving work device, wherein the autonomous driving work device comprises a driving unit, a shooting unit, a measurement unit, and a device-side communication unit, and the operating terminal comprises a terminal-side communication unit and a display unit, and in the autonomous driving work device, the shooting unit takes a picture of the front of the autonomous driving work device and acquires the captured data, and the measurement unit takes a picture of the autonomous driving work device It does not exist in the environment map generated in advance through training runs. Up to the unknown obstacle 1 Distance and, Present in the aforementioned environmental map From known non-work targets, the above unknown Up to the obstacle 2ndThe distance is measured and measurement data is acquired, and the device-side communication unit, photograph The data and the measurement data are transmitted to the operation terminal, and the terminal-side communication unit in the operation terminal is the photograph The display unit receives the data and the measurement data, and in the captured data, 、 The aforementioned unknown Obstacles The first distance notation indicating the first distance is superimposed, and the second distance notation indicating the second distance is superimposed on the known non-working object, and simultaneously It is characterized by displaying information.

[0009] According to the first driving work information processing system of the present invention, while an autonomous driving work device is performing automatic driving cleaning, information that does not exist in the pre-stored environmental map, etc., is processed. unknown Even when encountering an obstacle, the autonomous driving work device can check the displayed data image on the control terminal that remotely operates the device. unknown It is possible to easily determine whether or not to avoid an obstacle. At this time, by checking the image of the displayed data on the operating terminal, the autonomous driving work device can make a decision. unknown Up to the obstacle 1 Distance and known non-work targets Unknown obstacles up to 2nd Distance and other factors can be easily determined by the operator. unknown It is possible to improve the recognition of the surrounding conditions, including obstacles. Therefore, according to the driving work information processing system, in the work area unknown It allows for easy determination of how to respond to obstacles and enables safe remote operation of autonomous work devices.

[0010] To solve the above problems, the second driving work information processing system of the present invention further comprises a management server for managing the autonomous driving work device, the management server comprises a server-side control unit and a server-side communication unit, the autonomous driving work device further comprises a device-side control unit, in the autonomous driving work device, the device-side communication unit transmits the image data and the measurement data to the management server, in the management server, the server-side communication unit receives the image data and the measurement data transmitted from the autonomous driving work device, and the server-side control unit displays the image of the image data at a display position related to the obstacle 1 The system generates display data by overlaying distance notation, the server-side communication unit transmits the display data to the operating terminal, the terminal-side communication unit receives the display data transmitted from the management server, and the display unit displays an image of the display data.

[0011] According to the second driving work information processing system of the present invention, an autonomous driving work device that performs automatic driving cleaning and an operating terminal that remotely controls the autonomous driving work device can be managed by linking them together via a management server. When the autonomous driving work device detects an unknown obstacle, the management server will provide information to the operating terminal linked to the autonomous driving work device. unknown It is possible to transmit and display data related to obstacles. Furthermore, by performing high-load image processing such as the creation of display data on the management server, there is no concern about a decrease in the control processing speed of the autonomous driving work device or operating terminal.

[0012] To solve the above problems, in the third driving operation information processing system of the present invention, the operation terminal further comprises an operation input unit, the management server further comprises a remote operation command unit, and in the operation terminal, the operation input unit on the image of the display data displayed on the display unit, unknown Obstacles and / or The aforementioned The system accepts a selection operation at a display position related to a known non-working object, and the terminal-side communication unit transmits a remote operation corresponding to the display position and the selection operation to the management server in response to the selection operation. The management server's server-side communication unit receives the remote operation transmitted from the operation terminal, and the remote operation command unit generates a remote operation command based on the remote operation and the direction of travel of the autonomous driving work device. The server-side communication unit transmits the remote operation command to the autonomous driving work device. The autonomous driving work device's device-side communication unit receives the remote operation command transmitted from the management server, and the device-side control unit controls the driving unit to drive in accordance with the remote operation command.

[0013] According to the third traveling work information processing system of the present invention, even when the autonomous traveling work device detects an unknown obstacle, in the display data displayed on the operation terminal unknown By performing a selection operation at the display position related to the obstacle or the known non-working target, the unknown Remote operation for avoiding the obstacle and traveling can be performed on the autonomous traveling work device. Thereby, even when remotely operating the autonomous traveling work device, it is easy to unknown Judge the response to the obstacle, and the convenience of the operator can be improved. Also, unknown There may be a case where it is recognized that it is impossible to avoid the obstacle, and the response to the obstacle, including the content of the remote operation, such as detouring to another route or arranging for on-site staff unknown Can be easily judged.

[0014] In order to solve the above problems, the fourth traveling work information processing system of the present invention is such that in the management server, the server-side control unit, based on the measurement result from the autonomous traveling work device to the non-working target by the measurement unit and the photographed data photographed by the photographing unit, when detecting a door, issues a deceleration command to a predetermined speed around the door, and the server-side communication unit transmits the deceleration command to the autonomous traveling work device. In the autonomous traveling work device, the device-side communication unit receives the deceleration command transmitted from the management server, and the device-side control unit controls the traveling unit to travel at the speed around the door according to the deceleration command.

[0015] According to the fourth traveling work information processing system of the present invention, the traveling speed of the autonomous traveling work device traveling by remote operation can be forcibly reduced near the door. Thereby, even when a person suddenly intrudes from the door into the work area, the autonomous traveling work device can be immediately stopped, and the safety can be improved.

[0016] In order to solve the above problems, in the fifth traveling work information processing system of the present invention, in the management server, the server-side control unit determines the boundary between the work target and the non-work target based on the distance measurement result from the autonomous traveling work device to the non-work target by the measurement unit and the captured data captured by the imaging unit, and edits the display data by superimposing an image of a boundary line on the display position corresponding to the boundary in the image of the display data. The server-side communication unit is characterized by transmitting the edited display data to the operation terminal.

[0017] According to the fifth traveling work information processing system of the present invention, even when the autonomous traveling work device performs automatic traveling cleaning in a dark work area, Known by displaying the display data showing the boundary line between the non-work target such as a wall and the work target such as a floor on the operation terminal, the operator can easily grasp the floor surface where the autonomous traveling work device can travel, improve the safety of traveling, and improve the convenience of the operator. For example, by running the autonomous traveling work device along the boundary line between the wall and the floor, cleaning omission can be eliminated. Also, even in the display data in a dark work area, the first distance notation from the autonomous traveling work device to the obstacle and unknown the second distance notation from the obstacle to the wall unknown are displayed, so that the operator can Known non-work targets such as perform a remote operation to avoid the obstacle without overlooking the obstacle. unknown Furthermore, in the learning mode, when the operator performs a remote operation while referring to the remote operation screen of the operation terminal, the operator can perform learning traveling cleaning of the autonomous traveling work device while confirming the boundary line between the wall and the floor. unknown

[0018] For cleaning work In order to solve the above problems, the traveling work information processing method of the present invention The aforementioned cleaning work travels autonomously and automatically The aforementioned cleaning workA method for processing driving work information in an autonomous driving work device capable of performing the same, and an operating terminal for operating the autonomous driving work device, comprising: an imaging step in the autonomous driving work device to photograph the area in front of the autonomous driving work device and acquire photographic data; and in the autonomous driving work device, from the autonomous driving work device It does not exist in the environment map generated in advance through training runs. Up to the unknown obstacle 1 Distance and, Present in the aforementioned environmental map From known non-work targets, the above unknown Up to the obstacle 2nd A measurement process to measure distance and acquire measurement data, and in the autonomous driving work device, photograph A device-side communication process that transmits data and the measurement data to the operation terminal, and at the operation terminal, the autonomous driving work device photograph A terminal-side communication process for receiving data and the measurement data, and in the operating terminal 、 In the aforementioned shooting data 、 The aforementioned unknown Obstacles The first distance notation indicating the first distance is superimposed, and the second distance notation indicating the second distance is superimposed on the known non-working object, and simultaneously The present invention is characterized by having a display step, which includes a display step. [Effects of the Invention]

[0019] According to the present invention, the driving work information processing system and driving work information processing method can easily determine how to respond to obstacles in the work area and safely remotely operate the autonomous driving work device. [Brief explanation of the drawing]

[0020] [Figure 1] This is a block diagram showing the configuration of a driving operation information processing system according to the first embodiment of the present invention. [Figure 2] This is a schematic diagram showing the configuration of an autonomous driving device in a driving work information processing system according to the first embodiment of the present invention. [Figure 3] This is a block diagram showing the configuration of an autonomous driving device in a driving work information processing system according to the first embodiment of the present invention. [Figure 4]This is a block diagram showing the configuration of the management server in the driving operation information processing system according to the first embodiment of the present invention. [Figure 5] This is a block diagram showing the configuration of an operating terminal in a driving operation information processing system according to the first embodiment of the present invention. [Figure 6] This is a plan view showing an example of first captured data taken from the front of an autonomous driving work device in a driving work information processing system according to the first embodiment of the present invention. [Figure 7] This is a plan view showing an example of second image data captured from the rear and below of an autonomous driving work device in a driving work information processing system according to the first embodiment of the present invention. [Figure 8] This table shows examples of registration information for each user in the driving operation information processing system according to the first embodiment of the present invention. [Figure 9] This is a table showing examples of user authorization information in the driving operation information processing system according to the first embodiment of the present invention. [Figure 10] This is a plan view showing an example of display data based on first image data and measurement data in a driving work information processing system according to the first embodiment of the present invention. [Figure 11] This is a plan view showing an example of a login screen displayed by an operating terminal in a driving operation information processing system according to the first embodiment of the present invention. [Figure 12] This is a plan view showing an example of a remote control screen displayed by an operation terminal in a driving operation information processing system according to the first embodiment of the present invention. [Figure 13] This flowchart shows an example of the operation of an autonomous driving device in a driving work information processing system according to the first embodiment of the present invention. [Figure 14] This flowchart shows an example of the operation of the management server in the driving operation information processing system according to the first embodiment of the present invention. [Figure 15] This flowchart shows an example of the operation of an operating terminal in a driving operation information processing system according to the first embodiment of the present invention. [Figure 16]This is a block diagram showing the configuration of the management server in a driving operation information processing system according to a second embodiment of the present invention. [Figure 17] This is a plan view showing an example of display data based on first image data and measurement data in a driving work information processing system according to a second embodiment of the present invention. [Figure 18] This flowchart shows an example of the operation of an autonomous driving device in a driving work information processing system according to a second embodiment of the present invention. [Figure 19] This flowchart shows an example of the operation of the management server in the driving operation information processing system according to the second embodiment of the present invention. [Figure 20] This flowchart shows an example of the operation of an operating terminal in a driving operation information processing system according to a second embodiment of the present invention. [Figure 21] This is a plan view showing an example of display data based on first image data and measurement data in a driving work information processing system according to a third embodiment of the present invention. [Figure 22] This flowchart shows an example of the operation of the management server in the driving operation information processing system according to the third embodiment of the present invention. [Modes for carrying out the invention]

[0021] Embodiments of the present invention will be described below with reference to the drawings. The following embodiments are preferred examples of the present invention and disclose various preferred techniques, but the technical scope of the present invention is not limited to these embodiments.

[0022] A first embodiment of the present invention, a driving work information processing system 1, will now be described. As shown in Figure 1(a), the driving work information processing system 1 comprises an autonomous driving work device 2, a management server 3, and an operation terminal 4. The autonomous driving work device 2 is configured to perform automated driving work, which involves autonomous driving and automatic work. The management server 3 is configured to manage the autonomous driving work device 2, and the operation terminal 4 is configured to operate the autonomous driving work device 2. The autonomous driving work device 2 is connected to the management server 3 via a predetermined network 5 so as to be able to communicate with it, and the operation terminal 4 is connected to the management server 3 via the predetermined network 5 so as to be able to communicate with it.

[0023] The driving operation information processing system 1 may be equipped with multiple autonomous driving operation devices 2 having similar functions connected to a single management server 3, but this embodiment describes an example in which only one autonomous driving operation device 2 is provided. Furthermore, the driving operation information processing system 1 may be equipped with multiple operation terminals 4 having similar functions connected to a single management server 3, but this embodiment describes an example in which only one operation terminal 4 is provided.

[0024] First, let's describe the autonomous mobile work device 2. As shown in Figure 2, the autonomous mobile work device 2 comprises a main body 10 for housing various parts and a mobile unit 11 for moving the main body 10. The autonomous mobile work device 2 functions as an autonomous mobile cleaning device, equipped with a cleaning unit 12 as a work unit for performing predetermined tasks, for example, cleaning the floor surface FL below the main body 10. The autonomous mobile work device 2 can, for example, use all or part of an area such as a shopping mall, office, hotel, hospital, school, or factory as the cleaning area (work area), and the floor surface FL of the cleaning area as the cleaning target (work target).

[0025] Furthermore, the autonomous mobile work device 2 includes a measurement unit 13 that measures the positional relationship between the device body 10 and non-working objects such as surrounding walls and obstacles (for example, ornaments), and an obstacle detection unit 14 that detects walls and obstacles within a predetermined distance. The autonomous mobile work device 2 includes a first imaging unit 15 (imaging unit) that photographs the area in front of the device body 10, and a second imaging unit 16 that photographs the area behind the device body 10. The autonomous mobile work device 2 also includes an operation display unit 17 consisting of a touch panel or the like for operating and displaying various functions of the autonomous mobile work device 2, and a power supply unit 18 that supplies power to each part of the autonomous mobile work device 2 and controls the remaining battery level (not shown) and charging. Furthermore, non-working targets are not limited to walls and steps, but may also include stationary objects such as doors, glass automatic doors, sliding windows, curtains, partitions, other floor surfaces made of different materials than the work surface (floors where movement is not permitted), and cleaning indicators (e.g., work cones and signs). Moving objects such as people, small animals, and carts may also be included.

[0026] Furthermore, the autonomous mobile work device 2 includes a device-side control unit 20 that comprehensively controls each part and various functions of the autonomous mobile work device 2 (such as driving by the driving unit 11, cleaning by the cleaning unit 12, measurement by the measurement unit 13, and photography by the first imaging unit 15 and the second imaging unit 16), a device-side storage unit 21 that stores a cleaning plan (work plan) consisting of driving data from the driving unit 11 and cleaning data (work data) from the cleaning unit 12, and a device-side communication unit 22 for communicating with external devices such as the management server 3 and the operation terminal 4.

[0027] Next, we will explain the general operation of the autonomous driving work device 2.

[0028] The autonomous driving work device 2 is a vehicle capable of both manual and automatic driving, and operates by switching between one of three operating modes: learning mode, reproduction (automatic) mode, and manual mode. The autonomous driving work device 2 can switch operating modes in response to operations on the operation terminal 4 and the operation display unit 17.

[0029] In learning mode and manual mode, the autonomous mobile work device 2 performs learning-based cleaning (learning-based work) and manual-based cleaning (manual work), respectively, in response to manual operations on the operation terminal 4 and operation display unit 17 by workers such as cleaning staff operating the autonomous mobile work device 2 on-site, or by users such as operators remotely controlling the autonomous mobile work device 2. In addition, during learning-based cleaning, the device creates and stores an environmental map of the cleaning area in conjunction with the learning-based cleaning, and acquires driving data showing the driving route and driving status during the learning-based cleaning, as well as cleaning data showing the cleaning status during the learning-based cleaning, and stores a cleaning plan including the driving data and cleaning data.

[0030] In reproduction mode, the autonomous driving work device 2 performs automatic driving and cleaning (automatic driving work) in accordance with the automatic operation by the device-side control unit 20, based on the selected cleaning plan and the corresponding environmental map, in order to reproduce the cleaning plan selected by the worker from among the cleaning plans stored in the learned driving cleaning.

[0031] The cleaning plan consists of multiple steps along the travel route, with travel data and cleaning data associated with each step. The interval between steps may be set to a predetermined time interval (e.g., 25 ms) when performing a learning-type cleaning run, or it may be set to a predetermined travel distance of the autonomous work device 2 (e.g., 0.2 m). In addition to the above, the cleaning plan may also store the elapsed time from the start of the learning-type cleaning run associated with each step.

[0032] The driving data includes, for example, the self-position data of the driving unit 11 on its driving path (X and Y coordinates indicating its position on the environmental map, and the angle relative to the orientation of the starting position), steering flags (straight, left turn, right turn), driving speed [m / s] in the direction of travel, and turning speed [deg / s], and the driving path can be visualized based on the driving data. Alternatively, the driving speed may be switched to any of several speed levels, and the driving data may be stored by converting the speed levels into numbers (for example, 8 levels from 0 to 7).

[0033] The cleaning data includes, for example, the pad pressure of the cleaning member 24, the amount of water supplied by the cleaning fluid supply unit 25, and the suction amount of the suction unit 26. The pad pressure is the force with which the cleaning member 24 is pressed against the floor surface FL, the amount of water supplied is the amount of cleaning fluid (working fluid) supplied to the floor surface FL by the cleaning fluid supply unit 25, and the suction amount is the operating strength of the suction blower (not shown) when the suction unit 26 sucks up the wastewater after cleaning from the floor surface FL. The pad pressure, water supply amount, and suction amount may be switched to any of several strength levels, and the cleaning data may be stored by converting the stepped strengths of the pad pressure, water supply amount, and suction amount into numbers (for example, 3 levels from 0 to 2, or 5 levels from 0 to 4). The cleaning data may also include the operation / stop and rotation speed of the cleaning member 24.

[0034] Next, we will explain the various parts of the autonomous driving work device 2.

[0035] The running unit 11 is located at the bottom of the main body 10 and is equipped with one front wheel 11a as a drive wheel and a pair of rear wheels 11b as auxiliary wheels. Tires are attached to both the front wheel 11a and the pair of rear wheels 11b. The front wheel 11a is located at the front of the device in the width direction and is centered in the device width direction. It is equipped with a driving motor (not shown) and a front wheel rotation encoder (not shown). The running unit 11 moves the main body 10 forward by driving the driving motor to rotate the front wheel 11a, and stops the main body 10 by stopping the rotation of the front wheel 11a. The speed of the autonomous mobile work device 2 (running unit 11) is adjusted (accelerated / decelerated) by controlling the drive of the driving motor. The running unit 11 may also move the main body 10 backward by having the driving motor reverse the rotation of the front wheel 11a.

[0036] Furthermore, the front wheel 11a is equipped with a steering shaft (not shown), a steering motor (not shown), and a steering rotation encoder (not shown). The driving unit 11 steers the device body 10 by changing the direction of the front wheel 11a by driving the steering motor to rotate the steering shaft, and by moving the device body 10 forward while changing the direction of the front wheel 11a, the autonomous driving work device 2 (driving unit 11) turns left (left turn) or right (right turn). The steering angle of the autonomous driving work device 2 (driving unit 11) is adjusted by controlling the drive of the steering motor. For example, by rotating the steering shaft and controlling the steering motor to tilt the steering angle 90 degrees to the left or right with respect to the direction of travel, while turning the device body 10 forward or backward, the device body 10 makes a pivot turn to the left or right.

[0037] A pair of rear wheels 11b are provided at the rear in the direction of travel, spaced apart in the width direction (left-right direction) of the device, and each is equipped with an encoder (not shown) for determining the travel distance from the amount of rotation. The pair of rear wheels 11b rotate in response to the movement of the device body 10 driven by the front wheels 11a. The adjustment (acceleration / deceleration) of the travel speed (travel unit 11) and steering of the autonomous travel work device 2 (travel unit 11) may be performed by controlling the travel drive motor and the steering motor while feeding back the amount of rotation of each rear wheel 11b using the encoders provided on the rear wheels 11b. In this embodiment, an example in which the device is equipped with drive front wheels 11a and auxiliary rear wheels 11b has been described, but the present invention is not limited to this example, and in other embodiments, for example, the device may be equipped with auxiliary front wheels 11a and a pair of drive rear wheels 11b.

[0038] When the operating mode is set to learning mode or manual mode, the travel unit 11 operates in response to manual operation by the operator or worker on the operation terminal 4 or operation display unit 17. When the operating mode is set to reproduction mode, the travel unit 11 operates in response to control by the device-side control unit 20 (reproduction control unit 34) based on the travel data and environmental map of the cleaning plan selected by the operator or worker.

[0039] The cleaning unit 12 is located at the bottom of the main unit 10 and is configured to clean the floor surface FL below the main unit 10. When the operating mode is set to learning mode or manual mode, the cleaning unit 12 operates in response to manual operation by the operator or worker on the operation terminal 4 or operation display unit 17. When the operating mode is set to reproduction mode, the cleaning unit 12 operates in response to control (automatic operation) by the device-side control unit 20 (reproduction control unit 34) based on the cleaning data of the cleaning plan selected by the operator or worker.

[0040] The cleaning unit 12 is composed of, for example, a wet cleaning mechanism that cleans the floor surface FL using a cleaning solution, and includes a cleaning member 24 (working member) that contacts and cleans the floor surface FL, a cleaning solution supply unit 25 that supplies the cleaning solution to the floor surface FL, and a suction unit 26 that sucks up the used cleaning solution, i.e., the wastewater, after cleaning the floor surface FL. The cleaning unit 12 includes a motor for the cleaning member (not shown) that rotates the cleaning member 24 on the floor surface FL, and a actuator for the cleaning member (not shown) that moves the cleaning member 24 vertically relative to the floor surface FL and varies the pad pressure. The cleaning unit 12 also includes a wastewater collection unit (not shown) that collects the wastewater sucked up by the suction unit 26. Note that the cleaning unit 12 is not limited to a wet cleaning mechanism and may be composed of a dry cleaning mechanism such as a sweeper.

[0041] The cleaning components 24 consist of a pair of cleaning pads or a pair of cleaning brushes mounted side by side in the width direction (left-right direction) of the device approximately in the center of the direction of travel. The left cleaning pad or cleaning brush rotates clockwise when viewed from above, and the right cleaning pad or cleaning brush rotates counterclockwise when viewed from above, rotating from front to back towards the center in the width direction. This collects the dirty water and dust in front of the pair of cleaning pads or the pair of cleaning brushes towards the center in the width direction and discharges them to the rear.

[0042] The cleaning solution supply unit 25 is configured to supply and spray cleaning solution from a cleaning solution tank (not shown) onto the floor surface FL using a supply pump (not shown).

[0043] The suction unit 26 consists of a suction blower and the like, and collects the sucked-up wastewater into a wastewater collection unit (not shown). The wastewater collection unit uses a squeegee 27, which is set in contact with the floor surface FL behind the cleaning member 24, to receive and collect the wastewater discharged from the cleaning member 24 to the rear. The collected wastewater is then sucked up using the suction force of the suction unit 26 by a wastewater duct (not shown) connected to the suction unit 26, and the sucked-up wastewater is collected into a wastewater tank connected to the wastewater duct.

[0044] The measurement unit 13 includes a laser range finder (LRF) 13a that measures positional information (for example, the angle and distance with respect to the direction of travel of the device body 10) between the device body 10 and non-working objects such as surrounding walls and obstacles. The measurement unit 13 measures positional information with respect to non-working objects at predetermined intervals (for example, every 25 ms) while the device body 10 is moving. The LRF 13a is configured to have a detection range that extends to the front and both left and right sides of the device body 10.

[0045] The measurement unit 13 acquires measurement data by precisely measuring the distance from the autonomous driving work device 2 to non-working objects (such as obstacles). For example, the measurement unit 13 measures a first distance from the autonomous driving work device 2 to a non-working object in front of it (in the direction of travel) and a first direction to the non-working object. That is, based on the first distance and first direction, the measurement unit 13 can grasp the positional information of the non-working object relative to the autonomous driving work device 2. The measurement unit 13 also measures a second distance from a non-working object such as an obstacle in front of the autonomous driving work device 2 to another non-working object such as a wall located to the left or right (in the direction intersecting with the direction of travel) relative to that non-working object. The measurement unit 13 then acquires measurement data including the first distance, first direction, and second distance. If the measurement unit 13 detects multiple non-working objects, it acquires measurement data for each of the multiple non-working objects.

[0046] The obstacle detection unit 14 includes multiple proximity sensors 14a (detection units), such as ultrasonic sensors, that detect the presence or absence of non-working objects such as walls and obstacles within a predetermined distance from the main body of the device 10; step sensors, such as infrared sensors, that detect steps in the floor surface FL near the main body of the device 10; and bumper sensors, which are attached to the front lower bumper (not shown) of the main body of the device 10 and detect contact with non-working objects. The obstacle detection unit 14 may operate continuously while the main body of the device 10 is moving.

[0047] The first imaging unit 15 is equipped with a first imaging device 15a, such as an infrared camera or a CCD (Charge Coupled Device) camera, which photographs the area in front of the main body 10. The first imaging device 15a is mounted, for example, on the front upper side of the main body 10. The first imaging unit 15 photographs the area in front of the autonomous mobile work device 2 using the first imaging device 15a and acquires first imaging data 60 (imaging data) as shown in Figure 6. The first imaging unit 15, in which the first imaging device 15a is composed of multiple infrared cameras, can roughly measure positional information such as the distance to obstacles and walls in front of the autonomous mobile work device 2 based on the first imaging data 60. The second imaging unit 16 is equipped with a second imaging device 16a, such as a CCD camera, which photographs the area behind the main body 10. The second imaging device 16a is mounted, for example, on the rear lower side of the main body 10. For example, when the autonomous mobile work device 2 performs automatic mobile cleaning, the second imaging unit 16 uses the second imaging device 16a to photograph the area below and behind the autonomous mobile work device 2 and acquires second imaging data 61 as shown in Figure 7.

[0048] The operation display unit 17 is located on the upper rear side of the main body 10 of the device and is configured to include a touch panel or the like. Each part of the operation display unit 17 is connected to the device-side control unit 20. The operation display unit 17 includes, for example, a key switch (not shown) for operating the autonomous mobile work device 2 and an emergency stop button (not shown) for operating the autonomous mobile work device 2 in an emergency.

[0049] The touch panel of the operation display unit 17 displays various screens in response to control signals from the device-side control unit 20, and transmits operation signals to the device-side control unit 20 based on touch operations on each screen. For example, the touch panel of the operation display unit 17 may display a mode selection screen (not shown) on which the operating mode can be selected, or a setting screen (not shown) on which, in learning mode and manual mode, the travel speed of the travel unit 11, the operation / stop and pad pressure of the cleaning member 24, the operation / stop and water supply volume of the cleaning fluid supply unit 25, the operation / stop and suction volume of the suction unit 26, etc.

[0050] The power supply unit 18 includes a battery (power source) and a charging circuit mounted inside the main unit 10 of the device. When connected to an external power source, the battery is charged, and power is supplied to each part of the autonomous driving work device 2. The power supply unit 18 may also output a signal indicating the remaining battery level to the device-side control unit 20.

[0051] The device-side control unit 20 is composed of a computer such as a CPU (Central Processing Unit) and is connected to the device-side storage unit 21 and the device-side communication unit 22, which include ROM (Read Only Memory), RAM (Random Access Memory), hard disk, and flash memory, as shown in Figures 2 and 3. The device-side control unit 20 is also connected to each part of the autonomous driving work device 2, such as the driving unit 11, cleaning unit 12, measurement unit 13, obstacle detection unit 14, first imaging unit 15, second imaging unit 16, operation display unit 17, and power supply unit 18.

[0052] Furthermore, the device-side control unit 20 is connected to external devices via the device-side communication unit 22. The device-side communication unit 22 communicates wirelessly with external devices such as the management server 3 and operation terminals 4 held by users such as operators and workers, using wireless LAN communication standards such as Wi-Fi or Bluetooth (registered trademark). For example, while the autonomous mobile work device 2 is performing automatic mobile cleaning, the device-side communication unit 22 transmits first image data 60, which is an image taken by the first image unit 15 of the area in front of the autonomous mobile work device 2, and measurement data, which is the measurement result from the measurement unit 13, to the management server 3.

[0053] The device-side memory unit 21 stores programs and data for controlling each part and various functions of the autonomous driving work device 2, and the device-side control unit 20 performs calculation processing based on the programs and data stored in the device-side memory unit 21 to comprehensively control each part and various functions. For example, by executing a program stored in the device-side memory unit 21, the device-side control unit 20 operates as a mode switching unit 30, a learning control unit 31, a map creation unit 32, a cleaning plan creation unit 33, a reproduction control unit 34, and a remote operation control unit 35, as shown in Figure 3. As a result, the autonomous driving work device 2 can autonomously drive and perform work automatically according to a pre-stored program. In addition, the device-side memory unit 21 stores one or more cleaning plans, as well as environmental maps corresponding to each cleaning plan.

[0054] The mode switching unit 30 switches the operating mode to one of the following: learning mode, reproduction mode, or manual mode, in response to the operation of the operation terminal 4 or the operation display unit 17.

[0055] In learning mode, the learning control unit 31 acquires travel data from the travel unit 11 and cleaning data from the cleaning unit 12 at predetermined step intervals while learning travel cleaning is performed, and stores them in the device-side storage unit 21.

[0056] The learning control unit 31 acquires, for example, self-position data (X and Y coordinates, angle) based on position information measured by the measurement unit 13 as driving data. It also detects the number of rotations of the front wheels 11a using the front wheel rotation encoder of the front wheels 11a of the driving unit 11 and acquires the driving speed of the driving unit 11 based on the detection result. Furthermore, it detects the number of steering rotations of the front wheels 11a using the steering rotation encoder of the front wheels 11a of the driving unit 11 and acquires the turning speed of the driving unit 11 based on the detection result. The learning control unit 31 acquires, for example, cleaning data, including the operation / stop and pad pressure set for the cleaning member 24, the operation / stop and water supply amount set for the cleaning fluid supply unit 25, and the operation / stop and suction amount set for the suction unit 26.

[0057] The map creation unit 32, while learning-based driving and cleaning is performed in learning mode, uses technologies such as SLAM (Simultaneous Localization and Mapping) to estimate its own position and create an environmental map in real time.

[0058] Specifically, during the learning-type cleaning run in a predetermined cleaning area, the map creation unit 32 acquires positional information of the device body 10 and non-working objects such as the walls surrounding the device body 10 as measurement results from the measurement unit 13, and creates a local map of the area around the device body 10 at predetermined time intervals or predetermined distance intervals based on these measurement results from the measurement unit 13. Furthermore, the map creation unit 32 estimates the self-position (coordinates) of the autonomous driving work device 2 in the local map based on the local map and the detection results from each encoder of the driving unit 11 (amount of movement of the driving unit 11).

[0059] Then, when the learning run cleaning is completed, the map creation unit 32 creates an environmental map of the cleaning area by connecting (combining) each local map. The map creation unit 32 also creates a travel route by connecting (combining) its own position (each position information measured by the measurement unit 13) in the local maps. By using this environmental map, the drivable range along the travel route can be determined based on the drivable range and the position information of non-working targets measured by the measurement unit 13, and the condition of the travel route within the work area (such as the presence or absence of unknown obstacles) can be detected.

[0060] The cleaning plan creation unit 33 creates a cleaning plan (work plan) by associating the driving data and cleaning data stored in the device-side storage unit 21 by the learning control unit 31 during the learning driving cleaning process, step by step, when the learning driving cleaning is completed. It is also advisable for the cleaning plan creation unit 33 to transmit the created cleaning plan and environmental map to the management server 3 via the device-side communication unit 22 so that the management server 3 can recognize them. This allows other users accessing the management server 3 to check the cleaning plan and environmental map. Furthermore, when the autonomous driving work device 2 performs automatic driving cleaning, by notifying the management server 3 which cleaning plan and environmental map to use, other users accessing the management server 3 can know that the autonomous driving work device 2 is performing automatic driving cleaning based on that cleaning plan and environmental map.

[0061] The cleaning plan creation unit 33 accepts the input of a plan name for the created cleaning plan on a predetermined input screen (not shown) on the touch panel of the operation display unit 17, and adds the plan name entered by the operator or the like to the created cleaning plan. The cleaning plan creation unit 33 stores the created cleaning plan in the device-side storage unit 21, associating it with the environmental map created by the map creation unit 32 in the same learning-run cleaning run. In addition, the cleaning plan creation unit 33 may create a cleaning plan for environmental maps, running data, and cleaning data that have been created in advance by using the autonomous driving work device 2 as a computer, or by using external devices such as the management server 3 or operation terminal 4, or by automatically creating a driving route according to a pre-entered program, without performing a learning-run cleaning run.

[0062] When the operating mode is in reproduction mode, the reproduction control unit 34 accepts the selection of a cleaning plan on a predetermined cleaning plan selection screen (not shown) on the touch panel of the operation display unit 17, reads the selected cleaning plan from the device-side storage unit 21, and controls the driving unit 11 and the cleaning unit 12 to perform automatic driving cleaning based on the driving data and cleaning data of this cleaning plan and the environmental map corresponding to the cleaning plan.

[0063] At this time, the reproduction control unit 34 performs automated cleaning while estimating the self-position of the autonomous driving work device 2 on the environmental map corresponding to the cleaning plan. For example, the reproduction control unit 34 uses the map creation unit 32 to create a local map using technologies such as SLAM, estimates the self-position of the autonomous driving work device 2 in the local map, and matches the local map to the environmental map to estimate its self-position on the environmental map.

[0064] The reproduction control unit 34 then controls the driving unit 11 by combining the self-position data for each step of the driving data of the cleaning plan with the self-position estimated on the environmental map, and controls the cleaning work step by step based on the cleaning data of the cleaning unit 12.

[0065] Furthermore, in automated driving cleaning, the reproduction control unit 34 detects non-working objects such as obstacles using the laser rangefinder 13a of the measurement unit 13 and the proximity sensor 14a of the obstacle detection unit 14. It controls the driving unit 11 to decelerate when the distance to the non-working object approaches a predetermined safety distance, and controls the driving unit 11 to stop when the distance to the non-working object falls below the predetermined safety distance. In addition, the reproduction control unit 34 can determine whether there are any differences by overlaying the environmental map corresponding to the cleaning plan with the local map created using the map creation unit 32. If differences are found, the reproduction control unit 34 may determine that these differences are unknown obstacles that do not exist on the environmental map. At this time, the reproduction control unit 34 can calculate the distance from the autonomous driving work device 2 to the unknown obstacle based on the two-dimensional coordinates of the local map.

[0066] In automatic cleaning, the reproduction control unit 34 transmits the first image data 60, which is the image taken by the first image unit 15 of the area in front of the autonomous work device 2, and the measurement data, which is the measurement result from the measurement unit 13, to the management server 3 via the device-side communication unit 22. In automatic cleaning, if the reproduction control unit 34 detects an unknown obstacle in the direction of travel that is not on the environmental map based on the measurement result from the measurement unit 13, it stops the automatic cleaning when it approaches the obstacle until the distance from the autonomous work device 2 to the obstacle becomes a predetermined stopping distance, and enters a state where it is waiting for manual or remote operation. In addition, if the reproduction control unit 34 detects an unknown obstacle, it transmits an error notification indicating that the device is unable to move due to the obstacle to the management server 3 via the device-side communication unit 22.

[0067] When the operation mode is set to reproduction mode, if the device-side communication unit 22 receives a remote control command from an external device such as the management server 3 or the operation terminal 4 while the autonomous mobile work device 2 is performing automatic mobile cleaning, the remote control control unit 35 will perform remote mobile driving instead of automatic mobile cleaning. When the remote control control unit 35 performs remote mobile driving, it temporarily suspends automatic mobile cleaning, and when it finishes remote mobile driving, it resumes automatic mobile cleaning. During remote mobile driving, the remote control control unit 35 controls the mobile unit 11 and the cleaning unit 12 so that the autonomous mobile work device 2 moves according to the remote control commands that correspond to the remote control commands.

[0068] For example, if the remote control instructs the autonomous mobile work device 2 to move forward, stop, turn right, or turn left, the remote control unit 35 controls the mobile unit 11 so that the autonomous mobile work device 2 moves forward, stops, turns right, or turns left, and may also accept the setting of the mobile speed of the mobile unit 11 via remote control. The remote control unit 35 may accept the setting of execution / stop of cleaning work by the cleaning unit 12 via remote control, and may also accept individual settings such as operation / stop of cleaning of cleaning member 24 and pad pressure, operation / stop of cleaning fluid supply of cleaning fluid supply unit 25 and supplied water volume, and operation / stop of suction of suction unit 26 and suction volume. In addition, the remote control unit 35 may control the cleaning unit 12 to always stop cleaning work when remotely controlled travel is performed, regardless of remote control.

[0069] Furthermore, if the remote control unit 35 instructs the remote control to avoid an obstacle, in addition to controlling the driving unit 11 according to the remote control, it may also control the driving unit 11 to maintain a predetermined safe distance from the obstacle based on the measurement results of the measurement unit 13. At this time, the remote control unit 35 may continue remote driving while detecting an obstacle in the direction of travel of the autonomous driving work device 2, and terminate remote driving when it no longer detects an obstacle. Alternatively, the remote control unit 35 may terminate remote driving in response to the remote control. In addition, if the remote control unit 35 instructs automatic obstacle avoidance, it may control the driving unit 11 to avoid the obstacle by automatic driving, and terminate remote driving by automatic driving when it no longer detects an obstacle. When the remote control unit 35 terminates remote driving, it sends a notification of termination of remote driving to the management server 3 via the device-side communication unit 22.

[0070] Furthermore, when remote control instructs the first shooting unit 15 or the second shooting unit 16 to perform shooting operations, the remote control control unit 35 controls the shooting of the first shooting unit 15 or the second shooting unit 16, and the transmission of the first shooting data 60 captured by the first shooting unit 15 and the second shooting data 61 captured by the second shooting unit 16. The remote control control unit 35 may also control the shooting direction and zoom for the shooting of the first shooting unit 15 or the second shooting unit 16. The remote control control unit 35 is not limited to the remote control described above, and may accept various remote controls for each part of the autonomous driving work device 2, enabling control of each part.

[0071] Next, the management server 3 will be described. The management server 3 manages the autonomous driving work device 2 and also manages the operating rights of users such as operators and workers who use the operation terminal 4. As shown in Figure 4, the management server 3 comprises a server-side control unit 40, a server-side storage unit 41, and a server-side communication unit 42.

[0072] The server-side control unit 40 is responsible for the overall control of each part and various functions of the management server 3. It is composed of a computer such as a CPU and is connected to the server-side storage unit 41, which includes ROM, RAM, hard disk, flash memory, etc., and the server-side communication unit 42.

[0073] Furthermore, the server-side control unit 40 is connected to external devices via the server-side communication unit 42 to enable communication. The server-side communication unit 42 communicates wirelessly with external devices such as the autonomous driving work device 2 and the operation terminal 4 held by users such as operators and workers, using wireless LAN communication standards such as Wi-Fi or Bluetooth (registered trademark). For example, the server-side communication unit 42 receives from the autonomous driving work device 2 first image data 60 taken by the first image unit 15 of the autonomous driving work device 2, and measurement data which is the measurement result from the measurement unit 13 of the autonomous driving work device 2, and transmits the first image data 60 and the corresponding display data 62 (see Figure 10) to the operation terminal 4. The server-side communication unit 42 receives remote control from the operation terminal 4 and transmits a remote control command corresponding to the remote control to the autonomous driving work device 2.

[0074] The server-side storage unit 41 stores programs and data for controlling each part and various functions of the management server 3, and the server-side control unit 40 performs calculation processing based on the programs and data stored in the server-side storage unit 41 to comprehensively control each part and various functions. For example, the server-side control unit 40 operates as the device terminal linking unit 45, the display data generation unit 46, and the remote control command unit 47 by executing the programs stored in the server-side storage unit 41.

[0075] The management server 3 manages users such as operators and workers of the operation terminal 4. As shown in Figure 8, the server-side storage unit 41 stores registration information for each user, including personal identification information (personal code), name, and user type. The management server 3 also manages the autonomous mobile work devices 2. As shown in Figure 9, the server-side storage unit 41 stores authorization information for each user to remotely operate each autonomous mobile work device 2. The authorization information is stored associated with each user's personal code, the device identification information (e.g., identification number) of each autonomous mobile work device 2, and the type of remote operation authorization. The management server 3 also receives cleaning plans and environmental maps created by the autonomous mobile work devices 2 during learning-based cleaning, various photographic data, and the operation details of the operation terminal 4 by workers via the server-side communication unit 42, and stores the history of these in the server-side storage unit 41.

[0076] The device terminal linking unit 45 links the autonomous mobile work device 2 and the operation terminal 4 in order to enable remote control of the autonomous mobile work device 2 by the operation terminal 4. For example, the device terminal linking unit 45 receives a selection of the autonomous mobile work device 2 that the operator 4 has logged into the management server 3 to remotely control, and links the autonomous mobile work device 2 with the operation terminal 4. The device terminal linking unit 45 determines whether the autonomous mobile work device 2 can be remotely controlled and also determines the content of the remote control that will be accepted, based on the authority information of the user, such as the operator or worker, who possesses the operation terminal 4.

[0077] The display data generation unit 46 generates display data 62 to be displayed on the operation terminal 4, as shown in Figure 10, based on the first image data 60 captured by the first image capture unit 15 and the measurement data measured by the measurement unit 13 in the autonomous driving work device 2. Here, the display data generation unit 46 generates the display data 62 by associating the distance notation of the measurement data with obstacles in the image of the image data, for example, by superimposing the distance notation of the measurement data at a display position related to the obstacle. Alternatively, the distance notation may be displayed at a position away from the obstacle, or it may be displayed via a leader line or the like between the obstacle and the display data 62. If there are no obstacles in the first image data 60, the display data generation unit 46 may use the first image data 60 as is for display data 62. Furthermore, the display data generation unit 46 may generate the display data 62 to include display position information of the distance notation on the image in order to accept a selection operation of the distance notation on the image of the display data 62. The display position information may include position information on the image, the type of distance notation, and the type of remote operation corresponding to the selection operation of the distance notation. Alternatively, the display data generation unit 46 may generate the display data 62 such that the image of the display data 62 includes an operation icon for accepting the selection operation of the distance notation. The display data generation unit 46 transmits the generated display data 62 to the operation terminal 4 via the server-side communication unit 42.

[0078] For example, the display data generation unit 46 identifies the position information of a predetermined obstacle in front of the autonomous driving work device 2 based on the measurement data of the obstacle and the first distance and direction from the autonomous driving work device 2 to the obstacle. The display data generation unit 46 identifies the approximate position information of obstacles and walls on the image based on the first image data 60 captured by the first imaging unit 15, which is a plurality of infrared cameras. The display data generation unit 46 determines which of the approximate position information in the first image data 60 corresponds to the position information of the obstacle identified from the measurement data, thereby identifying the display position of the obstacle on the image of the first image data 60. The display data generation unit 46 then generates display data 62 by associating a first distance notation 63 indicating the first distance to the obstacle with the image of the first image data 60, for example, by editing the first image data 60 so that the first distance notation 63 is superimposed on the display position of the obstacle. In this case, the display data generation unit 46 may generate display data 62 by performing image processing such as projection transformation so that a marker (for example, a white triangle) indicating that the first distance notation 63 is the distance from the autonomous driving work device 2 to the obstacle is displayed together with the first distance notation 63. The display position information of the first distance notation 63 may include an instruction to automatically avoid obstacles as a remote operation corresponding to the selection operation of the first distance notation 63.

[0079] Furthermore, the display data generation unit 46 identifies surrounding information of a predetermined obstacle in front of the autonomous driving work device 2 based on the second distance from the obstacle to the wall, with respect to the measurement data of the obstacle. The display data generation unit 46 identifies rough positional information of the obstacle and wall on the image based on the first image data 60 captured by the first image unit 15, which is a plurality of infrared cameras. The display data generation unit 46 determines which of the rough positional information of the first image data 60 corresponds to the surrounding information of the obstacle identified from the measurement data, and identifies the display position of the surrounding information of the obstacle on the image (for example, the midpoint between the obstacle and the wall). Then, the display data generation unit 46 generates display data 62 by associating a second distance notation 64 indicating the second distance of the obstacle with the obstacle on the image of the first image data 60, for example, by editing the first image data 60 so that the second distance notation 64 is superimposed on the display position of the surrounding information of the obstacle. In this case, the display data generation unit 46 may generate display data 62 by performing image processing such as projection transformation so that a sign (for example, a white rectangle) indicating that the second distance notation 64 is the distance from the obstacle to the wall is displayed together with the second distance notation 64. Furthermore, if the second distance is shorter than the width that the autonomous driving work device 2 can travel, a sign indicating no entry may be displayed together with the second distance notation 64. The display position information of the second distance notation 64 may include an automatic driving instruction at the position corresponding to the second distance notation 64 as a remote operation corresponding to the selection operation of the second distance notation 64.

[0080] Furthermore, if measurement data exists for multiple obstacles, the display data generation unit 46 should generate display data 62 such that it displays a first distance notation 63 from the autonomous driving work device 2 to the obstacle and a second distance notation 64 from the obstacle to the wall for each of the multiple obstacles.

[0081] Furthermore, the display data generation unit 46 may generate display data 62 such that it displays the first distance notation 63 and the second distance notation 64 for measurement data where the first distance from the autonomous driving work device 2 to the obstacle is within a predetermined range, while not displaying the first distance notation 63 and the second distance notation 64 for measurement data where the first distance is outside the predetermined range. Alternatively, the display data generation unit 46 may generate display data 62 such that it highlights the first distance notation 63 and the second distance notation 64 for measurement data where the first distance from the autonomous driving work device 2 to the obstacle is within a predetermined range (for example, by displaying them in a dark color or large size), while displaying them more subtly for measurement data where the first distance is outside the predetermined range (for example, by displaying them in a light color or small size), thereby changing the display format of the distance notation according to the distance. Furthermore, the display data generation unit 46 may display the second distance indicator 64 as drivable for measurement data where the second distance from the obstacle to the wall is greater than or equal to the combined width of the autonomous driving work device 2's working width and safety distance, while displaying the second distance indicator 64 as not drivable for measurement data where the second distance is less than the combined width of the autonomous driving work device 2's working width and safety distance.

[0082] For example, when the remote control command unit 47 receives an error notification from the autonomous driving work device 2 that indicates it is unable to move due to an obstacle, it sends a remote control request to the operation terminal 4 via the server-side communication unit 42. When the remote control command unit 47 receives a remote control from the operation terminal 4 via the server-side communication unit 42, it generates a remote control command corresponding to that remote control. For example, the remote control command unit 47 may generate a remote control command based on the content of the remote control and the direction of travel of the autonomous driving work device 2. Specifically, if the autonomous driving work device 2 detects an obstacle in the direction of travel based on the first image data 60 captured by the first image capture unit 15 and the measurement data measured by the measurement unit 13, and the remote control is an instruction to avoid the obstacle, the remote control command unit 47 generates a remote control command instructing the autonomous driving work device 2 to move while avoiding the obstacle.

[0083] Furthermore, if the remote control is a driving instruction such as moving forward, stopping, turning right, or turning left, the remote control command unit 47 generates a remote control command for the driving instruction. If the remote control is a driving setting instruction such as the driving speed of the driving unit 11, the remote control command unit 47 generates a remote control command for the driving setting instruction. If the remote control is a work instruction such as executing / stopping cleaning work of the cleaning unit 12, the remote control command unit 47 generates a remote control command for the work instruction. If the remote control is an individual setting instruction such as operating / stopping cleaning of the cleaning member 24 and pad pressure, operating / stopping cleaning fluid supply of the cleaning fluid supply unit 25 and the amount of water supplied, or operating / stopping suction of the suction unit 26 and the amount of suction, the remote control command unit 47 generates a remote control command for the individual setting instruction.

[0084] If the remote control is an instruction to operate the first imaging unit 15 or the second imaging unit 16, the remote control command unit 47 generates a remote control command for the said imaging operation instruction. The remote control command unit 47 is not limited to the remote control described above, and may accept various remote controls for each part of the autonomous driving work device 2 and generate remote control commands to control each part.

[0085] The server-side communication unit 42 may determine the type of remote control command to send to the autonomous driving work device 2 according to the authorization information of the operation terminal 4 linked to the autonomous driving work device 2. For example, it may identify the operation terminal 4 that sends driving operations for the driving unit 11, the operation terminal 4 that sends cleaning operations for the cleaning unit 12, the operation terminal 4 that sends shooting operations for the first shooting unit 15, and the operation terminal 4 that sends shooting operations for the second shooting unit 16.

[0086] Furthermore, when the autonomous driving work device 2 has finished remotely controlled driving, the remote control command unit 47 receives a notification of the end of remotely controlled driving from the autonomous driving work device 2 via the server-side communication unit 42, and then transmits the notification of the end of remotely controlled driving to the operation terminal 4 via the server-side communication unit 42.

[0087] Next, the operating terminal 4 will be described. The operating terminal 4 is composed of a personal computer or a tablet terminal, and as shown in Figure 5, it includes a terminal-side control unit 50, a terminal-side storage unit 51, a terminal-side communication unit 52, and a display unit 53 which is composed of a touch panel or the like.

[0088] The terminal-side control unit 50 provides overall control over all parts and functions of the operating terminal 4. It is composed of a computer such as a CPU and is connected to the terminal-side storage unit 51, which includes ROM, RAM, hard disk, and flash memory, as well as the terminal-side communication unit 52 and the display unit 53.

[0089] Furthermore, the terminal-side control unit 50 is connected to external devices via the terminal-side communication unit 52 so as to be able to communicate with external devices. The terminal-side communication unit 52 communicates wirelessly with external devices such as the autonomous driving work device 2 and the management server 3 using communication standards such as Wi-Fi or Bluetooth (registered trademark). For example, the terminal-side communication unit 52 sends a selection instruction for the autonomous driving work device 2 to the management server 3 in order to link it with the autonomous driving work device 2 that is to be remotely controlled. The terminal-side communication unit 52 receives display data 62 related to the linked autonomous driving work device 2 from the management server 3 and also sends remote control instructions using the display data 62 to the management server 3.

[0090] The terminal-side storage unit 51 stores programs and data for controlling each part and various functions of the operating terminal 4, and the terminal-side control unit 50 performs calculation processing based on the programs and data stored in the terminal-side storage unit 51 to comprehensively control each part and various functions. For example, the terminal-side control unit 50 operates as a login processing unit 55, a device selection unit 56, a display control unit 57, and an operation input unit 58 by executing a program stored in the terminal-side storage unit 51.

[0091] The login processing unit 55 accepts login requests to the management server 3 from users such as operators or workers using the operation terminal 4. As shown in Figure 11, the login processing unit 55 displays a login screen 65 on the display unit 53 for performing the login process, and accepts input of the user's personal identification information (personal code) and password on the login screen 65. The login screen 65 may consist of a web-based screen managed by the management server 3. The personal identification information and password are managed by the management server 3. The login processing unit 55 transmits the login request, including the entered personal identification information and password, to the management server 3 via the terminal-side communication unit 52.

[0092] When the management server 3 authenticates the personal identification information and password based on the login request and sends a successful login notification to the operation terminal 4, the device selection unit 56 displays a device selection screen (not shown) on the display unit 53 for selecting the autonomous driving work device 2 to be remotely controlled, and accepts the selection of the autonomous driving work device 2 on the device selection screen. The device selection screen may consist of a web-based screen managed by the management server 3. The device selection unit 56 sends the selection instruction for the selected autonomous driving work device 2 to the management server 3 via the terminal-side communication unit 52. As a result, the management server 3 links the autonomous driving work device 2 and the operation terminal 4.

[0093] When the autonomous mobile work device 2, which is linked to the operation terminal 4, performs automatic mobile cleaning, the display control unit 57 displays a remote control screen 70 for remotely controlling the autonomous mobile work device 2 on the display unit 53, as shown in Figure 12. The remote control screen 70 may consist of a web-based screen managed by the management server 3.

[0094] The remote control screen 70 displays a login name 71 indicating the user, such as a logged-in operator or worker, and a device name 72 indicating the autonomous mobile work device 2 to be remotely controlled. The remote control screen 70 also displays a travel operation area 73 for operating the travel unit 11, a cleaning operation area 74 for operating the cleaning unit 12, a camera operation area 75 for operating the first shooting unit 15 and the second shooting unit 16, and an image display area 76 for displaying display data 62 based on the first shooting data 60 captured by the first shooting unit 15.

[0095] Each display area of ​​the remote control screen 70 is displayed according to the authorization information of the user, such as the operator or worker who possesses the operation terminal 4, to use the autonomous mobile work device 2. For example, if the authorization information allows for the operation of the mobile unit 11, the mobile operation area 73 is displayed as operable, but if it is not permitted, the mobile operation area 73 is not displayed or is displayed as inoperable. If the authorization information allows for the operation of the cleaning unit 12, the cleaning operation area 74 is displayed as operable, but if it is not permitted, the cleaning operation area 74 is not displayed or is displayed as inoperable. If the authorization information allows for the operation of the first shooting unit 15 or the second shooting unit 16, the camera operation area 75 is displayed as operable, but if it is not permitted, the camera operation area 75 is not displayed or is displayed as inoperable. In addition, if the authorization information allows for viewing of the display data 62, the image display area 76 is displayed as viewable, but if it is not permitted, the image display area 76 is not displayed.

[0096] In the driving operation area 73, a speed change button 80 for changing the driving speed of the driving unit 11 is displayed in an operable manner. The speed change button 80 includes, for example, an up button that increases the driving speed in stages each time it is operated, and a down button that decreases the driving speed in stages each time it is operated. In the driving operation area 73, an operation key 81 for inputting driving commands for the driving unit 11, such as moving forward, stopping, turning right, or turning left, is displayed in an operable manner.

[0097] The cleaning operation area 74 displays an operable cleaning switch 82 for switching the operation / stop of the cleaning member 24 of the cleaning unit 12, and a pad pressure adjustment button 83 for adjusting the pad pressure of the cleaning member 24. Each time the pad pressure adjustment button 83 is operated, the pad pressure of the cleaning member 24 is switched in a stepwise and cyclical manner.

[0098] The cleaning operation area 74 displays an operable supply switch 84 for switching the operation / stop of the cleaning fluid supply unit 25 of the cleaning unit 12, and a water supply volume adjustment button 85 for adjusting the amount of water supplied to the cleaning fluid supply unit 25. Each time the water supply volume adjustment button 85 is operated, the amount of water supplied to the cleaning fluid supply unit 25 is switched in a stepwise and cyclical manner.

[0099] The cleaning operation area 74 displays an operable suction switch 86 for switching the operation / stop of the suction unit 26 of the cleaning unit 12, and a suction volume adjustment button 87 for adjusting the suction volume of the suction unit 26. Each time the suction volume adjustment button 87 is operated, the suction volume of the suction unit 26 is switched in a stepwise and cyclical manner.

[0100] The camera operation area 75 displays an operable zoom button 88 for controlling the zoom of the first shooting unit 15 and the second shooting unit 16, and a shooting direction button 89 for controlling the shooting direction of the first shooting unit 15 and the second shooting unit 16. The display control unit 57 adjusts and moves the display position of the display data 62 in response to the operation of the zoom button 88.

[0101] The image display area 76 displays display data 62 related to the associated autonomous driving work device 2. The display data 62 mainly displays the image of the first shooting data 60 taken by the first shooting unit 15, and the distance notation of the measurement data measured by the measurement unit 13 is superimposed on the image at the display position related to the obstacle. In the image display area 76, the image of the display data 62 displays the first distance notation 63 from the autonomous driving work device 2 to the obstacle and the second distance notation 64 from the obstacle to the wall as distance notations. The display data 62 includes display position information of the first distance notation 63 and the second distance notation 64 on the image, and by determining whether the selection operation on the image of the display data 62 displayed in the image display area 76 corresponds to the display position information, it is possible to determine the selection operation of the first distance notation 63 and the second distance notation 64.

[0102] Furthermore, when the autonomous driving work device 2 detects an unknown obstacle, the display control unit 57 receives a remote operation request from the management server 3 via the terminal-side communication unit 52, and displays a notification recommending remote operation of the autonomous driving work device 2 on the remote operation screen 70. In addition, when the autonomous driving work device 2 finishes remote operation, the display control unit 57 receives a notification of completion of remote operation from the management server 3 via the terminal-side communication unit 52, and displays the notification of completion of remote operation on the remote operation screen 70.

[0103] The operation input unit 58 accepts selection operations at display positions related to obstacles on the image of the display data 62 displayed in the image display area 76, for example, by accepting selection operations for the first distance notation 63 and the second distance notation 64. The operation input unit 58 may accept selection operations for the first distance notation 63 and the second distance notation 64 regardless of the recommendation notification for remote operation of the autonomous driving work device 2.

[0104] When a selection operation is performed for the first distance notation 63 or the second distance notation 64, for example, the operation input unit 58 acquires the type of remote operation corresponding to the selection operation of the first distance notation 63 or the second distance notation 64 based on the display position information corresponding to the first distance notation 63 or the second distance notation 64. The operation input unit 58 transmits the acquired remote operation to the management server 3 via the terminal-side communication unit 52. For example, if the display position information of the first distance notation 63 includes an instruction to automatically avoid obstacles as a remote operation corresponding to the selection operation of the first distance notation 63, the operation input unit 58 transmits a remote operation instructing automatic avoidance to the management server 3 via the terminal-side communication unit 52. If the display position information of the second distance notation 64 includes an instruction to automatically drive at the position corresponding to the second distance notation 64 as a remote operation corresponding to the selection operation of the second distance notation 64, the operation input unit 58 transmits a remote operation instructing automatic driving to the management server 3 via the terminal-side communication unit 52.

[0105] Alternatively, the operation input unit 58 may enable operation of the operation key 81 for remotely controlling the autonomous driving work device 2 when a selection operation of the first distance notation 63 or the second distance notation 64 is performed on the image of the display data 62 displayed in the image display area 76. In this case, the operation input unit 58 transmits remote control commands indicating driving instructions such as moving forward, stopping, turning right, or turning left to the management server 3 via the terminal-side communication unit 52 in response to the operation of the operation key 81.

[0106] Next, an example of how the autonomous driving work device 2 avoids obstacles in response to remote control while performing automatic driving cleaning in the driving work information processing system 1 will be explained with reference to the flowcharts in Figures 13 to 15. Figure 13 shows the operation of the autonomous driving work device 2, Figure 14 shows the operation of the management server 3, and Figure 15 shows the operation of the operation terminal 4.

[0107] First, when the autonomous mobile work device 2 starts automatic mobile cleaning (step S11 in Figure 13), while performing automatic mobile cleaning, the first imaging unit 15 acquires first imaging data 60 of the area in front of the autonomous mobile work device 2, and the measurement unit 13 acquires measurement data of the distance from the autonomous mobile work device 2 to the non-working target, and transmits the first imaging data 60 and the measurement data to the management server 3 (step S12 in Figure 13). In this example of operation, the autonomous mobile work device 2 performing automatic mobile cleaning always transmits the first imaging data 60 and the measurement data to the management server 3 at predetermined time intervals. However, in other examples, only the first imaging data 60 may be transmitted to the management server 3, or the first imaging data 60 and the measurement data may be transmitted to the management server 3 only if an error caused by an obstacle occurs.

[0108] In this example, the process proceeds to step A in Figure 14(a), where the management server 3 receives the first image data 60 and measurement data from the autonomous driving work device 2 (step S21 in Figure 14(a)). Based on the first image data 60 and measurement data, the management server 3 generates display data 62 (step S22 in Figure 14(a)). If an unknown obstacle not present on the environmental map is detected in the first image data 60 and measurement data, the management server 3 generates display data 62 that includes the first distance from the autonomous driving work device 2 to the obstacle, the first direction to the obstacle, and the second distance from the obstacle to the wall. The management server 3 then transmits the generated display data 62 to the operation terminal 4 (step S23 in Figure 14(a)).

[0109] In this example, the process moves to step B in Figure 15, where the operation terminal 4 receives display data 62 from the management server 3 (step S31 in Figure 15), and displays the display data 62 on the remote control screen 70 (step S32 in Figure 15). This allows the operation terminal 4 to check the display data 62 based on the first captured data 60, which is a photograph of the area in front of the autonomous mobile work device 2, while the autonomous mobile work device 2 is performing automatic cleaning.

[0110] Incidentally, if the autonomous driving work device 2 does not detect an unknown obstacle that is not present on the environmental map during the execution of automatic driving cleaning (step S13 in Figure 13: No), it will continue automatic driving cleaning. If it finishes the automatic driving cleaning (step S14 in Figure 13: Yes), it will terminate the operation of the driving work information processing system 1. On the other hand, if the autonomous driving work device 2 detects an unknown obstacle that is not present on the environmental map during the execution of automatic driving cleaning (step S13 in Figure 13: Yes), it will send an error notification indicating that it is unable to drive due to the obstacle to the management server 3 (step S15 in Figure 13), and will stop the automatic driving cleaning and wait (step S16 in Figure 13).

[0111] In this example, the process moves to step C in Figure 14(b), where the management server 3 receives an error notification from the autonomous driving work device 2 (step S24 in Figure 14(b)) and sends a remote operation request to the operation terminal 4 (step S25 in Figure 14(b)).

[0112] In the example operation, the process moves to step D in Figure 15, where the operation terminal 4 receives a remote operation request from the management server 3 (step S33 in Figure 15), and displays a notification recommending remote operation of the autonomous driving work device 2 (step S34 in Figure 15).

[0113] On the operation terminal 4, the remote operation screen 70 displaying the display data 62 accepts remote operation of the autonomous driving work device 2, and when an permission response is given to the remote operation recommendation notification and remote operation is performed (step S35 in Figure 15), for example, when remote operation is performed by selecting the first distance notation 63 or the second distance notation 64 of the display data 62, the remote operation is sent to the management server 3 (step S36 in Figure 15).

[0114] In this example, the process proceeds to step E in Figure 14(c), where the management server 3 receives a remote operation from the operation terminal 4 (step S26 in Figure 14(c)) and transmits a remote operation command corresponding to that remote operation to the autonomous driving work device 2 (step S26 in Figure 14(c)).

[0115] In the example operation, the process moves to step F in Figure 13. The autonomous mobile work device 2 stops its automatic cleaning and is in standby mode. When it receives a remote control command from the management server 3 (step S17 in Figure 13: Yes), it acquires the first image data 60 and measurement data and transmits them to the management server 3 in order to perform remote control driving (step S18 in Figure 13). Alternatively, the first image data 60 and measurement data recorded in the device-side storage unit 21 may be transmitted to the operation terminal 4 via the management server 3 a few seconds or minutes prior to the time the remote control was received, allowing the operation terminal 4 to refer to the data from a few seconds or minutes earlier. This allows not only confirmation of the current image (state) but also confirmation of the circumstances under which the current image (state) came to be. Then, the autonomous mobile work device 2 drives while avoiding obstacles according to the remote control (step S19 in Figure 13). When the autonomous driving device 2 finishes such remotely controlled driving (step S20 in Figure 13: Yes), it resumes automatic driving cleaning and proceeds to step S12 in Figure 13 in the example operation.

[0116] As described above, according to this embodiment, the driving work information processing system 1 includes an autonomous driving work device 2 capable of performing autonomous driving cleaning (autonomous driving work) which autonomously drives and performs (cleaning) work automatically, and an operation terminal 4 for operating the autonomous driving work device 2. The autonomous driving work device 2 includes a driving unit 11, a first imaging unit 15 (imaging unit), and a measurement unit 13. The operation terminal 4 includes a display unit 53. In the autonomous driving work device 2, the first imaging unit 15 takes a picture of the area in front of the autonomous driving work device 2 and acquires first imaging data 60 (imaging data), and the measurement unit 13 measures the distance from the autonomous driving work device 2 to the non-working object, thereby measuring a first distance from the autonomous driving work device 2 to an unknown non-working object (obstacle) and a second distance from a known non-working object (such as a wall or step) to the obstacle and acquiring measurement data. In the operation terminal 4, the display unit 53 displays an image of display data 62 generated by associating the distance notation of the measurement data with the obstacle in the image of the first imaging data 60.

[0117] With this configuration, the driving work information processing system 1 can easily determine whether or not to avoid an obstacle even when it encounters an unknown obstacle that is not present in a pre-stored environmental map while the autonomous driving work device 2 is performing automatic cleaning. This can be done by checking the image of the displayed data 62 on the operation terminal 4 that remotely controls the autonomous driving work device 2. At this time, by checking the image of the displayed data 62 on the operation terminal 4, it is possible to easily grasp the first distance from the autonomous driving work device 2 to the obstacle, and the distance from the autonomous driving work device to known non-working targets such as walls and steps, thereby improving the operator's awareness of the surrounding conditions, including obstacles. Therefore, the driving work information processing system 1 makes it easy to determine how to respond to obstacles in the work area and allows for safe remote control of the autonomous driving work device 2.

[0118] Furthermore, in this embodiment, the driving work information processing system 1 further includes a management server 3 for managing the autonomous driving work device 2, and the management server 3 includes a server-side control unit 40 and a server-side communication unit 42. The autonomous driving work device 2 includes a device-side control unit 20 and a device-side communication unit 22. The operation terminal 4 includes a terminal-side communication unit 52. In the autonomous driving work device 2, the device-side communication unit 22 transmits the first image data 60 and measurement data to the management server 3. In the management server 3, the server-side communication unit 42 receives the first image data 60 and measurement data transmitted from the autonomous driving work device 2, and the server-side control unit 40 generates display data 62 by superimposing the distance notation of the measurement data onto the display position related to the obstacle in the image of the first image data 60, and the server-side communication unit 42 transmits the display data 62 to the operation terminal 4. In the operating terminal 4, the terminal-side communication unit 52 receives the display data 62 transmitted from the management server 3, and the display unit 53 displays the image of the display data 62.

[0119] With this configuration, the driving operation information processing system 1 can manage the autonomous driving operation device 2 that performs automatic driving cleaning and the operation terminal 4 that remotely controls the autonomous driving operation device 2, by linking them via the management server 3. When the autonomous driving operation device 2 detects an unknown obstacle, the management server 3 can send display data 62 regarding the obstacle to the operation terminal 4 linked to the autonomous driving operation device 2 for display. Furthermore, by performing high-load image processing such as the creation of display data on the management server, there is no concern about a decrease in the control processing speed of the autonomous driving operation device or the operation terminal.

[0120] Furthermore, in this embodiment, the driving work information processing system 1 further comprises an operation input unit 58 in the operation terminal 4 and a remote control command unit 47 in the management server 3. In the operation terminal 4, the operation input unit 58 receives selection operations at display positions related to obstacles and / or known non-working objects on the image of display data 62 displayed on the display unit 53, and the terminal-side communication unit 52 transmits the display position and the remote operation corresponding to the selection operation to the management server 3. In the management server 3, the server-side communication unit 42 receives the remote operation transmitted from the operation terminal 4, the remote control command unit 47 generates a remote operation command based on the remote operation and the direction of travel of the autonomous driving work device 2, and the server-side communication unit 42 transmits the remote operation command to the autonomous driving work device 2. In the autonomous driving work device 2, the device-side communication unit 22 receives the remote operation command transmitted from the management server 3, and the device-side control unit 20 controls the driving unit 11 to drive according to the remote operation command.

[0121] With this configuration, even when the autonomous driving work device 2 detects an unknown obstacle, the driving work information processing system 1 can remotely control the autonomous driving work device 2 to avoid the obstacle by selecting a display position related to the obstacle or a known non-work target in the display data 62 displayed on the operation terminal 4. This makes it easy to determine how to respond to obstacles even when remotely controlling the autonomous driving work device 2, improving the convenience of the operator. Furthermore, even if it is recognized that avoiding the obstacle is impossible, it is easy to determine how to respond to the obstacle, including the content of the remote control, such as detouring to an alternative route or arranging for on-site personnel to respond.

[0122] Next, the driving operation information processing system 1 of the second embodiment will be described. In the following description of the second embodiment, components similar to those in the driving operation information processing system 1 of the first embodiment will be denoted by the same reference numerals, and detailed explanations will be omitted.

[0123] In the second embodiment in particular, the driving work information processing system 1, based on first image data 60 taken from the front of the autonomous driving work device 2 and measurement data measuring the distance from the autonomous driving work device 2 to the non-working object, slows down the autonomous driving work device 2 near the door when it detects (recognizes) a door adjacent to the work area or a door being opened.

[0124] In the second embodiment, the server-side control unit 40 of the management server 3 operates as a deceleration command unit 48 in addition to the device terminal linking unit 45, display data generation unit 46, and remote operation command unit 47, as shown in Figure 16, by executing a program stored in the server-side storage unit 41. The deceleration command unit 48 issues a deceleration command to slow down the autonomous driving work device 2 when it detects a door or door opening on the direction of travel of the autonomous driving work device 2.

[0125] While the autonomous driving work device 2 is performing remote-controlled driving, the deceleration command unit 48 receives first image data 60 taken by the first image unit 15 of the area in front of the autonomous driving work device 2, and measurement data measured by the measurement unit 13 of the distance from the autonomous driving work device 2 to the non-working target, via the server-side communication unit 42. Based on the first image data 60 and the measurement data, the deceleration command unit 48 detects whether or not there is a door to be decelerated on the side of the autonomous driving work device 2's direction of travel. If the deceleration command unit 48 detects a door to be decelerated, it obtains the distance from the autonomous driving work device 2 to the door and the direction to the door based on the measurement data, and obtains the door's position information in the work area. Furthermore, the deceleration command unit 48 identifies the approximate position information to the door based on the first image data 60, and by comparing the door's position information based on the measurement data with the door's position information based on the first image data 60, it obtains the door's display position information in the image of the first image data 60.

[0126] The deceleration command unit 48 detects, for example, whether there is a door or whether the door is open by analyzing the image of the first captured data 60, or, when a continuous wall is detected based on measurement data, it determines the shape of the wall based on the measurement data and detects whether there is a door by detecting a recessed area by a predetermined width from the wall, and also detects whether the door is open by detecting whether the depth in the recessed direction of the recessed area is greater than or equal to a predetermined depth threshold, or it detects whether there is a door or whether the door is open by using these methods in combination.

[0127] Furthermore, the deceleration command unit 48 may designate all detected doors as doors to be decelerated, or it may designate doors within a predetermined range from the travel path of the autonomous driving work device 2 as doors to be decelerated, or it may designate open doors as doors to be decelerated, or it may designate doors that have been determined to have entered the room in which the door is located as doors to be decelerated.

[0128] For example, the deceleration command unit 48 may determine entry into a room with a door if it detects light coming from a small window in a door that has a small window. Also, if the driving work information processing system 1 is linked with an entry system that manages entry into each room of a building that has a work area, the deceleration command unit 48 may determine entry into a room with a detected door based on the entry system's detection results for entry into the room, the detection results for locking / unlocking the room, and the detection results for opening and closing the door of the room.

[0129] Furthermore, when the deceleration command unit 48 detects a door to be decelerated, and the image brightness of the first captured data 60 is above a predetermined brightness threshold, the display data generation unit 46 may determine the display position of the door in the image of the display data 62 based on the display position information of the door in the image of the first captured data 60, and generate the display data 62 so as to display a marker 66 (for example, a dashed line surrounding the outline of the door) that highlights the door at its display position, as shown in Figure 17. The display data generation unit 46 generates the display data 62 so as to superimpose a third distance notation 67 indicating the distance from the autonomous driving work device 2 to the door on the display position of the door in the image of the display data 62. In addition, the display data generation unit 46 may generate the display data 62 so as to include the display position information of the door to be decelerated on the image in order to accept the selection operation of the door to be decelerated on the image of the display data 62.

[0130] In the operation terminal 4, which receives such display data 62 from the management server 3 and displays it on the remote operation screen 70, the operation input unit 58 accepts selection operations for the highlighted deceleration target door on the image of the display data 62 displayed in the image display area 76 of the remote operation screen 70. For example, it accepts selection operations for the display position of the deceleration target door, selection operations for the highlighted sign 66, and selection operations for the third distance indicator 67. The display data 62 includes display position information of the deceleration target door on the image, and by determining whether the selection operation on the image of the display data 62 displayed in the image display area 76 corresponds to the display position information, it is possible to determine the selection operation for the deceleration target door.

[0131] Furthermore, when the autonomous driving work device 2 performing automatic driving cleaning approaches a door to be decelerated and travels within a predetermined range from the door, the deceleration command unit 48 generates a deceleration command to the autonomous driving work device 2 to a predetermined speed around the door and transmits it to the autonomous driving work device 2 via the server-side communication unit 42. Here, the deceleration command unit 48 may automatically generate a deceleration command when the autonomous driving work device 2 reaches within a predetermined range from the door to be decelerated, or it may generate a deceleration command in response to a deceleration operation by the operator. Alternatively, when the autonomous driving work device 2 itself detects that it has approached a door to be decelerated and reached within a predetermined range from the door, the autonomous driving work device 2 may notify the management server 3 of a warning due to approaching the door, and the management server 3 may generate a deceleration command to the speed around the door in response to this warning notification and transmit it to the autonomous driving work device 2.

[0132] For example, when a door to be decelerated is selected on the image of the display data 62 displayed in the image display area 76 on the remote control screen 70 of the operation terminal 4, the operation input unit 58 transmits the selection of the door to the management server 3 via the terminal-side communication unit 52. When the management server 3 receives the selection of the door to be decelerated from the operation terminal 4, the deceleration command unit 48 generates a deceleration command to the door surrounding speed and transmits it to the autonomous driving work device 2 when the autonomous driving work device 2, which is performing automatic driving cleaning, reaches a predetermined range from the selected door to be decelerated. As a result, the autonomous driving work device 2 will travel at a speed around the selected door to be decelerated. The autonomous driving work device 2 may continue or stop the cleaning work by the cleaning unit 12 while traveling at a speed around the door.

[0133] Furthermore, the deceleration command unit 48 generates a command to release the door-area speed when the autonomous driving work device 2, which has decelerated to the door-area speed, moves outside a predetermined range from the selected door to be decelerated, and transmits it to the autonomous driving work device 2. As a result, when the autonomous driving work device 2 passes the selected door to be decelerated, it releases the deceleration to the door-area speed and travels at its original speed. Alternatively, when the autonomous driving work device 2 itself detects that it has moved outside a predetermined range from the door to be decelerated and has passed the door to be decelerated, the autonomous driving work device 2 may notify the management server 3 to release the warning, and the management server 3 may, in response to this notification of warning release, generate a command to release the door-area speed and transmit it to the autonomous driving work device 2.

[0134] Next, an example of the operation in which the autonomous driving work device 2 decelerates around the door in response to remote control while performing automatic driving cleaning in the driving work information processing system 1 of the second embodiment will be explained with reference to the flowcharts in Figures 18 to 20. Figure 18 shows the operation of the autonomous driving work device 2, Figure 19 shows the operation of the management server 3, and Figure 20 shows the operation of the operation terminal 4.

[0135] First, when the autonomous mobile work device 2 starts automatic mobile cleaning (step S41 in Figure 18), while it is performing automatic mobile cleaning, the first imaging unit 15 captures first imaging data 60 of the area in front of the autonomous mobile work device 2, and the measurement unit 13 captures measurement data of the distance from the autonomous mobile work device 2 to the non-working target, and transmits the first imaging data 60 and the measurement data to the management server 3 (step S42 in Figure 18).

[0136] In this example, the process moves to step G in Figure 19, where the management server 3 receives the first image data 60 and measurement data from the autonomous driving work device 2 (step S51 in Figure 19). Based on the first image data 60 and measurement data, it generates display data 62 (step S52 in Figure 19).

[0137] Furthermore, based on the first image data 60 and measurement data, the management server 3 determines whether or not there is a door to be decelerated on the side of the autonomous driving work device 2 in the direction of travel (step S53 in Figure 19). If no door is detected (step S53 in Figure 19: No), the management server 3 sends the generated display data 62 as is to the operation terminal 4 (step S54 in Figure 19). On the other hand, if a door is detected (step S53 in Figure 19: Yes), the management server 3 adds a sign 66 to highlight the door and a third distance notation 67 indicating the distance from the autonomous driving work device 2 to the door to the generated display data 62 (step S55 in Figure 19) before sending the display data 62 to the operation terminal 4 (step S54 in Figure 19).

[0138] In this example, the process proceeds to step H in Figure 20. When the operation terminal 4 receives display data 62 from the management server 3 (step S61 in Figure 20), it displays the display data 62 on the remote control screen 70 (step S62 in Figure 20). This allows the operation terminal 4 to check the display data 62 based on the first captured data 60, which is a photograph of the area in front of the autonomous mobile work device 2, while the autonomous mobile work device 2 is performing automatic cleaning.

[0139] On the remote control screen 70 displaying the display data 62, the operation terminal 4 accepts the selection of the door to be decelerated. If no selection of the door to be decelerated is made (step S63 in Figure 20: No), the operation example returns. On the other hand, if a selection of the door to be decelerated is made (step S63 in Figure 20: Yes), the operation terminal 4 sends the selection of the door to be decelerated to the management server 3 (step S64 in Figure 20), and the operation example proceeds to step I in Figure 19, returning. In either case, the operation terminal 4 continues to display the remote control screen 70.

[0140] When the management server 3 receives the selection of the door to be decelerated from the operation terminal 4 (step S56 in Figure 19: Yes), it sends a deceleration command to the autonomous driving work device 2 to slow down to a predetermined door-area speed in order to decelerate around the door to be decelerated (step S57 in Figure 19), and in the example operation, proceeds to step J in Figure 18.

[0141] When the autonomous driving work device 2 receives a deceleration command from the management server 3 during the execution of automatic driving cleaning (step S43 in Figure 18: Yes), it will control itself to decelerate to the speed around the door (step S44 in Figure 18).

[0142] Furthermore, in the management server 3, if the autonomous driving work device 2 is not decelerating (step S58 in Figure 19: No), the operation example returns. However, if the autonomous driving work device 2 is decelerating (step S58 in Figure 19: Yes), the management server 3 determines whether the autonomous driving work device 2 has passed the door to be decelerated and moved out of the predetermined range (step S59 in Figure 19). In the management server 3, if the autonomous driving work device 2 has not passed the door to be decelerated (step S59 in Figure 19: No), the operation example returns. However, if it has passed the door to be decelerated (step S59 in Figure 19: Yes), the management server 3 sends a release command to the autonomous driving work device 2 to release the deceleration (step S60 in Figure 19), and the operation example proceeds to step K in Figure 18 and returns.

[0143] When the autonomous driving work device 2 receives a release command from the management server 3 during the execution of automatic driving cleaning (step S45 in Figure 18: Yes), it will control itself to release the deceleration to door-area speed and return to its original speed (step S46 in Figure 18).

[0144] Furthermore, if the autonomous driving work device 2 does not receive a deceleration command from the management server 3 (step S43 in Figure 18: No), and does not receive a release command from the management server 3 (step S45 in Figure 18: No), the autonomous driving work device 2 will terminate its operation when it finishes automatic driving cleaning (step S47 in Figure 18: Yes).

[0145] As described above, according to the second embodiment, in the driving work information processing system 1, the management server 3, based on the measurement results from the autonomous driving work device 2 to the non-work target by the measurement unit 13 and the first image data 60 captured by the first image unit 15, issues a deceleration command to a predetermined door surrounding speed when it detects a door, and the server-side communication unit 42 transmits the deceleration command to the autonomous driving work device 2. In the autonomous driving work device 2, the device-side communication unit 22 receives the deceleration command transmitted from the management server 3, and the device-side control unit 20 controls the driving unit 11 to drive at the door surrounding speed in accordance with the deceleration command.

[0146] With this configuration, the mobile work information processing system 1 can forcibly reduce the speed of the remotely controlled autonomous mobile work device 2 near doors. This allows the autonomous mobile work device 2 to stop immediately even if a person or other object suddenly enters the work area through a door, thereby improving safety. Furthermore, when a door is detected, the system can determine which door is the target for deceleration, allowing the autonomous mobile work device 2 to decelerate only near doors where people are expected to enter or exit.

[0147] Next, the third embodiment of the driving operation information processing system 1 will be described. In the following description of the third embodiment, components similar to those in the first and second embodiments of the driving operation information processing system 1 will be denoted by the same reference numerals, and detailed explanations will be omitted.

[0148] In the third embodiment in particular, the driving work information processing system 1 determines the boundary between the wall (non-work object) and the floor (work object) based on first image data 60 taken from the front of the autonomous driving work device 2 and measurement data measuring the distance from the autonomous driving work device 2 to the non-work object. The system edits the display data 62 by superimposing an image of the boundary line 68 (see Figure 21) onto the image of the display data 62, and displays the edited image of the display data 62 on the operation terminal 4.

[0149] In the third embodiment, the first imaging unit 15 of the autonomous mobile work device 2 is equipped with a first imaging device 15a consisting of an infrared camera, enabling imaging even when the work area is dark, such as when the work is done at night or when the lights in the work area are turned off. The first imaging data 60 captured by the first imaging device 15a of the infrared camera is displayed as a monochrome image, but positional information such as the distance to obstacles and walls in front of the autonomous mobile work device 2 is roughly measured. The first imaging device 15a may be equipped with both an infrared camera for distance measurement and a CCD camera for capturing display data.

[0150] The display data generation unit 46 of the management server 3 is configured to determine the boundary between the wall and the floor based on the first image data 60 and measurement data, and to overlay the boundary line 68 onto the display data 62 for editing. The display data generation unit 46 transmits the generated display data 62 to the operation terminal 4 via the server-side communication unit 42.

[0151] Specifically, similar to the first embodiment, when the display data generation unit 46 detects an unknown obstacle that does not exist on the environmental map, it generates display data 62 that includes a first distance notation 63 from the autonomous driving work device 2 to the obstacle and a sign corresponding to the first distance notation 63, based on the first photographic data 60 taken by the first photographic unit 15 and the measurement data measured by the measurement unit 13. Also, similar to the first embodiment, the display data generation unit 46 generates display data 62 that includes a second distance notation 64 from the obstacle to the wall and a sign corresponding to the second distance notation 64, based on the first photographic data 60 and the measurement data.

[0152] Furthermore, in the third embodiment in particular, the display data generation unit 46 identifies rough positional information to non-working objects such as obstacles and walls on the image based on the first captured data 60, and determines which of the rough positional information of the first captured data 60 corresponds to the positional information of the non-working object identified from the measurement data, thereby identifying the display position of the non-working object on the image of the first captured data 60. At this time, the display data generation unit 46 detects non-working objects such as walls by detecting continuous non-working objects, determines the boundary between non-working objects such as walls and working objects such as floors based on the positional information of the detected walls, and identifies the display position information of the boundary. Alternatively, the display data generation unit 46 may detect obstacles by detecting continuous non-working objects, determine the boundary between obstacles and floors based on the positional information of the detected obstacles, and identify the display position information of the boundary.

[0153] As shown in Figure 21, the display data generation unit 46 edits the display data 62 image so that the boundary line 68 is superimposed on the display position corresponding to the boundary between the wall and the floor. For example, the display data generation unit 46 may use techniques such as superimposition to superimpose the boundary line 68 onto the display data 62. Figure 21 shows the display data 62 generated based on the first captured data 60 taken by the autonomous driving work device 2 while it was performing automatic driving cleaning in a dark work area. Here, the display data generation unit 46 may display the boundary line 68 in a display format that allows it to be identified as the boundary between the wall and the floor (for example, a dashed line or blinking). If the display data 62 is based on the monochrome first captured data 60, it may be possible to make it distinguishable from the wall and floor by displaying the boundary line 68 in a primary color display format such as red, blue, or green. The display data generation unit 46 edits the display data 62 image so that the boundary line 68 is superimposed on the display position corresponding to the boundary between the obstacle and the floor. Here, the display data generation unit 46 should display the boundary line 68 in a display format (for example, a dashed line or flashing) that allows it to be identified as the boundary between an obstacle and the floor. If the display data 62 is based on monochrome first shooting data 60, it is preferable to display the boundary line 68 in a primary color display format so that it can be identified as an obstacle or the floor.

[0154] Next, an example of the operation in which the autonomous driving work device 2 generates display data 62 while performing automatic driving cleaning in a dark work area in the driving work information processing system 1 of the third embodiment will be explained with reference to the flowchart in Figure 22. Note that the operation of the autonomous driving work device 2 is the same as the operation of the first embodiment shown in Figure 13, the operation of the management server 3 is the same as the operation of the first embodiment shown in Figures 14(b) and 14(c), and the operation of the operation terminal 4 is the same as the operation of the first embodiment shown in Figure 15, so the explanation of these operations will be omitted. In other words, in the third embodiment, the management server 3 performs the operation shown in Figure 22 instead of the operation shown in Figure 14(a).

[0155] In the autonomous driving work device 2 performing automatic driving cleaning, when the first imaging unit 15 transmits the first imaging data 60 and the measurement unit 13 transmits the measurement data to the management server 3 (step S12 in Figure 13), the operation example of the third embodiment proceeds to step A in Figure 22.

[0156] When the management server 3 receives the first image data 60 and measurement data from the autonomous driving work device 2 (step S71 in Figure 22), it generates display data 62 based on the first image data 60 and measurement data (step S72 in Figure 22). If an unknown obstacle that does not exist on the environmental map is detected in the first image data 60 and measurement data, the management server 3 will generate display data 62 that includes a first distance notation 63 from the autonomous driving work device 2 to the obstacle and a second distance notation 64 from the obstacle to the wall. Furthermore, if the management server 3 detects a boundary between the wall and the floor based on the first image data 60 and measurement data (step S73 in Figure 22: Yes), it edits the display data 62 to add a boundary line 68 corresponding to that boundary (step S74 in Figure 22). The management server 3 transmits the generated display data 62 to the operation terminal 4 (step S75 in Figure 22).

[0157] At this point, in the example operation, the process moves to step B in Figure 15, where the operation terminal 4 displays the display data 62, including the boundary line 68, received from the management server 3, on the remote operation screen 70 (step S32 in Figure 15).

[0158] As described above, according to the third embodiment, in the driving work information processing system 1, the server-side control unit 40 in the management server 3 determines the boundary between the wall and the floor based on the measurement results from the autonomous driving work device 2 to the non-work target by the measurement unit 13 and the first photographic data 60 taken by the first photographic unit 15, and edits the image of the display data 62 by superimposing a boundary line 68 on the display position corresponding to the boundary.

[0159] With this configuration, even when the autonomous mobile work device 2 performs automatic cleaning in a dark work area, the mobile work information processing system 1 displays display data 62 showing the boundary line 68 between the wall and the floor on the operation terminal 4. This allows the operator to easily understand the floor surface FL that the autonomous mobile work device 2 can travel on, improving safety during operation and convenience for the operator. Furthermore, even if the first photographic data 60, which is the source of the display data 62, is taken using auxiliary lighting or by the first photographic unit 15 of a night vision camera, the boundary line 68 is superimposed, making the boundary between the wall and the floor easier to see. For example, by having the autonomous mobile work device 2 travel along the boundary line 68 between the wall and the floor, any missed areas during cleaning can be eliminated. In addition, in the third embodiment, even in the display data 62 for a dark work area, the first distance indication 63 from the autonomous mobile work device 2 to an obstacle and the second distance indication 64 from the obstacle to the wall are displayed, so the operator can perform remote operations to avoid obstacles without overlooking them. Furthermore, in learning mode, the operator can perform learning-style cleaning of the autonomous mobile work device 2 while remotely operating it by referring to the remote operation screen 70 of the operation terminal 4 and confirming the boundary line 68 between the wall and the floor.

[0160] In the above-described embodiment, the driving work information processing system 1 communicates various data between the autonomous driving work device 2 and the operation terminal 4 via the management server 3. However, the present invention is not limited to this example. For example, in another embodiment, the driving work information processing system 1 may be configured without a management server 3, as shown in Figure 1(b), by having the autonomous driving work device 2 or the operation terminal 4 possess the functions of the management server 3.

[0161] Furthermore, the present invention can be modified as appropriate, as long as it does not contradict the gist or idea of ​​the invention as can be read from the claims and the specification as a whole, and a driving operation information processing system with such modifications is also included in the technical concept of the present invention. [Industrial applicability]

[0162] The present invention can be suitably used in a mobile work information processing system equipped with an industrial (commercial) work robot, such as an automatic floor cleaning and scrubbing device or a security device that uses cameras for monitoring. This device is capable of autonomously driving and performing automatic work tasks, such as cleaning the floors of commercial facilities like shopping malls, or performing automatic work in work areas like factories and railway terminals. [Explanation of Symbols]

[0163] 1. Driving Operation Information Processing System 2. Autonomous driving work device 3. Management Server 4. Operating terminal 11. Running section 12 Cleaning Department (Working Department) 13 Measurement Unit 15. First Photography Department 20 Device-side control unit 22 Device-side communication unit 34 Reproduction Control Unit 35 Remote Control Unit 40 Server-side control unit 42 Server-side communication unit 45 Inter-device terminal linking section 46 Display Data Generation Unit 47 Remote Control Command Unit 48 Deceleration command part 50 Terminal-side control unit 52 Terminal-side communication unit 53 Display section 57 Display Control Unit 58 Operation Input Section 60 First shooting data 62 Display Data 63 First distance notation 64 Second distance notation 68 Boundary Line 70 Remote control screen

Claims

1. A mobile work information processing system comprising an autonomous mobile work device capable of autonomously moving for cleaning work and automatically performing the cleaning work, and an operating terminal for operating the autonomous mobile work device, The autonomous driving work device comprises a driving unit, a shooting unit, a measurement unit, and a device-side communication unit. The aforementioned operating terminal comprises a terminal-side communication unit and a display unit. In the aforementioned autonomous driving work device, The aforementioned imaging unit captures images of the area in front of the autonomous driving work device and acquires the image data. The measurement unit measures a first distance from the autonomous driving work device to an unknown obstacle that does not exist on the environmental map generated in advance by learning driving, and a second distance from a known non-work target that exists on the environmental map to the unknown obstacle, and acquires measurement data. The device-side communication unit transmits the shooting data and the measurement data to the operation terminal. In the aforementioned operating terminal, The terminal-side communication unit receives the shooting data and the measurement data. The display unit superimposes a first distance notation indicating the first distance onto the unknown obstacle in the captured data, and a second distance notation indicating the second distance onto the known non-working object, and displays them simultaneously. A driving operation information processing system characterized by the following:

2. The system further includes a management server for managing the aforementioned autonomous driving work device, The management server comprises a server-side control unit and a server-side communication unit. The autonomous driving work device further comprises a device-side control unit, In the aforementioned autonomous driving work device, The device-side communication unit transmits the image data and the measurement data to the management server. In the aforementioned management server, The server-side communication unit receives the image data and measurement data transmitted from the autonomous driving work device. The server-side control unit generates display data by superimposing the first distance notation on the display position related to the unknown obstacle in the image of the captured data. The server-side communication unit transmits the display data to the operation terminal. In the aforementioned operating terminal, The terminal-side communication unit receives the display data transmitted from the management server, The display unit displays an image of the display data. The driving operation information processing system according to feature 1.

3. The aforementioned operating terminal further comprises an operating input unit, The management server further comprises a remote control command unit, In the aforementioned operating terminal, The operation input unit accepts selection operations on the image of the display data displayed on the display unit at display positions related to the unknown obstacle and / or the known non-work object. The terminal-side communication unit transmits the display position and the remote operation corresponding to the selection operation to the management server in response to the selection operation. In the aforementioned management server, The server-side communication unit receives the remote control transmitted from the operating terminal, The remote control command unit generates remote control commands based on the remote control and the direction of travel of the autonomous driving work device. The server-side communication unit transmits the remote control command to the autonomous driving work device. In the aforementioned autonomous driving work device, The device-side communication unit receives the remote control command transmitted from the management server, The device-side control unit controls the traveling unit to travel in accordance with the remote control command. The driving operation information processing system according to feature 2.

4. In the aforementioned management server, Based on the measurement results from the autonomous driving device to the non-working object by the measurement unit and the image data captured by the image capture unit, the server-side control unit, upon detecting a door, issues a deceleration command to a predetermined speed around the door. The server-side communication unit transmits the deceleration command to the autonomous driving device. In the aforementioned autonomous driving work device, The device-side communication unit receives the deceleration command transmitted from the management server, The device-side control unit controls the traveling unit to travel at the door-side speed in accordance with the deceleration command. The driving operation information processing system according to claim 2 or 3, characterized in that it is the same as described in claim 2 or 3.

5. In the aforementioned management server, The server-side control unit determines the boundary of the non-working object based on the distance measurement result from the autonomous driving work device to the non-working object by the measurement unit and the image data captured by the image capture unit, and edits the display data by superimposing an image of the boundary line at the display position corresponding to the boundary in the image of the display data. The server-side communication unit transmits the edited display data to the operation terminal. A driving operation information processing system according to any one of claims 2 to 4.

6. A method for processing driving work information in an autonomous driving work device that drives autonomously for cleaning work and automatically performs the cleaning work, and an operating terminal for operating the autonomous driving work device, The autonomous driving work device includes a shooting step of taking a picture of the area in front of the autonomous driving work device and acquiring the captured data, The autonomous driving work device includes a measurement step of measuring a first distance from the autonomous driving work device to an unknown obstacle that does not exist in the environmental map generated in advance by learning driving, and a second distance from a known non-work object that exists in the environmental map to the unknown obstacle, and acquiring measurement data. The autonomous driving work device includes a device-side communication process that transmits the captured data and the measured data to the operation terminal, The aforementioned operating terminal includes a terminal-side communication process for receiving the image data and the measurement data from the autonomous driving work device, The operation terminal includes a display step in which, in the captured data, a first distance notation indicating the first distance is superimposed on the unknown obstacle, and a second distance notation indicating the second distance is superimposed on the known non-working object, and these are displayed simultaneously. A method for processing driving operation information, characterized by having the following features.

Images