Autonomous operation apparatus control system and autonomous operation apparatus control method

Abstract

To realize an efficient remote operation of autonomous operation apparatuses.SOLUTION: An autonomous operation apparatus control system comprises: autonomous operation apparatuses capable of autonomous operation and operable by a plurality of operators each via an operator terminal 1; and a cloud server 3. Therein: there exists a plurality of operator terminals 1; when the cloud server 3 receives a remote operation request D1 from the autonomous operation apparatus 2, the cloud server transmits the remote operation request D1 to the operator terminals 1; and on a display screen of each of the operator terminals 1, at least information relating to a state of the autonomous operation apparatus 2, information relating to an operation state of the autonomous operation apparatus 2, and information relating to an operation of the autonomous operation apparatus 2 are displayed.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a technology for an autonomous operation device control system and an autonomous operation device control method. [Background technology]

[0002] When an autonomously operating device that operates autonomously needs to be remotely controlled by an operator, the device is switched to remote control.

[0003] Patent Document 1 describes that when a communication robot needs to be handled by an operator, the robot calls the operator. In Patent Document 1, the robot controls itself based on operation commands input by the called operator.

[0004] Patent Document 2 describes that when an event occurs that requires operator intervention, the robot system selects an operator and notifies the selected operator. At this time, the robot system selects an operator based on information about the operator (the operator's abilities, roles, and responsibilities), the operator's condition (mental and physical condition), etc.

[0005] Patent Document 3 describes that when manual operation of a robot becomes necessary, an operator group is selected from an operator group that has the authority to operate the robot that requires manual operation. Patent Document 3 also describes that a request to operate the robot is made to an operator console operated by the selected operator group. The operator group is selected based on the skills of the operators (transportation skills, movement skills, interaction skills, etc.). [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Patent No. 4639343 specification [Patent Document 2] Patent Publication No. 2021-064147 [Patent Document 3] Patent Publication No. 2021-062426 Summary of the Invention [Problem to be solved by the invention]

[0007] The technology described in Patent Document 1 is based on the premise that an operator is available to respond to calls from the robot. Therefore, if the operator is away from their desk for other business, they cannot respond to calls from the robot.

[0008] Furthermore, the techniques described in Patent Documents 2 and 3 do not take into consideration the case where the operator is away from his desk for other business.

[0009] The present invention has been made in view of the above background, and an object of the present invention is to realize efficient remote control of an autonomously operating device. [Means for solving the problem]

[0010] In order to solve the above-mentioned problems, the present invention comprises an autonomous operating device that is capable of autonomous operation and can be operated by multiple operators via an operating terminal, and a server, wherein there are multiple operating terminals, and when the server receives a request from the autonomous operating device, it notifies the operating terminal, and the display screen of the operating terminal displays at least information regarding the state of the autonomous operating device, information regarding the operating state of the autonomous operating device, and information regarding the operation of the autonomous operating device. Other solutions will be described as appropriate in the embodiments. [Effects of the Invention]

[0011] According to the present invention, efficient remote control of an autonomously operated device can be realized. [Brief explanation of the drawings]

[0012] [Figure 1] 1 is a diagram illustrating a configuration of an autonomous operation device monitoring system according to a first embodiment. [Figure 2] FIG. 1 is a functional block diagram of an autonomous operation device monitoring system according to a first embodiment. [Figure 3A] FIG. 1 is a diagram illustrating the appearance of an autonomously operated device. [Figure 3B] FIG. 1 is a diagram illustrating the appearance of an autonomously operated device. [Figure 4] 4 is a flowchart showing the procedure of a remote monitoring method performed in the first embodiment. [Figure 5] FIG. 10 is a diagram illustrating an example of a remote monitoring screen displayed on an operator terminal. [Figure 6] FIG. 4 is a diagram showing an example of a notification window in the first embodiment. [Figure 7] 6 is a flowchart showing details of the operation of the operator terminal in the first embodiment. [Figure 8] FIG. 10 is a diagram illustrating an example of a remote monitoring screen when the operator terminal is a mobile terminal. [Figure 9] FIG. 10 is a diagram illustrating an example of an autonomous operation device operation screen displayed on a display unit of an operator terminal. [Figure 10] FIG. 10 is a diagram illustrating an example of operator information stored in a database of a cloud server. [Figure 11] 10 is a flowchart showing the procedure of a remote monitoring method performed in the second embodiment. [Figure 12] FIG. 2 is a diagram illustrating a hardware configuration of a computer. [Figure 13] FIG. 2 is a diagram illustrating a hardware configuration of a control computer provided in the autonomous operation device. DETAILED DESCRIPTION OF THE INVENTION

[0013] Next, modes for carrying out the present invention (referred to as "embodiments") will be described in detail with reference to the drawings as appropriate.

[0014] [First embodiment] <System configuration diagram> FIG. 1 is a diagram showing the configuration of an autonomous operation device monitoring system Z in the first embodiment.

[0015] In an autonomous operation device monitoring system Z, which is an autonomous operation device control system, a cloud server 3, operator terminals 1 owned by a plurality of operators P, and a plurality of autonomous operation devices 2 are each connected via a network N so that they can communicate with each other.

[0016] The cloud server 3, which is a server, has a server main body device 31 and a database (DB) 32, which is a storage unit.

[0017] The operator terminal 1 is an operation terminal that can be operated by an operator P. Furthermore, one operator P owns multiple operator terminals 1, such as an office PC, a home PC, and a mobile terminal 1a. In other words, there are multiple operator terminals 1. The mobile terminal 1a is, for example, a smartphone or a tablet terminal. The office PC is a PC that the operator P uses in the office, and the home PC is a PC that the operator P uses at home.

[0018] The operator P does not necessarily have to own all of the office PC, home PC, and mobile terminal 1a. Alternatively, the operator P may own only one operator terminal 1.

[0019] The autonomous operating device 2 is capable of autonomous operation according to a program. The autonomous operating device 2 can be remotely operated (operated) by multiple operators P, or can be operated by field workers (not shown). However, multiple operators P cannot operate one autonomous operating device 2 simultaneously.

[0020] When an event occurs that requires remote operation, the autonomous operating device 2 transmits a remote operation request D1 to the server main body device 31 of the cloud server 3. Upon receiving the remote operation request D1, the server main body device 31 transmits the remote operation request D1 simultaneously to all operator terminals 1 registered in the database 32.

[0021] The database 32 registers operator terminals 1 that can operate the autonomous operating devices 2. In the first embodiment, each of the operator terminals 1 registered in the database 32 can operate all of the autonomous operating devices 2 operating in the autonomous operating device monitoring system Z.

[0022] Each operator terminal 1 that has received the remote operation request D1 outputs a message indicating that the remote operation request D1 has been received. The output is a sound output from the speaker unit 103 (see FIG. 2) or a display on the display unit 102 (see FIG. 2).

[0023] Operator Pa, who notices the output of remote operation request D1, remotely operates autonomous operating device 2, where an event requiring remote operation has occurred, via operator terminal 1. Then, control information D2 for operating autonomous operating device 2 is transmitted to autonomous operating device 2 via network N. At this time, control information D2 is transmitted directly from operator terminal 1 to autonomous operating device 2 without going through cloud server 3.

[0024] The autonomous operating device 2 that has received the control information D2 operates in accordance with the received control information D2, thereby attempting to resolve the event that requires remote operation.

[0025] If the operator P has a mobile terminal 1a such as a smartphone as the operator terminal 1, the operator P can be aware of the remote operation request D1 even while on the move.

[0026] Furthermore, by having a plurality of operator terminals 1 that the operator P can operate, the operator P can remotely operate the autonomous operating device 2 from a plurality of locations such as a home or an office.

[0027] 1, cloud server 3 is used as a relay point for remote operation request D1, but a server other than cloud server 3 may be used. However, using cloud server 3 enables operators P around the world to remotely monitor autonomous operation devices 2 using autonomous operation device monitoring systems Z. Also, using cloud server 3 can reduce the effort and cost required to install a computer at each location.

[0028] <Function block diagram> 2 is a functional block diagram of the autonomous operation device monitoring system Z according to the first embodiment. FIG. 1 will be referred to as appropriate.

[0029] (Autonomous Operation Device 2) The autonomous operating device 2 includes an operation monitoring unit 203 , an operation request notification unit 201 , and a control unit 202 .

[0030] The operation monitoring unit 203 monitors the operation of the autonomous operating device 2 and monitors whether an event requiring remote operation has occurred. An event requiring remote operation is when the autonomous operating device 2 cannot recover autonomously due to some kind of malfunction, or when a predetermined task needs to be handled remotely and the time has come to perform that task, etc.

[0031] A malfunction occurring in the autonomous operating device 2 is an event that makes it impossible to operate according to the program. For example, when an obstacle makes it impossible to move or operate, the operation monitoring unit 203 determines that an event requiring remote operation has occurred. That is, the operation monitoring unit 203 monitors whether operation according to the program is being performed, and whether operation according to the program is possible (whether an obstacle exists). Then, when a discrepancy occurs between the operation according to the program and the actual operation, or when it is determined that operation according to the program is impossible due to an obstacle, the operation monitoring unit 203 determines that an event requiring remote operation due to a malfunction has occurred.

[0032] When an event occurs that requires remote operation, the operation request notification unit 201 transmits a remote operation request D1 to the cloud server 3 (the operation request broadcast unit 311 of the autonomous operation device management unit 300).

[0033] The control unit 202 controls the operation of the autonomously operating device 2 in accordance with control information D2 transmitted from the remote control unit 105 of the operator terminal 1.

[0034] (Cloud Server 3) The server main body device 31 provided in the cloud server 3 has an autonomous operation device management unit 300. The autonomous operation device management unit 300 has an operation request broadcast unit 311.

[0035] The operation request broadcast unit 311 receives a remote operation request D1, which is a request, from the operation request notification unit 201 of the autonomous operating device 2. Then, the operation request broadcast unit 311 simultaneously distributes (broadcasts) the remote operation request D1, which is a notification, to all operator terminals 1 registered in the database 32 of the cloud server 3. In this way, when the cloud server 3 accepts the remote operation request D1 from the autonomous operating device 2, it notifies the operator terminal 1.

[0036] (Operator terminal 1) Each operator terminal 1 includes an input unit 101 , a display unit 102 which is an output unit, a speaker unit 103 which is an output unit, a remote monitoring unit 104 , and a remote control unit 105 .

[0037] When the remote monitoring unit 104 receives the remote operation request D1 transmitted from the operation request broadcast unit 311 provided in the server main unit 31, it notifies the operator P that the remote operation request D1 has been received by sound or by output to the display unit 102 or speaker unit 103.

[0038] When the operator P operates the autonomous operation device 2 via the input unit 101, the remote control unit 105 transmits to the autonomous operation device 2 control information D2 corresponding to this operation.

[0039] Furthermore, the operator terminal 1 may be installed as a remote monitoring terminal 1A that includes the input unit 101 to the remote monitoring unit 104 but does not include the remote control unit 105. The remote monitoring terminal 1A is an operator terminal 1 that monitors whether or not a remote operation request D1 has been transmitted from the server main body device 31. Similarly, the operator terminal 1 may be installed as a remote operation terminal 1B that includes the input unit 101 to the speaker unit 103 and the remote operation unit 105 but does not include the remote monitoring unit 104. The remote operation terminal 1B is an operator terminal 1 that remotely controls the autonomous operating device 2.

[0040] <Autonomous Operation Device 2> 3A and 3B are diagrams showing the appearance of the autonomously operating device 2. FIG.

[0041] Fig. 3A shows a front view of the autonomously operated device 2, and Fig. 3B shows a side view of the autonomously operated device 2. Figs. 1 and 2 will be referred to as appropriate.

[0042] The autonomously operating device 2 is composed of a main body 21 and an arm 22. The tip of the arm 22 is equipped with an end effector 23 for grasping an object and a hand-eye camera 25. Images captured by the hand-eye camera 25 are displayed on the display 102 of the operator terminal 1 as needed. By checking the images captured by the hand-eye camera 25, the operator P can determine whether the autonomously operating device 2 is correctly grasping the object. Furthermore, if the end effector 23 fails to grasp the object, the operator P operates the operator terminal 1 based on the images captured by the hand-eye camera 25 to operate the autonomously operating device 2 by remote manual control (manual control).

[0043] Periphery monitoring camera 26 captures images of the environment surrounding autonomously operating device 2. Images captured by periphery monitoring camera 26 are displayed on the screen (display unit 102) of operator terminal 1. Operator P checks the images captured by periphery monitoring camera 26 to confirm the operating environment of autonomously operating device 2. Furthermore, if autonomously operating device 2 becomes unable to travel due to an obstacle or the like, operator P operates operator terminal 1 based on images captured by periphery monitoring camera 26 to operate autonomously operating device 2 by remote manual control (manual control).

[0044] The lower part of the main body 21 is provided with running wheels 24. The main body 21 also has a control computer 200 inside. The main body 21 also has an information acquisition computer 27.

[0045] The information acquisition computer 27 transmits and receives information to and from the server main body device 31 of the cloud server 3 and the operator terminal 1 via the network N. It also passes received information to the control computer 200 and transmits information passed from the control computer 200. Furthermore, the information acquisition computer 27 collects the status of the autonomously operating device 2 from the hand-eye camera 25, the surrounding monitoring camera 26, and various sensors (not shown).

[0046] The control computer 200 controls the movement, gripping, and other operations of the autonomous operating device 2 by performing processing when the autonomous operating device 2 operates autonomously and by performing processing based on commands received by the information acquisition computer 27. By providing the information acquisition computer 27 and the control computer 200 separately in this way, it is possible to reduce the risk of unexpected performance impairment due to, for example, modification of existing software of the autonomous operating device 2, which is an existing product. Note that the functions of the information acquisition computer 27 may be provided in the control computer 200, and the functions of the information acquisition computer 27 may be provided in the control computer 200. Furthermore, the information acquisition computer 27 may be configured as an antenna. Incidentally, the operation request notification unit 201 to the operation monitoring unit 203 shown in FIG. 2 correspond to the control computer 200.

[0047] 3A and 3B, the autonomous operating device 2 is assumed to be an autonomously mobile type equipped with running wheels 24, but is not limited to this. A stationary autonomous operating device 2 not equipped with a means of movement may also be used. Alternatively, even if a means of movement is provided, a walking autonomous operating device 2 having legs, such as bipedal or quadrupedal walking, may be used instead of running wheels 24. Furthermore, the autonomous operating device 2 shown in this embodiment performs work by grasping and moving an object with the end effector 23, but is not limited to this, and for example, a communication robot may also be used as the autonomous operating device 2.

[0048] (flowchart) FIG. 4 is a flowchart showing the procedure of the autonomous operation device control method performed in the first embodiment.

[0049] First, the operation monitoring unit 203 of the autonomous operation device 2 determines whether or not an event requiring remote operation has occurred (S101).

[0050] If the operation monitoring unit 203 determines that no event requiring remote control has occurred (S101→No), the operation monitoring unit 203 returns the process to step S101.

[0051] If the operation monitoring unit 203 determines that an event requiring remote control has occurred (S101→Yes), the operation monitoring unit 203 sends a remote control request D1 to the autonomous operation unit management unit 300 of the server main body device 31 (S102).

[0052] The operation request simultaneous transmission unit 311 of the autonomous operation device management unit 300 simultaneously transmits (broadcasts) the remote operation request D1 to all operator terminals 1 registered in the database 32 (S103: notification step).

[0053] The remote monitoring unit 104 of the operator terminal 1 that has received the remote operation request D1 outputs the fact that it has received the remote operation request D1 via the display unit 102 or the speaker unit 103 (S104: display step, output step). The output will be described later, but the remote monitoring unit 104 displays on the display unit 102 or issues a sound from the speaker unit 103.

[0054] The operator P notices the output in step S104 and confirms the remote operation request D1.

[0055] Then, an operator P who notices the output in step S104 remotely controls the autonomous operating device 2 via the input unit 101. In other words, one operator P who notices the output remotely controls the autonomous operating device 2 of his or her own will (S105).

[0056] When the operator P performs remote operation, control information D2 is transmitted from the remote control unit 105 of the operator terminal 1 operated by the operator P to the control unit 202 of the autonomous operation device 2.

[0057] The control unit 202 of the autonomous operating device 2 receives the control information D2 transmitted from the operator terminal 1 and operates in accordance with the control information D2 (S106). In this way, the operator P attempts to resolve the event by remote operation. The event is one that requires remote operation.

[0058] Note that control information D2 is sent directly from the autonomous operating device 2 to the operator terminal 1. In other words, the autonomous operating device management unit 300 (cloud server 3) does not manage the operation state of the autonomous operating device 2. This reduces the risk of inconsistencies in the operation state compared to when the autonomous operating device management unit 300 manages the operation state. For example, while the autonomous operating device 2 is being operated, communication may be interrupted between the autonomous operating device management unit 300 and the operator terminal 1, or between the autonomous operating device management unit 300 and the autonomous operating device 2. Even if such communication interruption occurs, the communication between the operator terminal 1 and the autonomous operating device 2 may remain connected.

[0059] In such a case, if the autonomous operating device 2 operation state is managed by the autonomous operating device management unit 300, a mismatch will occur between the actual operation state of the autonomous operating device 2 and the operation state managed by the autonomous operating device management unit 300. As described above, in this embodiment, control information D2 is sent directly from the autonomous operating device 2 to the operator terminal 1. In this way, the risk of a mismatch in the operation state can be reduced compared to when the autonomous operating device management unit 300 manages the operation state.

[0060] The operation monitoring unit 203 of the autonomous operating device 2 determines whether the event requiring remote operation has been resolved (S107). Specifically, the operation monitoring unit 203 determines whether operation according to the program has become possible, thereby determining whether the event requiring remote operation has been resolved.

[0061] If the operation monitoring unit 203 determines that the event requiring remote control has been resolved (S107→Yes), the operation monitoring unit 203 returns the process to step S101.

[0062] If the operation monitoring unit 203 determines that the event requiring remote operation has not been resolved (S107→No), the operation monitoring unit 203 notifies the on-site worker (S108). Specifically, the operation monitoring unit 203 notifies the on-site worker terminal (not shown), which is a terminal owned by the on-site worker.

[0063] The field worker who owns the field worker terminal that received the notification attempts to resolve the event by performing field work.

[0064] (Remote monitoring screen 400) 5 is a diagram showing an example of a remote monitoring screen 400 displayed on the operator terminal 1. FIGS. 1 and 2 will be referred to as appropriate.

[0065] The remote monitoring screen 400, which is a display screen, has an alarm icon 401, a display switching section 410, a main display section 420, an operator information display section 431, and a logout button 441.

[0066] The alert icon 401 is an icon indicating that a remote operation request D1 is being transmitted from the autonomous operating device 2. The alert icon 401 may be in a lit state or a flashing state. In addition, when the alert icon 401 is displayed, a sound may be output from the speaker unit 103 of the operator terminal 1.

[0067] The operator P can operate the display switching unit 410 to switch the content displayed on the main display unit 420 .

[0068] In the example shown in FIG. 5, a monitor display button 411, an autonomous operating device list display button 412, and an operator information display button 413 are displayed in the display switching section 410.

[0069] When operator P selects and inputs monitoring display button 411, remote monitoring display section 421 is displayed on main display section 420. When operator P selects and inputs autonomous operation device list display button 412, a list (not shown) of autonomous operation devices 2 managed by operator P is displayed on main display section 420. The autonomous operation devices 2 managed by operator P mean the autonomous operation devices 2 managed by operator P who is operating the operator terminal 1 on which remote monitoring screen 400 is displayed. In the first embodiment, the autonomous operation devices 2 managed by operator P are all of the autonomous operation devices 2 operating in autonomous operation device monitoring system Z.

[0070] Note that the selection input is a click, a double click, etc. when the operator terminal 1 is a PC or the like, or a tap, a double tap, etc. when the operator terminal 1 is a mobile terminal 1a such as a smartphone or a tablet terminal.

[0071] When the operator P selects and inputs the operator information display button 413, information about the operator P is displayed on the main display unit 420. The information about the operator P is the login information of the operator P, etc. The operator P can change the login password, etc. by editing the displayed information about the operator P.

[0072] The main display section 420 displays information based on the result of selection made by the display switching section 410 .

[0073] In the example shown in FIG. 5, a remote monitoring display section 421 is displayed when the monitoring display button 411 is selected and input in the display switching section 410.

[0074] Information relating to the autonomous operating device 2 in which an event requiring remote operation has occurred is displayed on the remote monitoring display unit 421. The remote monitoring display unit 421 has a time column 421a, an autonomous operating device ID column 421b, an event column 421c, a details column 421d, an operation information column 421e, and the like.

[0075] The time column 421a displays the time when an event occurred in the autonomous operating device 2 that required remote operation.

[0076] The autonomous operation device ID column 421b displays identification information assigned to each autonomous operation device 2 in which an event requiring remote operation has occurred.

[0077] The event column 421c displays the name of the event occurring in the autonomous operating device 2 as information relating to the state of the autonomous operating device 2. Furthermore, the details column 421d displays details of the event occurring in the autonomous operating device 2 as information relating to the state of the autonomous operating device 2. Based on the information displayed in the event column 421c and the details column 421d, the operator P determines whether or not the event is one that the operator P can handle with his or her own level of proficiency. Note that either the event column 421c or the details column 421d may be displayed.

[0078] The operation information field 421e displays information related to the operation of the autonomous operating device 2 and information related to the operation status of the autonomous operating device 2. The operation information field 421e displays, for example, an operation start button 422 as information related to the operation of the autonomous operating device 2. When the operator P selects and inputs the operation start button 422, operation of the autonomous operating device 2 having the autonomous operating device ID corresponding to the operation start button 422 is started. It is desirable to place the operation start button 422 in a position that is readily accessible when the operator P notices the remote operation request D1.

[0079] In this way, operation start button 422 is an indication for starting operation of the autonomous operating device 2.

[0080] Furthermore, the in-operation display 423, which is information regarding the operation status of the autonomous operation device 2, indicates that the corresponding autonomous operation device 2 is being operated by someone. In this way, the content displayed in the operation information field 421e changes when another operator P begins operation. When the in-operation display 423 is displayed, the operator P operating the operator terminal 1 displaying the remote monitoring screen 400 cannot operate the corresponding autonomous operation device 2. In other words, the autonomous operation device 2 can only be operated by one person, that is, by one operator terminal 1. The autonomous operation device 2 that receives the control information D2 transmits this information to the cloud server 3. Then, the cloud server 3 transmits a notification to all operator terminals 1 that the autonomous operation device 2 is being operated, including identification information of the autonomous operation device 2 being operated. The operator terminal 1 that receives this notification displays the in-operation display 423.

[0081] In this way, the remote monitoring screen 400 displays at least one of the event column 421c and the details column 421d, the operation start button 422, and the operation in progress display 423.

[0082] If no event requiring remote control has occurred, the remote monitoring display section 421 is not displayed.

[0083] The operator information display section 431 displays information about the operator P who is operating the operator terminal 1 that is displaying the remote monitoring screen 400.

[0084] When the operator P selects and inputs the logout button 441, the screen returns to the login screen (not shown). When the operator P logs in via the login screen (not shown), the remote monitoring screen 400 shown in Fig. 5 is displayed. However, the login screen may be omitted.

[0085] (Notification window 500) FIG. 6 is a diagram showing an example of a notification window 500 in the first embodiment.

[0086] 6, window 601 is used to perform processing other than the operation of the autonomous operation device 2. In other words, operator P uses window 601 to perform work other than the operation of the autonomous operation device 2.

[0087] 4, a sound is output from the speaker unit 103 of the operator terminal 1 as a predetermined output (reference numeral 511). Also, as a predetermined output, a notification window 500 is displayed on the display screen 600 of the operator terminal 1. In this way, when the remote monitoring unit 104 receives the remote operation request D1 from the cloud server 3, it performs a predetermined output from the output unit.

[0088] The notification window 500, which is a notification screen, is displayed small on the display screen 600 of the operator terminal 1. Displayed small on the display screen 600 means that the area of ​​the notification window 500 is smaller than the area of ​​the display screen 600 (the screen of the operating terminal). In this way, the notification window 500 is displayed in a part of the display screen 600 of the operator terminal 1.

[0089] As described above, when the remote monitoring unit 104 receives the broadcast remote control request D1, the output includes not only the notification window 500 but also a sound emitted from the speaker unit 103 (reference numeral 511).

[0090] As shown in FIG. 6, notification window 500 displays event information 522 as information relating to the state of the autonomous operating device 2. Event information 522 displays the name of the event occurring in the autonomous operating device 2 ("self-location lost" in the example shown in FIG. 6). Furthermore, notification window 500 displays remote start button 521 (a display for starting operation of the autonomous operating device 2) as information relating to operation of the autonomous operating device 2. In this way, notification window 500 displays at least event information 522 and remote start button 521.

[0091] Then, operator P selects and inputs notification window 500, or selects and inputs remote start button 521 displayed in notification window 500. When remote start button 521 is selected and input, remote control unit 105 of operator terminal 1 transmits an inquiry request to the autonomous operating device 2 ("Robot #1" in the example shown in FIG. 6) corresponding to remote start button 521. The content of the inquiry request is whether or not the autonomous operating device 2 that is the destination of the inquiry request is being operated by an operator P (another operator P) other than operator P that is the sender of the inquiry request.

[0092] The autonomous operating device 2 that has received the inquiry request transmits a response to the inquiry request to the operator terminal 1 that is the sender of the inquiry request. The content of the response is the status of the autonomous operating device 2 itself (whether or not it is being operated by an operator P other than the operator P that is the sender of the inquiry request).

[0093] Upon receiving the response, the operator terminal 1 displays the remote monitoring screen 400 shown in FIG. 5 or the autonomous operation device operation screen 700 shown in FIG.

[0094] If the response is "not being operated by another operator P," the autonomous operation device operation screen 700 shown in FIG. 9 is displayed on the display unit 102. This allows the operator P to immediately begin remotely operating the autonomous operation device 2.

[0095] If the response is "operated by another operator P," the remote control unit 105 of the operator terminal 1 displays a warning message saying "Another operator is operating" on the display unit 102. After that, the remote monitoring screen 400 shown in FIG. 5 is displayed on the display unit 102 of the operator terminal 1.

[0096] After remote start button 521 is selected and input, a login screen (not shown) may be displayed, and after logging in on the login screen, autonomous operating device operation screen 700 shown in Fig. 9 may be displayed. This allows operator P to start remote operation of the autonomous operating device 2.

[0097] As described above, the autonomous operation device 2 that has accepted an operation by the operator P transmits a message to that effect to the cloud server 3.

[0098] When no event requiring remote operation has occurred, the notification window 500 is not displayed, and when the operator terminal 1 receives the remote operation request D1, the notification window 500 is displayed. Alternatively, even when no event requiring remote operation has occurred, the notification window 500 may be displayed, but when the operator terminal 1 receives the remote operation request D1, the notification window 500 may be highlighted, for example, by flashing.

[0099] (Operator terminal 1 operation) FIG. 7 is a flowchart showing the details of the operation of the operator terminal 1 in the first embodiment.

[0100] The flowchart shown in Fig. 7 shows in detail the processing of steps S104 and S105 in Fig. 4. Fig. 2 and Fig. 6 will be referred to as appropriate.

[0101] The remote monitoring unit 104 is executed in the background on the operator terminal 1 (S201: background step).

[0102] Then, the remote monitoring unit 104 determines whether or not a remote control request D1 transmitted from the cloud server 3 (server main body device 31) has been received (S202). The determination in step S202 is made at predetermined time intervals.

[0103] If the remote control request D1 has not been received (S202→No), the remote monitoring unit 104 returns the process to step S201.

[0104] If the remote control request D1 has been received (S202→Yes), the remote monitoring unit 104 displays the notification window 500 shown in FIG. 6 and outputs sound from the speaker unit 103 (S203).

[0105] Next, the remote control unit 105 determines whether the remote start button 521 has been selected and input (S204).

[0106] If the remote start button 521 has not been selected (S204→No), the operator terminal 1 returns the process to step S203.

[0107] If the remote start button 521 is selected (S204→Yes), the remote operation unit 105 determines whether another operator P is operating the autonomous operating device 2 (S205). As described above, this determination is made by sending an inquiry request to the target autonomous operating device 2 and determining the content of the response to the inquiry request.

[0108] If another operator P is not operating the autonomous operating device 2 (S205→No), the remote operation unit 105 displays the autonomous operating device operation screen 700 shown in Fig. 9 on the display unit 102. Then, the autonomous operating device 2 is remotely operated (S207).

[0109] If another operator P is operating the autonomous operation device 2 (S205→Yes), the remote operation unit 105 displays the remote monitoring screen 400 shown in FIG. 5 (S206).

[0110] Then, the operator P selects and inputs the operation start button 422 (see FIG. 5) displayed on the remote monitoring screen 400, thereby remotely operating the autonomous operating device 2 (S207). Thereafter, the operator terminal 1 returns the process to step S201.

[0111] 7, steps S201 to S207 are performed on the same operator terminal 1, but steps S201 to S203 and steps S204 to S207 may be performed on different operator terminals 1. In other words, operator P who recognizes the notification window 500 may perform remote operation on an operator terminal 1 other than the operator terminal 1 displaying the notification window 500.

[0112] By displaying the notification window 500, the operator P can easily notice that the autonomous operation device 2 has transmitted the remote operation request D1.

[0113] (Remote monitoring screen 800) FIG. 8 is a diagram showing an example of a remote monitoring screen 800 when the operator terminal 1 is a mobile terminal 1a.

[0114] As mentioned above, the mobile terminal 1a refers to a smartphone, a tablet terminal, etc. In Fig. 8, it is assumed that the operator terminal 1 is a smartphone.

[0115] As shown in FIG. 8, a remote monitoring screen 800 displays a notification display 810 and an operation status display 820.

[0116] Notification display 810 is displayed when operator terminal 1 (portable terminal 1a) receives remote operation request D1 transmitted in step S103 of Fig. 4. Notification display 810 displays information about the autonomous operating device 2 in which an event requiring remote operation has occurred (autonomous operating device ID in the example shown in Fig. 8). Notification display 810 also displays the name of the event occurring in the autonomous operating device 2 (reference numeral 812) and the like as information about the state of the autonomous operating device 2. Operator P determines, based on this information, whether or not the event is one that operator P can handle with his or her own level of proficiency.

[0117] Furthermore, notification display 810 has a remote start button 811 that is information related to the operation of the autonomous operating device 2. When operator P selects and inputs remote start button 811, remote monitoring screen 400 shown in FIG. 5 or autonomous operating device operation screen 700 shown in FIG. 9 is displayed on display unit 102 of mobile terminal 1a. As in the case when remote start button 521 shown in FIG. 6 is selected and input, if another operator P is operating the target autonomous operating device 2, remote start screen 400 shown in FIG. 5 is displayed. If another operator P is not operating the target autonomous operating device, autonomous operating device operation screen 700 shown in FIG. 9 is displayed.

[0118] Whether remote monitoring screen 400 shown in Fig. 5 or autonomous operation device operation screen 700 shown in Fig. 9 is displayed is determined by the inquiry request to the autonomous operation device 2 and the content of the response to the inquiry request. This processing is the same as when remote start button 521 shown in Fig. 6 is selected and input.

[0119] Furthermore, when an operator P other than the operator P operating the operator terminal 1 remotely operates the autonomous operating device 2, an operation status display 820 is displayed, which is information relating to the operation status of the autonomous operating device 2. The operation status display 820 displays information relating to the autonomous operating device 2 (the autonomous operating device ID in the example shown in FIG. 8) and information relating to the operation status of the autonomous operating device 2. The information relating to the operation status of the autonomous operating device 2 includes information indicating that it is being remotely operated and information indicating that the remote operation has ended.

[0120] The operation status display 820 enables the operator P to recognize that he or she no longer needs to operate the autonomous operating device 2.

[0121] The process performed by the mobile terminal 1a is generally the same as that shown in FIG. 7, except that the notification window 500 in FIG. 7 is replaced with a notification display 810, and therefore will not be illustrated or described again.

[0122] By displaying the remote monitoring screen 800 on the display unit 102 of the portable terminal 1a, the operator P can easily notice that the autonomous operating device 2 has transmitted the remote operation request D1.

[0123] A link may be displayed instead of the remote start button 811. In this case, when the operator P selects and inputs the link, the remote monitoring screen 400 shown in Fig. 5 is displayed on the display unit 102 of the mobile terminal 1a.

[0124] (Autonomous operation device operation screen 700) 9 is a diagram showing an example of an autonomous operation device operation screen 700 displayed on the display unit 102 of the operator terminal 1. FIGS. 3A and 3B will be referenced as appropriate.

[0125] The autonomous operating device operation screen 700 is an operation screen for operating the autonomous operating device 2. The autonomous operating device operation screen 700 is displayed when the operator P selects and inputs the operation start button 422 in FIG. 5 via the input unit 101. Alternatively, the autonomous operating device operation screen 700 is displayed when the remote start button 521 shown in FIG. 6 is input and the autonomous operating device 2 corresponding to the remote start button 521 is not being operated by another operator P. Alternatively, the autonomous operating device operation screen 700 is displayed when the remote start button 811 shown in FIG. 8 is input and the autonomous operating device 2 corresponding to the remote start button 811 is not being operated by another operator P. In this manner, remote operation of the autonomous operating device 2 can be started promptly when the remote operation request D1 is received.

[0126] In this way, when operator P selects and inputs operation start button 422 in Figure 5 via input unit 101, autonomous operation device operation screen 700 is displayed. In this way, operation start button 422 in Figure 5 is a display for starting operation of the autonomous operation device 2.

[0127] Furthermore, as described above, autonomous operating device operation screen 700 is displayed when remote start button 521 shown in FIG. 6 is pressed and the autonomous operating device 2 corresponding to remote start button 521 is not being operated by another operator P. Alternatively, autonomous operating device operation screen 700 is displayed when remote start button 811 shown in FIG. 8 is pressed and the autonomous operating device 2 corresponding to remote start button 811 is not being operated by another operator P. In this way, remote start button 422 in FIG. 6 and remote start button 811 in FIG. 8 are also displays for starting operation of the autonomous operating device 2.

[0128] As shown in FIG. 9, an autonomous operation device operation screen 700 has a map display section 711, an image display section 712, a comment display section 730, a jog operation button 720, and an operation target selection radio button 721.

[0129] The map display unit 711 displays a map of the area in which the autonomous device 2 operates autonomously. The image display unit 712 displays an image captured by the periphery monitoring camera 26 provided in the autonomous device 2. In the example shown in FIG. 9 , in this embodiment, the image displayed on the image display unit 712 is an image captured by the periphery monitoring camera 26, but is not limited to this. Images captured by the hand-eye camera 25 (see FIGS. 3A and 3B ) provided in the autonomous device 2, a rear camera (not shown), an image captured by a camera (not shown) provided in the vehicle interior, and the like may also be displayed on the image display unit 712. The map display unit 711 and the image display unit 712 can be switched to other screens by the display image selection unit 713. In the example shown in FIG. 9 , only the map display unit 711 and the image display unit 712 are shown; however, for example, an image from the hand-eye camera 25 may be displayed in addition to these images. If three images are displayed, the autonomous device operation screen 700 may become cluttered. In such a case, the operator P can select and input the display image selection section 713 to display the desired image in the foreground.

[0130] The jog operation button 720 is a button for performing a jog operation. A jog operation is an operation of moving the autonomous operating device 2 in small increments. By pressing the jog operation button 720, the operator P can manually move the autonomous operating device 2 in small increments.

[0131] The operation target selection radio button 721 is a radio button for selecting the target for performing a jog operation. For example, if the arm unit 22 is selected as the operation target, the operator P can move the end effector 23 in the X-axis direction by pressing "X-" or "X+" shown in FIG. 9. Similarly, the operator P can move the end effector 23 in the Y-axis direction by pressing "Y-" or "Y+" shown in FIG. 9. The X-axis and Y-axis are the X-axis and Y-axis set on the map. Then, the operator P can move the end effector 23 in the Z-axis direction by pressing "Z-" or "Z+" shown in FIG. 9.

[0132] Furthermore, when the operator P presses the "A-" or "A+" button, the end effector 23 can be rotated in the roll direction. Similarly, when the operator P presses the "B-" or "B+" button, the end effector 23 can be rotated in the pitch direction. Similarly, when the operator P presses the "C-" or "C+" button, the end effector 23 can be rotated in the yaw direction.

[0133] When the dolly (traveling wheels 24) is selected as the operation target using the operation target selection radio button 721, the operator P can move the autonomous operating device 2 by jogging. In this case, the jog operation button 720 may be redisplayed as buttons for only the directions in which the autonomous operating device 2 can move, such as the X direction, Y direction, and yaw direction.

[0134] The comment display section 730 displays the current state of the autonomous device 2. In the example shown in Fig. 9, a comment is displayed to the effect that the autonomous device 2 has started up normally.

[0135] As described above, the autonomously operated device 2 operates autonomously, but manual control is performed by remote operation when avoiding an obstacle or when the autonomously operated device 2 is unable to operate autonomously due to an error.

[0136] 9 assumes that the operator terminal 1 is a PC or the like. By displaying the autonomous operating device operation screen 700 on the display unit 102 of a PC, operability can be improved compared to when the autonomous operating device operation screen 700 is displayed on a mobile terminal 1a such as a smartphone.

[0137] When the operator terminal 1 is a mobile terminal 1a (see FIG. 1) such as a smartphone, and the autonomous operation device operation screen 700 is displayed on the display unit 102, only the map display section 711 and the jog operation button 720 may be displayed.

[0138] As described above, the technology described in Patent Document 1 is premised on the premise that an operator is available to respond to calls from the robot. Therefore, if the operator is away from their desk for other business, they cannot respond to calls from the robot.

[0139] Thus, whether the operator P is currently in a situation where they can remotely operate the autonomously operating device (whether they are free) is an important criterion for selecting an operator. Furthermore, since the operator P is not always in front of the operator terminal 1, there is a possibility that the operator P will not notice even if a notification is sent to the connected operator terminal 1. Furthermore, the operator P will not always use the same operator terminal 1. Therefore, for efficient operation of the autonomously operating device 2, it is also essential to enable a specific operator to respond to a specific autonomously operating device 2 even if a different operator terminal 1 is used. Patent documents 1 to 3 fail to take these issues into consideration.

[0140] According to the first embodiment, a remote operation request D1 is simultaneously transmitted to all operator terminals 1 that can operate the autonomous operating device 2. As a result, an operator P who notices the remote operation request D1 can operate the autonomous operating device 2. Therefore, even if an operator P is away from his or her desk for other business, a prompt response to an event that requires remote operation is possible.

[0141] 8, by displaying a notification display 810 on the display unit 102 of the portable terminal 1a, the operator P can easily notice that the remote operation request D1 has been sent even when he or she is away from his or her desk. Furthermore, because the remote operation request D1 is sent to all of the multiple operator terminals 1 owned by the operator P, a specific operator can handle a specific autonomous operating device 2 even if the operator P is using a different operator terminal 1.

[0142] 6, by displaying the notification window 500, the operator P can easily notice that the remote operation request D1 has been transmitted, even if the operator P is performing a task other than the operation of the autonomous operating device 2 on the operator terminal 1. This eliminates the need for the operator P to constantly be concerned about the transmission of the remote operation request D1, and allows the operator P to concentrate on the task other than the operation of the autonomous operating device 2. Patent Documents 1 to 3 do not take into consideration the situation when the operator P is performing a task other than the operation of the autonomous operating device 2.

[0143] Then, the notification window 500 is displayed in a part of the display screen 600 of the operator terminal 1. As a result, even if the notification window 500 is displayed while the operator P is performing a task on the operator terminal 1 other than operating the autonomous motion device 2, the task is not interrupted. In addition to the notification window 500, a sound is emitted from the speaker unit 103. This makes it even easier for the operator P to notice that the remote operation request D1 has been transmitted.

[0144] Notification window 500 displays at least event information 522 and remote start button 521. This allows operator P to understand what event is occurring, and by selecting and inputting remote start button 521, remote monitoring screen 400 in Fig. 5 or autonomous operation device operation screen 700 in Fig. 9 is displayed. This allows remote operation of the autonomous operation device 2 to be started promptly.

[0145] In this way, according to the first embodiment, efficient remote operation of the autonomous operation device 2 can be realized.

[0146] [Second embodiment]

[0147] (Operator information 321) 10 is a diagram showing an example of operator information 321 stored in database 32 of cloud server 3. FIG. 1 will be referred to as appropriate. 10, operator information 321, which is correspondence information, associates the ID of the autonomous operating apparatus 2, which is information related to the autonomous operating apparatus 2, with the ID of operator P, who is in charge of operating the autonomous operating apparatus 2. In this way, operator information 321 is correspondence information between the autonomous operating apparatus 2 and operator P.

[0148] Since the cloud server 3 has the operator information 321 in this way, the cloud server 3 has a database 32a storing the operator information 321 as a means for managing the operator terminal 1. In the operator information 321, an operator P who can remotely operate a specific autonomous operating device 2 is registered in advance. In this manner, the autonomous operating device management unit 300 of the server main body device 31 extracts an operator P who can operate the autonomous operating device 2 in which an event requiring remote operation has occurred. Furthermore, the cloud server 3 extracts an operator terminal 1 to which an operator P who can operate the autonomous operating device 2 in which an event requiring remote operation has occurred is logged in. The extracted operator terminal 1 is the operator terminal 1 that can operate the autonomous operating device 1. In this way, a responsible operator P is determined for each autonomous operating device 2. The ID of the operator P in the operator information 321 is information about the operator terminal 1 that can operate the autonomous operating device 1.

[0149] (flowchart) FIG. 11 is a flowchart showing the procedure of the remote monitoring method performed in the second embodiment.

[0150] In FIG. 11, the same processes as those in FIG. 4 are denoted by the same step numbers as those in FIG. 4, and the description thereof will be omitted.

[0151] The autonomous operation unit management unit 300 of the server main body device 31, which received the remote operation request D1 transmitted in step S103, references the operator information 321 stored in the database 32. The autonomous operation unit management unit 300 then extracts an operator P who can operate the autonomous operation unit 2 that transmitted the remote operation request D1 (S301). In step S301, when the autonomous operation unit management unit 300 receives the remote operation request D1 from the autonomous operation unit 2, it selects, based on the operator information 321, the operator P corresponding to the autonomous operation unit 2 that transmitted the remote operation request D1. The remote operation request transmitted in step S102 of FIG. 4 includes the ID of the logged-in operator P and the ID of the operator terminal 1 that transmitted the remote operation request.

[0152] The remote control request D1 transmitted in step S103 contains information (such as an ID) about the autonomous operating device 2 that is the source of the remote control request D1.

[0153] Then, the autonomous operation unit management unit 300 of the server main body device 31 extracts the operator terminal 1 to which the operator P extracted in step S301 is logged in. Then, the autonomous operation unit management unit 300 simultaneously transmits a remote operation request D1 to all of the extracted operator terminals 1 (performs notification: S103a).

[0154] According to the second embodiment, the remote operation request D1 is also transmitted to all operator terminals 1 that can operate the autonomous operating device 2. Therefore, as in the first embodiment, an operator P who notices the remote operation request D1 can operate the autonomous operating device 2. Therefore, even if an operator P is away from his or her desk for business, a prompt response to an event that requires remote operation is possible.

[0155] When the autonomous operation device monitoring system Z becomes large-scale and the number of operator terminals 1 and autonomous operation devices 2 increases, it is not efficient to send remote operation requests D1 to all of the operator terminals 1. In such cases, by grouping the autonomous operation devices 2 by the operator terminals 1 that can operate them, as in the second embodiment, efficient remote operation of the autonomous operation devices 2 becomes possible even when the autonomous operation device monitoring system Z becomes large-scale.

[0156] It is also conceivable that multiple companies may use the autonomous operation device monitoring system Z. In such a case, in FIG. 10 , the autonomous operation device 2 and the operator terminal 1 may be associated with each other for each company. In this way, even if multiple companies use the autonomous operation device monitoring system Z, it is possible to avoid cross-talk between the autonomous operation devices 2 and the operator terminal 1 between the companies.

[0157] 10 associates the autonomous operating device 2 with information about the operator P. In this way, the operator P can operate the autonomous operating device 2 by logging in to any operator terminal 1 (which may be an operator terminal 1 owned by another person). However, in the operator information 321, the autonomous operating device 2 may also be associated with information about an operator terminal 1 that can operate the autonomous operating device 2.

[0158] [Hardware configuration diagram] FIG. 12 is a diagram illustrating the hardware configuration of the computer C. As shown in FIG.

[0159] The computer C corresponds to the operator terminal 1 and the server main body device 31.

[0160] The computer C includes a memory 121 , an arithmetic unit 122 , a storage device 123 , an input device 124 , a display device 125 , a speaker 126 , and a communication device 127 .

[0161] The memory 121 is configured with a RAM (Random Access Memory), etc. The arithmetic device 212 is configured with a CPU (Central Processing Unit), a GPU (Graphic Processing Unit), etc. The storage device 123 is configured with an SSD (Solid State Drive), an HDD (Hard Disk Drive), etc.

[0162] A program stored in storage device 123 is loaded into memory 121, and the loaded program is executed by arithmetic unit 212. This embodies remote monitoring unit 104, remote operation unit 105, autonomous operation device management unit 310, and operation request broadcast unit 311 shown in FIG.

[0163] The input device 124 corresponds to the input unit 101 in Fig. 2 and is composed of a keyboard, a mouse, and a touch panel display of a smartphone. The display device 125 corresponds to the display unit 102 in Fig. 2 and is composed of a monitor and a touch panel display of a smartphone. The speaker 126 corresponds to the speaker unit 103 in Fig. 2.

[0164] The communication device 127 communicates with other devices via the network N.

[0165] FIG. 13 is a diagram showing the hardware configuration of the control computer 200 provided in the autonomous operation device 2. As shown in FIG.

[0166] The control computer 200 includes a memory 211, a computing device 212, a communication device 227, and the like.

[0167] The memory 211 is composed of a ROM (Read Only Memory) etc., and the arithmetic unit 212 is composed of a CPU etc. The program stored in the memory 211 is executed by the arithmetic unit 212. As a result, the operation request notification unit 201 to the operation monitoring unit 203 shown in FIG. 2 are realized.

[0168] The communication device 227 communicates with the operator terminal 1 and the server main body device 31 via the network N. [Explanation of symbols]

[0169] 1 Operator terminal (operation terminal) 1A Remote monitoring terminal (operation terminal) 1B Remote control terminal (operation terminal) 1a Mobile terminal (operation terminal) 2 Autonomous Operation Device 3 Cloud Server (Server) 31 Server main unit 32 Database (storage section) 101 Input section 102 Display unit (output unit) 103 Speaker section (output section) 104 Remote Monitoring Unit 105 Remote control unit 200 Control computer 201 Operation request notification section 202 Control section 203 Operation monitoring section 310 Autonomous operation device management section 311 Operation Request Broadcasting Unit 321 Operator Information (Support Information) 400 Remote monitoring screen (display screen) 401 alarm icon 410 Display switching unit 411 Monitoring display button 412 Autonomous operation device list display button 413 Operator information display button 420 Main display 421 Remote monitoring display unit 421a Time field 421b Autonomous operation device ID column 421c Events column (displays information about the status of the autonomous device) 421d Details (displays information about the status of the autonomous device) 421e Operation information column (displays information about the operation of the autonomous operating device and information about the operating status of the autonomous operating device) 422 Operation start button (information about the operation of the autonomous operation device, display for starting the operation of the autonomous operation device) 423 Operational Indicator (Information about the operational status of the autonomous operating device) 431 Operator information display section 441 Logout button 500 Notification window (predetermined output, notification screen) 511 Code (predetermined output) 521 Remote Start Button 522 Event Information 600 Display screen (screen of operation terminal) 700 Autonomous operation device operation screen (operation screen for operating the autonomous operation device) 800 Remote monitoring screen 810 Notification display 811 Remote Start Button (Information on the Operation of Autonomous Operation Devices) 812 Code (Information about the state of the autonomous operating device) 820 Operational Status Indicator (Information about the operational status of the autonomous operating device) D1 Remote control request (request, notification) D2 Control Information P Operator Z Autonomous Operation Device Monitoring System (Autonomous Operation Device Control System) S103: Send remote control to all operator terminals simultaneously (notification step) S104: Output that remote control has been received (display step, output step) S201 Run the remote monitoring unit in the background (background step)

Claims

1. an autonomously operated device capable of autonomous operation and operable by a plurality of operators via operation terminals; A server; and There are a plurality of the operation terminals, The server When a request is received from the autonomous operation device, a notification is sent to the operation terminal; The display screen of the operation terminal displays: At least information relating to the state of the autonomous operation device, information relating to the operation state of the autonomous operation device, and information relating to the operation of the autonomous operation device are displayed. Autonomous operating device control system.

2. the information regarding the operation of the autonomous operation device is a display for starting the operation of the autonomous operation device; When the operator selects and inputs a display for starting operation of the autonomous operation device via an input unit, an operation screen for operating the autonomous operation device is displayed.

2. The autonomous operation device control system according to claim 1.

3. an autonomously operated device capable of autonomous operation and operable by a plurality of operators via operation terminals; A server; and There are a plurality of the operation terminals, The server When a request is received from the autonomous operation device, a notification is sent to the operation terminal; a remote monitoring unit is executed in the background of the operation terminal; When the remote monitoring unit receives the notification from the server, the remote monitoring unit performs a predetermined output from an output unit. Autonomous operating device control system.

4. The predetermined output is A notification screen is displayed on a part of the screen of the operation terminal.

4. The autonomous operation device control system according to claim 3.

5. The predetermined output is In addition to the notification screen, the sound emitted from the speaker unit is included.

5. The autonomous operation device control system according to claim 4.

6. The notification screen includes: At least information about the state of the autonomous operating device and information about the operation of the autonomous operating device are displayed.

5. The autonomous operation device control system according to claim 4.

7. the information regarding the operation of the autonomous operation device is a display for starting the operation of the autonomous operation device; When the operator selects and inputs a display for starting operation of the autonomous operation device via an input unit, an operation screen for operating the autonomous operation device is displayed.

7. The autonomous operation device control system according to claim 6.

8. The server Correspondence information in which information about the autonomous operating device and information about the operating terminal capable of operating the autonomous operating device are associated with each other is stored in a storage unit, When the request is received from the autonomous operating device, the operation terminal that can operate the autonomous operating device that sent the request is selected based on the correspondence information; The notification is sent to the selected operating terminal.

4. The autonomous operation device control system according to claim 1 or 3.

9. an autonomously operated device capable of autonomous operation and operable by a plurality of operators via operation terminals; A server; and There are a plurality of the operation terminals, The server: a notification step of issuing a notification to the operation terminal when a request is received from the autonomous operation device; The operation terminal, A display step is carried out in which at least information regarding the state of the autonomous operation device, information regarding the operation state of the autonomous operation device, and information regarding the operation of the autonomous operation device are displayed on a display unit. Method for controlling an autonomously operating device.

10. an autonomously operated device capable of autonomous operation and operable by a plurality of operators via operation terminals; A server; and There are a plurality of the operation terminals, The server a notification step of issuing a notification to the operation terminal when a request is received from the autonomous operation device; The operation terminal, a background step that executes the remote monitoring unit in the background; an output step in which the remote monitoring unit performs a predetermined output from an output unit when receiving the notification from the server; Run Method for controlling an autonomously operating device.

Images