Management device, system, information processing method and program

Abstract

To provide a mechanism that enables alert processing according to priority.SOLUTION: A management device includes a control unit that determines whether to activate an action plan based on sensor data acquired by a terminal device and controls an operator terminal to present the action plan that has been determined to have been activated, and the control unit controls the operator terminal to present the action plan with the highest priority when it is determined that multiple action plans should be activated.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a management device, a system, an information processing method, and a program. [Background technology]

[0002] In recent years, technology that uses robots to perform various tasks has been expected as a means of resolving labor shortages associated with a declining population. One example is a robot system in which an operator makes various decisions based on information collected by the robot and remotely operates the robot. In this robot system, when an alert is determined to have been generated from data collected by the robot, the operator checks the content of the alert and causes the robot to perform the next action. As a technology for supporting operators in handling alerts, for example, Patent Document 1 listed below discloses a technology that supports operators in call centers in taking appropriate action in response to incoming calls. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 6996129 Summary of the Invention [Problem to be solved by the invention]

[0004] However, in the technology disclosed in the above-mentioned Patent Document 1, each incoming call is assigned to one operator in turn. In other words, since the priority of the alert is not taken into consideration, there is a risk that the processing of an alert that should be handled immediately will be postponed.

[0005] The present invention has been made in view of the above problems, and an object of the present invention is to provide a mechanism that enables alert processing according to priority. [Means for solving the problem]

[0006] In order to solve the above problem, according to one aspect of the present invention, a management device is provided which includes a control unit that determines the activation of an action plan based on sensor data acquired by a terminal device and controls an operator terminal to present the action plan that has been determined to have been activated, and when the activation of multiple action plans has been determined, the control unit controls the operator terminal to present the action plan with the highest priority.

[0007] The control unit may control the operator terminal to switch the action plan to be presented to the action plan with the higher priority when it is determined that an action plan with a higher priority than the action plan currently being presented by the operator terminal should be activated.

[0008] When processing of the action plan is completed, the control unit may control the operator terminal to present the action plan with the highest priority among the other action plans for which processing has not yet been completed.

[0009] When an instruction to switch the action plan to be presented is input to the operator terminal, the control unit may control the operator terminal to switch the action plan to be presented in accordance with the switching instruction.

[0010] When the control unit switches the action plan to be presented in response to the switching instruction, the control unit may lower the priority of the action plan before the switching and / or raise the priority of the action plan after the switching.

[0011] The control unit may perform control so that the operator terminal of an operator who has been called less frequently is given priority in presenting the action plan.

[0012] In addition, in order to solve the above problem, according to another aspect of the present invention, a system is provided, comprising: a terminal device that acquires sensor data; an operator terminal operated by an operator; and a management device that determines whether to activate an action plan based on the sensor data and controls the operator terminal to present the action plan that has been determined to have been activated, wherein when it is determined that multiple action plans should be activated, the management device controls the operator terminal to present the action plan with the highest priority.

[0013] When it is determined that a plurality of the action plans are to be activated, the operator terminal may display a list of the plurality of action plans in order of priority.

[0014] The terminal device may be a robot capable of autonomous movement.

[0015] In addition, in order to solve the above-mentioned problem, according to another aspect of the present invention, there is provided an information processing method executed by a computer, which includes determining the activation of an action plan based on sensor data acquired by a terminal device, and controlling an operator terminal to present the action plan that has been determined to have been activated, wherein controlling the operator terminal includes, when the activation of multiple action plans has been determined, controlling the operator terminal to present the action plan with the highest priority.

[0016] In addition, in order to solve the above problem, according to another aspect of the present invention, a program is provided that causes a computer to function as a control unit that determines the activation of an action plan based on sensor data acquired by a terminal device and controls an operator terminal to present the action plan that has been determined to have been activated, and when it has been determined that multiple action plans should be activated, the control unit controls the operator terminal to present the action plan with the highest priority. [Effects of the Invention]

[0017] As described above, according to the present invention, a mechanism is provided that enables alert processing according to priority. [Brief explanation of the drawings]

[0018] [Figure 1] 1 is a block diagram showing an example of the configuration of a system 1 according to an embodiment of the present invention. [Figure 2] FIG. 10 is a diagram showing an example of a restricted area. [Figure 3] FIG. 2 is a diagram showing an example of a UI screen displayed on an operator terminal 20 according to the present embodiment. [Figure 4] FIG. 2 is a diagram showing an example of a UI screen displayed on an operator terminal 20 according to the present embodiment. [Figure 5] 10 is a flowchart showing an example of the flow of processing executed by the system 1 according to the present embodiment. [Figure 6] 10 is a flowchart showing an example of the flow of processing executed by the system 1 according to the present embodiment. [Figure 7] 10 is a flowchart showing an example of the flow of processing executed by the system 1 according to a modified example. [Figure 8] FIG. 2 is a block diagram showing an example of a hardware configuration of the information processing device according to the present embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0019] Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In this specification and drawings, components having substantially the same functional configurations are designated by the same reference numerals, and redundant explanations will be omitted.

[0020] <1. Overview> An outline of one embodiment of the present invention will be described below with reference to FIG.

[0021] 1 is a block diagram showing an example of the configuration of a system 1 according to an embodiment of the present invention. As shown in FIG. 1, the system 1 includes a plurality of robots 10, a plurality of operator terminals 20, and a management device 30.

[0022] In the following description, it is assumed that the system 1 performs monitoring operations for a facility, such as a company building, a commercial facility, or a warehouse.

[0023] The robot 10 is a terminal device that executes a predetermined operation or task (hereinafter, these are also collectively referred to as an action). Examples of the actions include moving, stopping, waiting, and playing audio. The robot 10 may be capable of autonomous movement. For example, the robot 10 is placed on each floor of a facility to be monitored, and autonomously patrols the floor, acquiring various data such as camera images, and transmitting the data to the management device 30.

[0024] Here, the robot 10 may be interpreted as an "artificial human," "something that performs human-like movements and tasks," "a device that performs movements and tasks based on instructions and is operated by a computer," or "a device that performs autonomous movements and tasks and is operated by a computer." Alternatively, the robot 10 may be interpreted as "a machine for industrial use that has an automatically controlled manipulation function or a movement function, can perform various tasks by program, and is capable of being used" (JIS B 0134-1998). Alternatively, the robot 10 may be interpreted as "a robot that serves humans" (JIS B 0187:2005). Alternatively, the robot 10 may be interpreted as "an intelligent mechanical system that has three elemental technologies: a sensor, an intelligence / control system, and a drive system" (Robot Policy Study Group, Ministry of Economy, Trade and Industry).

[0025] The operator terminal 20 and the management device 30 are placed in a management center 2 and are communicatively connected to the robot 10 via a GW (Gateway) 3. The management center 2 may be placed, for example, in a remote location separate from the facility to be monitored. The communication path between the robot 10 and the operator terminal 20 or the management device 30 may conform to any wired / wireless communication standard such as the Internet or a local network.

[0026] The management device 30 is an information processing device that controls alert processing. Specifically, the management device 30 determines whether an alert has been issued based on data collected from the robot 10, and controls the execution of an action in response to the issued alert.

[0027] When an alert occurs, an action plan corresponding to the alert is activated, and the actions included in the action plan are selectively executed according to instructions from an operator. An action plan is a collection of one or more actions that can be executed in response to an alert.

[0028] Here, an alert may be considered to be a problem to be solved. An action plan may be considered to be a set of one or more actions that can be executed to solve the problem related to the alert. Executing the actions included in an action plan to solve the problem is also referred to as "handling the alert" or "handling the action plan." The occurrence of an alert is synonymous with the activation of an action plan.

[0029] The operator terminal 20 is a remote console for remotely operating the robot 10. The operator terminal 20 accepts an operation by an operator to process an activated action plan. Specifically, the operator terminal 20 displays a UI screen that presents the content of the generated alert, i.e., the content of the activated action plan, and accepts an operation to select an action to be executed from a selection of multiple actions included in the action plan. The action selected by the operator is then executed.

[0030] The actions included in the action plan are typically executed by the robot 10. Alternatively, the actions included in the action plan may be executed by a device other than the robot 10, such as the operator terminal 20 or the management device 30.

[0031] The operator terminal 20 may be operated by an administrator who manages the operations of the operators, in addition to the operators themselves. The administrator operates the operator terminal 20 on behalf of the operators as necessary.

[0032] Various alerts may occur simultaneously in the above-described system 1. For example, an alert indicating that the battery of the robot 10 is low and an alert indicating that a suspicious person has been detected may occur simultaneously while the robot 10 is patrolling.

[0033] In this case, it is desirable to interrupt and process multiple alerts that occur simultaneously according to priority. For example, if a suspicious person detection alert occurs immediately after a low battery alert occurs, it is desirable to suspend processing of the low battery alert and give priority to processing of the suspicious person detection alert.

[0034] However, in the past, it was difficult to process alerts according to priority. For example, when multiple alerts occurred simultaneously, the decision of which alert to prioritize was left up to the operator. As a result, operators who were unfamiliar with alert processing made incorrect judgments about priority, putting off the processing of alerts that should have been prioritized, leading to various inconveniences such as longer resolution times.

[0035] To solve this problem, it is conceivable to have multiple operators share the task of processing multiple alerts. However, in this case, multiple operators may instruct different actions on a single robot 10, which may cause inconsistencies and other inconveniences.

[0036] Therefore, the system 1 according to the present embodiment switches the action plan presented to the operator depending on the priority of the activated action plan, thereby realizing alert processing according to priority. As a result, it becomes possible to standardize and improve the quality of alert processing by the operator.

[0037] <2.Configuration example> Below, an example configuration of the system 1 will be described in detail with reference to FIG. 1 again.

[0038] <2.1. Example of Robot 10 Configuration> As shown in FIG. 1, the robot 10 has a sensor unit 11, a moving unit 12, an action unit 13, an operation unit 14, a communication unit 15, a memory unit 16, and a control unit 17.

[0039] The sensor unit 11 includes various sensors and acquires and outputs various types of information. The sensor unit 11 may include, for example, a camera, a microphone, a thermometer, a barometer, etc., and acquire various types of information within the facility to be monitored. The sensor unit 11 may include a position information sensor, an acceleration sensor, an angular velocity sensor, a battery sensor, etc., and acquire information indicating the state of the robot 10. The position information sensor may be a GNSS (Global Navigation Satellite System) receiver, or may acquire position information of the robot 10 by estimating its own position on an environmental map based on camera images, etc.

[0040] The moving unit 12 is a device that moves the robot 10. The moving unit 12 includes, for example, a motor and wheels, and moves the robot 10 by driving the wheels with the motor.

[0041] The action unit 13 is a device that exerts various actions on the surroundings of the robot 10. As one example, the action unit 13 may include an audio output device such as a speaker and play audio. As another example, the action unit 13 may include a display device such as a display and display an image. As another example, the action unit 13 may include a robot arm and pick up fallen trash or press a switch on a wall.

[0042] The operation unit 14 is a device that accepts user operations. As an example, the operation unit 14 may be configured with a touch panel. The operation unit 14 accepts operations for maintenance of the robot 10, for example.

[0043] The communication unit 15 is a device that communicates with other devices. The communication unit 15 transmits and receives information to and from other robots 10, the operator terminal 20, or the management device 30, for example.

[0044] The storage unit 16 stores various information for the operation of the robot 10. For example, the storage unit 16 stores map information of the facility to be monitored, and temporarily stores sensor data acquired by the sensor unit 11.

[0045] The control unit 17 controls the overall operation of the robot 10. The control unit 17 may be configured with AI (Artificial Intelligence). For example, the control unit 17 determines the surrounding situation surrounding the robot 10 based on sensor data acquired by the sensor unit 11. Then, the control unit 17 controls the operation of the various components of the robot 10 described above according to the determined surrounding situation. Specifically, the control unit 17 controls the movement unit 12 to move autonomously according to the determined surrounding situation, and controls the action unit 13 to perform an operation instructed by the management device 30.

[0046] 2.2. Example of configuration of operator terminal 20 As shown in FIG. 1, the operator terminal 20 includes a communication unit 21, a display unit 22, and an operation unit 23.

[0047] The communication unit 21 is a device that communicates with other devices. For example, the communication unit 21 transmits and receives information to and from the robot 10 or the management device 30. In addition, the communication unit 21 may perform communication to contact various personnel such as facility managers or security guards at the facility to be monitored. The facility managers are personnel who manage the facilities at the facility to be monitored. The security guards are personnel who guard the facility to be monitored.

[0048] The display unit 22 outputs information to the operator. The display unit 22 displays, for example, a UI screen for processing an action plan.

[0049] The operation unit 23 receives operations from an operator, such as input to a UI screen displayed on the display unit 22.

[0050] 2.3. Example of configuration of management device 30 As shown in FIG. 1, the management device 30 includes a communication unit 31, a storage unit 32, and a control unit 33.

[0051] (1) Communications Department 31 The communication unit 31 is a device that communicates with other devices. The communication unit 31 transmits and receives information to and from the robot 10 or the operator terminal 20, for example.

[0052] (2) Storage section 32 The storage unit 32 stores various types of information for the operation of the management device 30. For example, the storage unit 32 stores a database that stores various types of information related to the monitoring work of the facility. An example of the database stored in the storage unit 32 will be described below.

[0053] (Robot table) The robot table is a database that stores various information related to the robot 10. Table 1 below is an example of the robot table.

[0054] [Table 1]

[0055] The robot ID shown in Table 1 is identification information for uniquely identifying the robot 10.

[0056] The current position is information indicating the current horizontal position of the robot 10, and is expressed by, for example, XY coordinates.

[0057] The current floor is information indicating the current position of the robot 10 in the height direction, and is expressed, for example, by the number of floors in the facility to be monitored.

[0058] The current action plan ID is identification information of the action plan in which the robot 10 is currently involved.

[0059] The current action is the content of the action that the robot 10 is currently executing.

[0060] When the robot 10 executes an action included in an action plan, the memory unit 32 writes the action plan ID of the action plan in the current action plan ID. Then, the memory unit 32 writes the name of the action to be executed in the current action. The memory unit 32 may erase the current action after the action is completed. The memory unit 32 may also erase the current action plan ID when processing of the action plan is completed.

[0061] (No-go area table) The restricted area table is a database that stores areas that are restricted from entry by the robot 10. Table 2 below is an example of the restricted area table.

[0062] [Table 2]

[0063] The ranges shown in Table 2 are information indicating the horizontal range of the restricted area, and are expressed, for example, by a combination of two X and Y coordinates indicating two vertices that form diagonal corners of a rectangle.

[0064] The floor is information indicating the height position of the restricted area, and is expressed, for example, by the number of floors of the facility to be monitored.

[0065] Fig. 2 is a diagram showing an example of a no-entry place. As shown in Fig. 2, a no-entry place F may be set in map information M for each floor. Then, the coordinates defining the no-entry place F may be stored in a no-entry place table.

[0066] (detection table) The detection table is a database that stores detection data. Table 3 is an example of a detection table.

[0067] [Table 3]

[0068] The detection time shown in Table 3 is the time when the detection data was detected.

[0069] The detection name is a name corresponding to the detection data.

[0070] The detection data is information indicating the status of the monitoring work, and includes sensor data such as captured images, sounds, temperature and air temperature, acceleration, angular velocity, position information, and remaining battery power of the robot 10, which are acquired by the sensor unit 11 of the robot 10.

[0071] As an example, the detection data with the detection name "fluorescent light out" includes an image of a fluorescent light going out. As another example, the detection data with the detection name "person entering restricted area" includes an image of a person entering a restricted area.

[0072] The detection data may also include the results of processing based on the sensor data. The processing based on the sensor data may be performed by an AI installed in the robot 10 or may be performed by the control unit 33. Examples of processing based on the sensor data include image recognition, position estimation, and situation determination.

[0073] As an example, whether a person has entered a restricted area may be determined by the control unit 33. Specifically, the control unit 33 may estimate the position information of the person included in the captured image based on the image captured by the robot 10, the angle of view, and the position information of the robot 10. The control unit 33 may then determine whether a person has entered a restricted area by determining whether the estimated position information is included in the restricted area stored in the restricted area table shown in Table 2. Furthermore, the detection data for "person entering a restricted area" may include an estimated value of the number of people who have entered the restricted area. Similarly, with regard to "fluorescent light out," whether a fluorescent light is out may be estimated by the robot 10 or the control unit 33, and the detection data may include an estimated value of the number of burned-out fluorescent lights.

[0074] (Action Plan Table) The action plan table is a database that stores information about action plans. Table 4 is an example of the action plan table.

[0075] [Table 4]

[0076] The action plan ID shown in Table 4 is identification information that uniquely identifies an action plan.

[0077] The activation condition is information indicating the condition for activating the action plan, and includes, for example, detection data that, when detected, activates the action plan.

[0078] The priority is information indicating the priority of the action plan, and the higher the priority, the more the action plan is processed.

[0079] The initial action is the first action that is automatically executed when the action plan is activated.

[0080] The notification message is the content of the message to be displayed on the operator terminal 20.

[0081] The options are options for the next action to be performed, which are presented to the operator by the operator terminal 20.

[0082] (operator table) The operator table is a database that stores various information related to operators. Table 5 is an example of the operator table.

[0083] [Table 5]

[0084] The operator ID shown in Table 5 is identification information for uniquely identifying an operator.

[0085] The action plan ID is identification information for the action plan.

[0086] The number of calls is the number of times that the operator corresponding to the operator ID has been called in the past to process the action plan corresponding to the action plan ID, and corresponds to the number of times that the action plan has been processed in the past.

[0087] (others) The storage unit 32 may store an operation log that indicates the processing history of the action plan. The operation log includes, for example, the action plan that was activated, the activation time, the action that was executed, the execution result, and the like.

[0088] (3) Control unit 33 The control unit 33 controls the overall operation within the system 1. For example, the control unit 33 controls the operation of the robot 10 and the operation of the operator terminal 20.

[0089] The control unit 33 has a function of controlling the connection between the robot 10 and the management device 30. Specifically, the control unit 33 controls the communication unit 31 to send operation instructions to the robot 10 and collect data from the robot 10. Note that various technologies such as MQTT (Message Queue Telemetry Transport) and WebRTC (Web Real-Time Communication) may be used for connection control of the robot 10.

[0090] The control unit 33 has a function of controlling the monitoring work using the robot 10. In detail, the control unit 33 controls the display of the operator terminal 20, which will be described below, and updates the priority.

[0091] (Display control of operator terminal 20) The control unit 33 determines whether to activate an action plan based on the detection data collected from the robot 10. For example, the control unit 33 stores the detection data collected from the robot 10 in a detection table, and determines whether to activate an action plan whose activation condition is satisfied by the detection data stored in the detection table.

[0092] Then, the control unit 33 controls the operator terminal 20 to present the action plan that has been determined to have been activated. In particular, when it has been determined that multiple action plans have been activated, the control unit 33 controls the operator terminal 20 to present the action plan with the highest priority. Presentation of the action plans by the operator terminal 20 can be realized by the operator terminal 20 displaying a UI screen that presents the action plans. With this configuration, the operator can refer to the UI screen displayed on the operator terminal 20 and preferentially process action plans with high priority.

[0093] When it is determined that an action plan with a higher priority than the action plan currently being presented by the operator terminal 20 is to be activated, the control unit 33 controls the operator terminal 20 to switch the action plan to be presented to the action plan with the higher priority. With this configuration, it becomes possible to process the action plan with the higher priority by interruption.

[0094] On the other hand, when it is determined that an action plan with a lower priority than the action plan currently being presented by the operator terminal 20 is to be activated, the control unit 33 controls the operator terminal 20 to maintain the currently presented action plan. This configuration makes it possible to prevent an action plan with a lower priority from interrupting and continue processing an action plan with a higher priority.

[0095] Furthermore, when the processing of an action plan is completed, the control unit 33 controls the operator terminal 20 to present the action plan with the highest priority among the other action plans whose processing has not yet been completed. With this configuration, it is possible to process the action plans sequentially in descending order of priority.

[0096] Hereinafter, an example of a UI screen presenting an action plan, which is displayed on the operator terminal 20 in response to the activation of the action plan, will be described with reference to FIGS.

[0097] Fig. 3 is a diagram showing an example of a UI screen displayed on the operator terminal 20 according to this embodiment. The UI screen D1 shown in Fig. 3 is displayed by the operator terminal 20 when detection data of "fluorescent light out" is acquired and the corresponding action plan is activated.

[0098] In detail, first, as shown in Table 3, the control unit 33 acquires an image of the fluorescent light burning out as detection data for "fluorescent light out." Next, as shown in Table 4, the control unit 33 activates the action plan with action plan ID "A-202," whose activation condition is the acquisition of detection data for "fluorescent light out." Next, as shown in Table 4, the control unit 33 notifies the operator terminal 20 of the notification message "fluorescent light out" in accordance with the initial action "notify remote console." Then, the operator terminal 20 displays a UI screen D1 corresponding to the notification message "fluorescent light out." The UI screen D1 may be displayed as a pop-up.

[0099] As shown in FIG. 3, the UI screen D1 includes UI elements E1 to E6.

[0100] The UI element E1 displays the title of the UI screen D1. In the example shown in Figure 3, the UI element E1 displays the notification message "Fluorescent light out" shown in Table 4.

[0101] UI element E2 displays an image included in the detection data that satisfies the conditions for activating the action plan. UI element E3 displays the detection time of the detection data that satisfies the conditions for activating the action plan. In the example shown in FIG. 3, UI element E2 displays a captured image that shows the fluorescent light F1 at the back turned off, which is an image included in the detection data for "fluorescent light out" shown in Table 3. UI element E3 also indicates the capture time of the captured image.

[0102] The UI element E4 displays, as buttons, one or more action options that the operator can select. In the example shown in Fig. 3, the UI element E4 displays a "Contact facility manager" button, which is an option for the action plan corresponding to "Fluorescent light out" shown in Table 4. When the operator selects this button, the control unit 33 controls the operator terminal 20 to contact the facility manager to request replacement of the fluorescent light.

[0103] UI element E5 is a button that accepts input for completing the processing of the action plan. When the operator selects this button, the control unit 33 completes the processing of the action plan. The control unit 33 then records an operation log from when the action plan is activated until the processing is completed, and ends the pop-up display of the UI screen D1.

[0104] The UI element E6 displays a list of multiple action plans that are currently in progress in order of priority (i.e., in order of priority), and also displays information indicating the action plan that is currently being processed. In the example shown in Fig. 3, only the action plan corresponding to "Fluorescent light out" has been activated, and since that action plan is currently being processed, the UI element E6 displays "Fluorescent light out (processing)" as the first priority.

[0105] As shown in Table 3, the UI screen D1 described above is displayed as a pop-up on the operator terminal 20 immediately after the time "2023 / 12 / 14 12:35:30" when the detection data for "fluorescent light out" is detected.

[0106] Here, while UI screen D1 is displayed, that is, while the action plan for "fluorescent light out" is being processed, at the time "2023 / 12 / 14 12:36:29", detection data for "person intrusion into restricted area" is detected as shown in Table 3. An example of the UI screen displayed in this case is shown in Figure 4.

[0107] Fig. 4 is a diagram showing an example of a UI screen displayed on the operator terminal 20 according to this embodiment. The UI screen D2 shown in Fig. 4 is displayed by the operator terminal 20 when, while the UI screen D1 is being displayed, that is, while the action plan for "fluorescent light out" is being processed, detection data for "person entering restricted area" is detected and the corresponding action plan is activated.

[0108] In more detail, first, as shown in Table 3, the control unit 33 acquires an image capturing a person entering a restricted area as detection data for "person entering restricted area." Next, as shown in Table 4, the control unit 33 activates the action plan with action plan ID "A-207," whose activation condition is the acquisition of detection data for "person entering restricted area." Next, as shown in Table 4, the control unit 33 notifies the operator terminal 20 of the notification message "person entering restricted area" in accordance with the initial action "notify remote console." Then, the operator terminal 20 displays a UI screen D2 corresponding to the notification message "person entering restricted area." The UI screen D2 may be displayed as a pop-up. As a result, the UI screen D1 is hidden.

[0109] Here, as shown in Table 4, the priority of the action plan related to UI screen D1 is "10". The priority of the action plan related to the UI screen D2 is "300." Therefore, the operator terminal 20 displays the UI screen D2 related to the action plan with the higher priority above the UI screen D1, that is, in the forefront.

[0110] As shown in Fig. 4, the UI screen D2 includes UI elements E1 to E6, similar to the UI screen D1 shown in Fig. 3. The contents of the UI elements E1 to E6 are as described above.

[0111] However, in the example shown in FIG. 4, the UI element E1 displays the notification message "Person intruding into restricted area" shown in Table 4.

[0112] Furthermore, UI element E2 displays an image in which a frame F2 surrounding the intruder is superimposed on a captured image showing a person entering a restricted area, which is an image included in the detection data for "person entering restricted area" shown in Table 3. Furthermore, UI element E3 indicates the capture time of the captured image.

[0113] Furthermore, UI element E4 displays "Play Audio," "Call Security Guard," and "False Alarm" buttons, which are action options included in the action plan corresponding to "Person Entering Restricted Area" shown in Table 4. When the operator selects the "Play Audio" button, the control unit 33 controls the robot 10 to play audio warning the user to leave the restricted area. When the operator selects the "Call Security Guard" button, the control unit 33 controls the operator terminal 20 to call a security guard and have a voice conversation with the operator. When the operator selects the "False Alarm" button, the control unit 33 recognizes that the person entering the restricted area was a false alarm.

[0114] Furthermore, UI element E5 is a button that accepts input for completing the processing of the action plan. When the operator selects this button, the control unit 33 completes the processing of the action plan corresponding to "Person trespassing in a restricted area" and terminates the pop-up display of the UI screen D2. That is, the control unit 33 redisplays the UI screen D1 corresponding to "Fluorescent light out," which was hidden behind the UI screen D2.

[0115] 4, an action plan corresponding to "fluorescent light out" and an action plan corresponding to "person entering restricted area" are activated, and the action plan corresponding to "person entering restricted area" is currently being processed. Therefore, UI element E6 displays "person entering restricted area (processing)" as the first priority, and "fluorescent light out" as the second priority.

[0116] Furthermore, as shown in FIG. 4, UI screen D2 includes UI element E7. UI element E7 is a button for accepting a change in priority. When the operator selects this button, the priority of multiple active action plans is changed. For example, in the example shown in FIG. 4, when the operator selects this button, the action plan corresponding to "fluorescent light out" is set to the first priority, and the action plan corresponding to "person entering restricted area" is set to the second priority. As a result, UI screen D1 is popped up and displayed in the foreground, and UI screen D2 is hidden. The operator can proceed with the processing of the action plan corresponding to "fluorescent light out" on UI screen D1. Note that UI element E6 of UI screen D1 displayed in the foreground displays "fluorescent light out (processing)" as the first priority and "person entering restricted area" as the second priority.

[0117] (Priority Update) 4, when it is determined that multiple action plans are to be activated, the control unit 33 displays a list of the activated action plans in order of priority. With this configuration, the operator can recognize the action plans waiting to be processed and check whether the priority order is correct.

[0118] 4, when an instruction to switch the action plan to be presented is input to the operator terminal 20, the control unit 33 controls the operator terminal 20 to switch the action plan to be presented in accordance with the switching instruction. With this configuration, it becomes possible to preferentially process an action plan that the operator has determined should be given higher priority.

[0119] Here, when the control unit 33 switches the action plan to be presented in response to the switching instruction, it may lower the priority of the action plan before the switch and / or raise the priority of the action plan after the switch. For example, the control unit 33 may update the priorities of the action plan corresponding to "fluorescent light out," which is set as the first priority, so that the priority of the action plan corresponding to "person entering restricted area," which is set as the second priority, is higher. This configuration makes it possible to more appropriately prioritize the action plans.

[0120] <3. Operation processing> 5 and 6 are flowcharts showing an example of the flow of processing executed by the system 1 according to this embodiment. The processing according to this flow is executed while the robot 10 is in operation.

[0121] As shown in FIG. 5, first, the management device 30 collects detection data and stores and records the collected detection data in a detection table (step S102).

[0122] Next, the management device 30 determines whether there is an action plan that satisfies the activation condition (step S104). For example, the management device 30 first reads the restricted area table, the detection table, and the action plan table. Then, the management device 30 determines whether there is an action plan in the action plan table whose activation condition is the acquisition of detection data stored in the detection table.

[0123] If it is determined that there is no action plan that satisfies the activation condition (step S104: NO), the processing returns to step S102 again. At this time, the management device 30 may wait for a predetermined time.

[0124] If it is determined that there is an action plan that satisfies the activation condition (step S104: YES), the management device 30 determines whether there is an action plan currently being processed (step S106).

[0125] If it is determined that there is an action plan currently being processed (step S106: YES), the management device 30 determines whether the priority of the newly activated action plan is higher than the priority of the action plan currently being processed (step S108).

[0126] If it is determined that the priority of the newly activated action plan is higher than the priority of the action plan being processed (step S108: YES), the management device 30 controls the operator terminal 20 to switch the display from the UI screen presenting the action plan being processed to a UI screen presenting the newly activated action plan (step S110). For example, if an action plan corresponding to "person entering restricted area" is activated while the operator terminal 20 is displaying a UI screen presenting an action plan corresponding to "fluorescent light out," the operator terminal 20 displays a UI screen presenting the action plan corresponding to "person entering restricted area." Then, the process proceeds to step S116.

[0127] On the other hand, if it is determined that the priority of the newly activated action plan is the same as or lower than the priority of the action plan being processed (step S108: NO), the management device 30 controls the operator terminal 20 to continue displaying the UI screen presenting the action plan being processed (step S112). For example, if an action plan corresponding to "fluorescent light out" is activated while the operator terminal 20 is displaying a UI screen presenting an action plan corresponding to "person entering restricted area," the operator terminal 20 continues displaying the UI screen presenting the action plan corresponding to "person entering restricted area." However, in UI element E6, a display indicating "fluorescent light out" is added below the display indicating "person entering restricted area." Then, the process proceeds to step S116.

[0128] If it is determined in step S106 that no action plan is currently being processed (step S106: NO), the management device 30 controls the operator terminal 20 to display a UI screen presenting a newly activated action plan (step S114). For example, if only the action plan corresponding to "fluorescent light out" has been activated, the operator terminal 20 displays a UI screen presenting the action plan corresponding to "fluorescent light out." Then, the process proceeds to step S116.

[0129] Next, as shown in Fig. 6, the management device 30 determines whether the priority of the action plan has been changed (step S116). For example, the management device 30 determines whether a user operation to change the priority of the action plan has been performed on the UI screen displayed on the operator terminal 20 (i.e., selection of the button of UI element E7 shown in Fig. 4).

[0130] If it is determined that the priority of the action plan has been changed (step S116: YES), the management device 30 controls the operator terminal 20 to display a UI screen presenting the action plan with the highest changed priority (step S118). For example, if the priority of the action plan corresponding to "fluorescent light out" is changed to first while the UI screen presenting the action plan corresponding to "person trespassing in a restricted area" is being displayed, the operator terminal 20 switches the display to a UI screen presenting the action plan corresponding to "fluorescent light out". Then, the process proceeds to step S120.

[0131] On the other hand, if it is determined that the priority of the action plan has not been changed (step S116: NO), the operator terminal 20 maintains the currently displayed UI screen without switching it, and then the process proceeds to step S120.

[0132] Next, the management device 30 determines whether or not an action has been selected (step S120). For example, the management device 30 determines whether or not a user operation has been performed to select the next action to be executed on the UI screen displayed on the operator terminal 20 (i.e., selection of the button of the UI element E4 shown in FIGS. 3 and 4).

[0133] If no action is selected (step S120: NO), the process proceeds to step S124.

[0134] If an action is selected (step S120: YES), the management device 30 controls the robot 10 to execute the action, collects the execution results of the action, and records them as an operation log (step S122). As an example, the management device 30 controls the robot 10 to output a voice, and then records, as an operation log, the time when the robot 10 output the voice, the content of the voice output, and the fact that a person who entered a restricted area left as a result of the voice output. As another example, the management device 30 controls the operator terminal 20 to execute a voice call between a security guard and an operator, and then records, as an operation log, the time when the operator terminal 20 executed the voice call with the facility manager, the content of the voice call, and the fact that a fluorescent light was replaced as a result of the voice call. When the robot 10 executes an action, the management device 30 writes, in the robot table shown in Table 1, the action plan ID of the action plan being processed and the name of the action to be executed. Then, the process proceeds to step S124.

[0135] Next, the management device 30 determines whether the processing of the action plan has been completed (step S124). For example, the management device 30 determines that the processing has been completed when a user operation to input the completion of the processing on the UI screen displayed on the operator terminal 20 (i.e., selection of the button of the UI element E5 shown in FIGS. 3 and 4) has been performed, and determines that the processing has not been completed otherwise.

[0136] If it is determined that the processing of the action plan has not been completed (step S124: NO), the process returns to step S116 again.

[0137] If it is determined that the processing of the action plan has been completed (step S124: YES), the management device 30 determines whether there is an uncompleted action plan (step S126).

[0138] If it is determined that there is an uncompleted action plan (step S126: YES), the management device 30 controls the operator terminal 20 to display a UI screen presenting the action plan with the highest priority among the uncompleted action plans (step S128). After that, the process returns to step S116 again.

[0139] If it is determined that there are no incomplete action plans (step S126: NO), the management device 30 changes the priority of the action plans according to the change in the priority of the action plans (step S130). For example, assume that in step S116, the action plan corresponding to "fluorescent light out" has been given a higher priority than the action plan corresponding to "person entering restricted area." In this case, the management device 30 changes the priority of the action plan corresponding to "person entering restricted area" in the action plan table shown in Table 4 to, for example, "8," which is lower than the priority of the action plan corresponding to "fluorescent light out."

[0140] The process then ends.

[0141] <4. Modifications> The system 1 may enable sharing of the processing of action plans among multiple operators.

[0142] Specifically, the management device 30 may control the operator terminal 20 of an operator with a smaller number of calls to present an action plan preferentially. For example, when an action plan is activated, the control unit 33 selects the operator with the smallest number of calls in the operator table and controls the operator terminal 20 of that operator to present the action plan.

[0143] In this case, the control unit 33 may select the operator with the smallest total number of calls related to all action plans, or the control unit 33 may select the operator with the smallest number of calls related to the activated action plan.

[0144] However, operators who meet certain conditions may be excluded from being called, such as being in the middle of processing an action plan that has a higher priority than the newly activated action plan.

[0145] According to this configuration, it is possible to equalize the workload among the operators and equalize the experience levels of all the operators.

[0146] The flow of processing according to this modified example will be described below with reference to FIG.

[0147] 7 is a flowchart showing an example of the flow of processing executed by the system 1 according to this modification. The processing according to this flow is executed while the robot 10 is in operation.

[0148] Steps S202 to S208 shown in Figure 7 are the same as steps S102 to S108 described above with reference to Figure 5. However, if it is determined in step S206 that an action plan is currently being processed (step S206: YES), the process proceeds to step S208. On the other hand, if it is determined that an action plan is not currently being processed (step S206: NO), the process proceeds to step S232.

[0149] If it is determined in step S208 that the priority of the newly activated action plan is lower than the priority of the action plan being processed (step S208: NO), the operator terminal 20 is controlled to continue displaying the UI screen presenting the action plan being processed (step S212). Then, the process proceeds to step S116 shown in FIG. 6.

[0150] If it is determined in step S208 that the priority of the newly activated action plan is higher than the priority of the action plan being processed (step S208: YES), the management device 30 selects an operator to process the newly activated action plan (step S232). For example, the management device 30 refers to the operator table and selects the operator who has been called the least number of times.

[0151] Next, the management device 30 adds the number of calls to the selected operator and updates the operator table (step S234).

[0152] Next, the management device 30 causes the operator terminal 20 of the selected operator to display a UI screen presenting the newly activated action plan (step S236). Here, if the selected operator terminal 20 is currently processing an action plan, the management device 30 controls the selected operator terminal 20 to switch the displayed UI screen to a UI screen presenting the newly activated action plan. On the other hand, if the selected operator terminal 20 is not currently processing an action plan, the management device 30 controls the selected operator terminal 20 to display a UI screen presenting the newly activated action plan. Thereafter, the process proceeds to step S116 shown in FIG. 6.

[0153] <5. Hardware configuration example> Next, the hardware configuration of an information processing device according to this embodiment will be described with reference to Fig. 8. Fig. 8 is a block diagram showing an example of the hardware configuration of an information processing device according to this embodiment. Note that the information processing device 900 shown in Fig. 8 may realize, for example, a part of the robot 10, the operator terminal 20, or the management device 30 shown in Fig. 1. Information processing by the robot 10, the operator terminal 20, or the management device 30 according to this embodiment is realized by cooperation between software and hardware described below.

[0154] As shown in FIG. 8, the information processing device 900 includes a CPU (Central Processing Unit) 901, a ROM (Read Only Memory) 902, a RAM (Random Access Memory) 903, a host bus 904, a bridge 905, an external bus 906, an interface 907, an input device 908, an output device 909, a storage device 910, and a communication device 911.

[0155] The CPU 901 functions as an arithmetic processing unit and control unit, and controls the overall operation of the information processing device 900 in accordance with various programs. The CPU 901 may also be a microprocessor. The ROM 902 stores programs used by the CPU 901, calculation parameters, etc. The RAM 903 temporarily stores programs used in the execution of the CPU 901, and parameters that change as appropriate during the execution. These are interconnected by a host bus 904 that is composed of a CPU bus, etc. The CPU 901 may form, for example, the control unit 17 or the control unit 33 shown in FIG. 1.

[0156] The host bus 904 is connected to an external bus 906, such as a PCI (Peripheral Component Interconnect / Interface) bus, via a bridge 905. It is not necessary to configure the host bus 904, bridge 905, and external bus 906 separately, and these functions may be implemented on a single bus.

[0157] The input device 908 is composed of input means for the user to input information, such as a mouse, keyboard, touch panel, buttons, microphone, switches, and levers, and an input control circuit that generates an input signal based on the user's input and outputs it to the CPU 901. A user who operates the information processing device 900 can input various data and instruct the information processing device 900 to perform processing operations by operating this input device 908. The input device 908 can form, for example, the operation unit 14 or the operation unit 23 shown in FIG. 1.

[0158] The output device 909 includes, for example, a display device such as a CRT (Cathode Ray Tube) display device, a liquid crystal display (LCD) device, an OLED (Organic Light Emitting Diode) device, a lamp, etc., and an audio output device such as a speaker, etc. The output device 909 can form, for example, the action unit 13 or the display unit 22 shown in FIG.

[0159] The storage device 910 is a device for storing data. The storage device 910 may include a storage medium, a recording device for recording data on the storage medium, a reading device for reading data from the storage medium, and a deleting device for deleting data recorded on the storage medium. The storage device 910 is configured, for example, with an HDD (Hard Disk Drive). This storage device 910 drives a hard disk and stores programs executed by the CPU 901 and various data. The storage device 910 may form, for example, the storage unit 16 or the storage unit 32 shown in FIG. 1.

[0160] The communication device 911 is, for example, a communication interface configured with a communication device for connecting to a network. The communication device 911 may support either wireless communication or wired communication. The communication device 911 may form, for example, the communication unit 15, the communication unit 31, or the communication unit 21 shown in FIG. 1.

[0161] The above describes an example of a hardware configuration capable of realizing the functions of the information processing device 900 according to this embodiment. Each of the above components may be realized using general-purpose components, or may be realized by hardware specialized for the function of each component. Therefore, the hardware configuration used can be changed as appropriate depending on the technical level at the time of implementing this embodiment.

[0162] <6. Supplementary Information> Although the preferred embodiments of the present invention have been described in detail above with reference to the accompanying drawings, the present invention is not limited to these examples. It is clear that a person skilled in the art to which the present invention pertains can conceive of various modifications and alterations within the scope of the technical ideas set forth in the claims, and it is understood that these also naturally fall within the technical scope of the present invention.

[0163] In the above, an example has been described in which the management device 30 forcibly switches the display of the operator terminal 20 to display a UI screen presenting the newly activated action plan when a new action plan with a higher priority is activated. However, the present invention is not limited to such an example. The management device 30 may control the operator terminal 20 to switch the display after obtaining the operator's consent. If the operator does not consent to switching the display, the management device 30 may control the operator terminal 20 to continue displaying a UI screen presenting the action plan being processed. In this case, the management device 30 may update the priority of the action plan, assuming that the priority has been changed.

[0164] Although the above describes an example in which sensor data is acquired by the sensor unit 11 of the robot 10, the present invention is not limited to such an example. Sensor data may be collected by a wearable sensor attached to a facility manager or security guard, or by a sensor installed in the environment, and an action plan may be initiated based on such sensor data.

[0165] Although the robot 10 that moves autonomously on the ground has been cited above as an example of a terminal device that acquires sensor data and executes actions, the present invention is not limited to such an example. As an example, the terminal device may be an autonomously flying drone. As another example, the terminal device may be a device that does not have a mobile function, such as a surveillance camera or digital signage.

[0166] Each device described in this specification may be realized as a single device, or some or all of the devices may be realized as separate devices. For example, at least some of the functions of the management device 30 may be located on the cloud.

[0167] The series of processes performed by each device described herein may be implemented using software, hardware, or a combination of software and hardware. The software programs may be stored in advance, for example, on a recording medium (more specifically, a non-transitory computer-readable storage medium) internal or external to each device. Each program is loaded into a random access memory (RAM) and executed by a processing circuit such as a central processing unit (CPU). The recording medium may be, for example, a magnetic disk, an optical disk, a magneto-optical disk, or a flash memory. The computer program may be distributed, for example, via a network without using a recording medium. The computer may be an application-specific integrated circuit (ASIC), a general-purpose processor that executes functions by loading a software program, or a computer on a server used in cloud computing. The series of processes performed by each device described herein may be centrally processed by a single computer or distributed across multiple computers. Furthermore, in each of the above embodiments, two or more communication means present in one device may be physically implemented on a single medium.

[0168] Furthermore, the processes described herein using flowcharts or sequence diagrams do not necessarily have to be performed in the order shown. Some process steps may be performed in parallel. Furthermore, additional process steps may be employed, and some process steps may be omitted. [Explanation of symbols]

[0169] 1 System 2 Management Center 3 GW 10. Robot 11 Sensor section 12 Moving section 13 Acting part 14 Control section 15 Communications Department 16 Memory section 17 Control Unit 20 Operator terminal 21 Communications Department 22 Display section 23 Control section 30 Management device 31 Communications Department 32 Storage section 33 Control Unit

Claims

1. a control unit that determines whether to activate an action plan based on the sensor data acquired by the terminal device, and controls the operator terminal to present the action plan that has been determined to be activated; Equipped with the control unit controls the operator terminal to present the action plan with the highest priority when it is determined that a plurality of action plans should be activated. Management device.

2. the control unit, when it is determined that the action plan having a higher priority than the action plan being presented by the operator terminal is to be activated, controls the operator terminal to switch the action plan to be presented to the action plan having the higher priority. The management device according to claim 1 .

3. when processing of the action plan is completed, the control unit controls the operator terminal to present the action plan with the highest priority among the other action plans whose processing has not yet been completed. The management device according to claim 1 .

4. when an instruction to switch the action plan to be presented is input to the operator terminal, the control unit controls the operator terminal to switch the action plan to be presented in response to the switching instruction. The management device according to claim 1 .

5. When the control unit switches the action plan to be presented in response to the switching instruction, the control unit lowers the priority of the action plan before the switching and / or raises the priority of the action plan after the switching. The management device according to claim 4 .

6. the control unit controls the operator terminal of an operator having a smaller number of calls to preferentially present the action plan. The management device according to claim 1 .

7. a terminal device that acquires sensor data; an operator terminal operated by an operator; a management device that determines whether to activate an action plan based on the sensor data and controls the operator terminal to present the action plan that has been determined to be activated; Equipped with the management device controls the operator terminal to present the action plan with the highest priority when it is determined that a plurality of action plans should be activated. system.

8. When it is determined that a plurality of the action plans should be activated, the operator terminal displays a list of the plurality of action plans in order of priority. The system of claim 7.

9. The terminal device is a robot capable of autonomous movement. The system of claim 7.

10. 1. A computer-implemented information processing method, comprising: determining whether to activate an action plan based on the sensor data acquired by the terminal device, and controlling the operator terminal to present the action plan that has been determined to be activated; Including, controlling the operator terminal includes controlling the operator terminal to present the action plan with the highest priority when it is determined that a plurality of the action plans should be activated. Information processing methods.

11. Computer, a control unit that determines whether to activate an action plan based on the sensor data acquired by the terminal device, and controls the operator terminal to present the action plan that has been determined to be activated; It functions as the control unit controls the operator terminal to present the action plan with the highest priority when it is determined that a plurality of action plans should be activated. program.

Images