Information processing device, information processing method, and non-transitory storage medium

Abstract

[Problem] To provide a mechanism capable of supporting human decision-making at a work site. [Solution] An information processing device which comprises: an acquisition unit that repeatedly acquires situation data indicating a situation at a work site in which a user participates; a definition unit that defines an action plan selection table in which an action plan that is an option of one or more operations that can be executed when an event occurs at the work site and one or more situation data that are conditions for selecting the action plan are associated with each other; a storage unit that stores the action plan selection table for a plurality of the events that can occur at the work site; and a control unit that controls processing for recommending one or more action plans to be selected by the user on the basis of the situation data acquired by the acquisition unit and the action plan selection table related to the event that occurred at the work site.

Description

Information processing device, information processing method, and non-transitory storage medium

[0001] The present invention relates to an information processing device, an information processing method, and a non-transitory storage medium.

[0002] In recent years, robotics technology has been attracting attention as one means of resolving labor shortages in societies with declining populations, and technologies for smoothly utilizing robots in workplaces have been developed. For example, Patent Document 1 below proposes an algorithm for making decisions about robots introduced into logistics management workplaces that eliminates personal dependency. According to the algorithm in Patent Document 1, when an alert is generated based on the progress rate of each process in a warehouse, pattern matching or simulation based on various information obtained from the robots and each piece of equipment is used to send instructions to workers to optimize staffing, etc., thereby enabling optimal decision-making.

[0003] Furthermore, when an abnormality occurs, it may be difficult for the robot alone to respond appropriately, so a system in which a human responds on behalf of or assists the robot is proposed in Patent Document 2. In Patent Document 2, when a communication robot that works at the reception desk of an event venue, for example, provides directions to visitors, it may encounter situations that are difficult to handle using autonomous control alone. Therefore, by calling an operator who can meet the request of the person it is interacting with, the request can be appropriately responded to. However, due to factors such as difficulty in properly grasping the situation, even humans may find it difficult to appropriately respond to an abnormality that has occurred.

[0004] Japanese Patent No. 6427285 Japanese Patent Application Laid-Open No. 2009-90420

[0005] As disclosed in Patent Document 1, the robot can operate autonomously according to a predetermined algorithm. However, in Patent Document 1, when an abnormality occurs, it may be difficult for the robot to respond appropriately on its own, so a human often responds on behalf of or assists the robot. In Patent Document 2, when a situation becomes difficult to handle using autonomous control alone, an operator can be called, allowing the request of the other party to be appropriately responded to. However, due to factors such as difficulty in appropriately grasping the situation, even humans may find it difficult to appropriately respond to the abnormality that has occurred.

[0006] The present invention has been made in view of the above problems, and an object of the present invention is to provide a mechanism capable of supporting human decision-making at a work site.

[0007] In order to solve the above problem, according to one aspect of the present invention, a system includes an acquisition unit that repeatedly acquires situation data indicating the situation of a work site in which a user is involved, a definition unit that defines an action plan selection table that associates action plans, which are options for one or more actions that can be performed when an event occurs at the work site, with one or more situation data that are conditions for selecting the action plan, a memory unit that stores the action plan selection table for multiple events that can occur at the work site, and a control unit that controls a process of recommending one or more action plans to be selected by the user based on the situation data acquired by the acquisition unit and the action plan selection table for the event that occurred at the work site.

[0008] The control unit may select the recommended action plan based on a degree of coincidence between the situation data acquired by the acquisition unit and the situation data defined in the action plan selection table.

[0009] The control unit may control a process of generating a display screen for accepting input of the situation data and presenting the display screen to the user, and the acquisition unit may acquire the situation data input on the display screen.

[0010] The control unit may control a process of presenting to the user information that the user should refer to when inputting the situation data.

[0011] The control unit may control a process of preferentially accepting input of situation data for items of situation data defined in the action plan selection table, for which the situation data associated between multiple action plans is different.

[0012] The control unit may determine the skill of the user, and if the skill of the user is lower than a threshold, may notify a center administrator who manages the user.

[0013] The control unit may control a process of presenting the display screen being presented to the user to the center manager when the skill of the user is lower than a threshold value.

[0014] The control unit may transfer authority to select the action plan to be executed from the user to the center administrator when the skill of the user is lower than a threshold value.

[0015] The memory unit may store historical data including the event that occurred and the situation data acquired from the time the event occurred until it was completed, and the control unit may determine the skill of the user based on the historical data.

[0016] The control unit may determine the skill of the user based on the number of times the user has completed the event in the past without notifying the center administrator.

[0017] The control unit may determine the skill of the user for each event based on the number of times the user has completed the event in the past.

[0018] The control unit may determine the skill of the user for each action plan based on the number of times the user has selected the action plan in the past.

[0019] The memory unit may store historical data including the event that occurred and the situation data acquired from the time the event occurred until it was completed, and the control unit may update the action plan selection table based on the historical data.

[0020] The acquisition unit may acquire the situation data acquired by a sensor arranged at the work site.

[0021] The situation data may include information about a robot working at the work site.

[0022] The situation data may include information about workers working at the work site.

[0023] The situation data may include information about an item that is the subject of work at the work site.

[0024] In addition, in order to solve the above problem, according to another aspect of the present invention, there is provided an information processing method executed by a computer, which includes repeatedly acquiring situation data indicating the situation of a work site in which a user is involved, a definition unit that defines an action plan selection table that associates action plans, which are options for one or more actions that can be performed when an event occurs at the work site, with one or more situation data that are conditions for selecting the action plan, storing the action plan selection table for multiple events that can occur at the work site, and controlling a process that recommends one or more action plans to be selected by the user based on the acquired situation data and the action plan selection table for the event that occurred at the work site.

[0025] In addition, in order to solve the above problem, according to another aspect of the present invention, a computer is made to function as an acquisition unit that repeatedly acquires situation data indicating the situation of a work site in which a user is involved, a definition unit that defines an action plan selection table that associates action plans, which are options for one or more actions that can be performed when an event occurs at the work site, with one or more situation data that are conditions for selecting the action plan, a memory unit that stores the action plan selection table for multiple events that can occur at the work site, and a control unit that controls a process of recommending one or more action plans to be selected by the user based on the situation data acquired by the acquisition unit and the action plan selection table for the event that occurred at the work site.

[0026] As described above, the present invention provides a mechanism capable of supporting human decision-making at a work site.

[0027] FIG. 1 is a diagram showing a suitable application example of a system 1 according to an embodiment of the present invention. FIG. 2 is a block diagram showing an example of the configuration of an operation device 40 according to the present embodiment. FIG. 3 is a diagram showing an example of a panel P4 displayed on an operation device 40 according to the present embodiment. FIG. 4 is a diagram showing an example of a panel P5 displayed on an operation device 40 according to the present embodiment. FIG. 5 is a diagram showing an example of a panel P6 displayed on an operation device 40 according to the present embodiment. FIG. 6 is a diagram showing an example of a panel P7 displayed on an operation device 40 according to the present embodiment. FIG. 7 is a diagram showing an example of a panel P1 displayed on an operation device 40 according to the present embodiment. FIG. 8 is a diagram showing an example of a panel P2 displayed on an operation device 40 according to the present embodiment. FIG. 9 is a diagram showing an example of a panel P3 displayed on an operation device 40 according to the present embodiment. FIG. 10 is a diagram showing an example of a panel P8 displayed on an operation device 40 according to the present embodiment. FIG. 11 is a block diagram showing an example of the configuration of a control device 30 according to the present embodiment. FIG. 12 is a flowchart showing an example of the flow of processing executed by the system 1 according to the present embodiment. FIG. 13 is a flowchart showing an example of the flow of processing executed by the system 1 according to the present embodiment. FIG. 14 is a block diagram showing an example of the hardware configuration of an information processing device according to the present embodiment. FIG. 15 is a diagram showing another example of a screen displayed on an operation device 40 according to the present embodiment.

[0028] Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In this specification and drawings, components having substantially the same functional configurations are designated by the same reference numerals, and redundant explanations will be omitted.

[0029] <1. Overview> Fig. 1 is a diagram showing a suitable application example of a system 1 according to an embodiment of the present invention. As shown in Fig. 1, the system 1 according to this embodiment includes a sensor 10, a robot 20, a control device 30, and an operation device 40. The control device 30, the sensor 10, the robot 20, and the operation device 40 are connected to each other via a network so as to be able to communicate with each other.

[0030] The system 1 supports human decision-making at a work site where human workers and autonomously operating robots coexist. In the example shown in Figure 1, the work site to which the system 1 is applied is a highway construction site S1.

[0031] Restrictive materials O (O1 to O5) are placed at the highway construction site S1. The restrictive materials O are used on highways and the like to indicate to passing vehicles that they are not to enter restricted areas or to instruct them to change lanes. The restrictive materials O are an example of items that are the subject of work at the highway construction site S1.

[0032] A sensor 10 is placed at a highway construction site S1. The sensor 10 acquires various information about the highway construction site S1. The sensor 10 includes, for example, a camera and acquires images (moving images or still images) of the highway construction site S1. The sensor 10 may also include, for example, an infrared sensor, a depth sensor, a microphone, or the like.

[0033] At the highway construction site S1, workers W and robots 20 are deployed to perform predetermined tasks. Examples of predetermined tasks include monitoring whether or not problems such as tipping over of the restricted material O have occurred, and dealing with any problems that do occur. For example, if the area of ​​responsibility is 1 km per person or vehicle, then for a 10 km restricted area, the number of workers W and the number of robots 20 are deployed so that the total number of workers W and the number of robots 20 is 10.

[0034] The robot 20 has a moving means such as a motor and wheels, and is capable of moving. Furthermore, the robot 20 has a robot arm, and is capable of grasping and moving the regulating material O and changing the posture of the regulating material O. The robot 20 has various sensors such as a camera, and is capable of operating autonomously while recognizing the surrounding situation. The robot 20 has a rechargeable battery, and is powered by the battery.

[0035] The workers W and the robot 20 deployed at the highway construction site S1 are managed by one site supervisor D. For example, the site supervisor D issues instructions regarding the work to the workers W and the robot 20 while checking the status of the workers W and the robot 20. The site supervisor D is an example of a user involved in the work site.

[0036] The operation device 40A is an information processing device used by the site supervisor D. The operation device 40A may be implemented as, for example, a tablet terminal or a smartphone. The operation device 40A will also be referred to as a site terminal 40A below.

[0037] The on-site terminal 40A functions as an interface between the system 1 and the site supervisor D. As one example, the on-site terminal 40A outputs information indicating the status of the highway construction site S1 obtained by the sensor 10 or the like. As another example, the on-site terminal 40A accepts input of instructions for the worker W and the robot 20. As shown in FIG. 1, the screen displayed on the monitor 42A (corresponding to the display unit 42 shown in FIG. 2) of the on-site terminal 40A can be divided into two left and right screen blocks, a left screen block B1 and a right screen block B2. Of course, the screen may be divided into three or more blocks, or the screen may not be divided at all. The left screen block B1 displays a panel P4 (FIG. 3), which will be described later.

[0038] The site supervisor D is managed by a center manager M. For example, the center manager M monitors whether instructions from the site supervisor D are appropriate, and issues instructions to the workers W and the robot 20 on behalf of the site supervisor D. The center manager M can manage multiple site supervisors D who manage multiple highway construction sites S1.

[0039] The operation device 40B is an information processing device used by the center manager M. The operation device 40B is implemented as, for example, a PC (Personal Computer). The operation device 40B will also be referred to as a center terminal 40B below.

[0040] The center terminal 40B functions as an interface between the system 1 and the center manager M. As an example, the center terminal 40B outputs information (hereinafter also referred to as status data) indicating the status of multiple highway construction sites S1 managed by the center manager M. As another example, the center terminal 40B accepts input of instructions for workers W and robots 20 managed by multiple site supervisors D managed by the center manager M. As shown in FIG. 1, the center terminal 40B may share its screen with the site terminal 40A and may display the same screen displayed by the site terminal 40A on the monitor 42B (corresponding to the display unit 42 shown in FIG. 2). This allows the center manager M to monitor whether the site supervisor D's management is appropriate while referring to the same screen displayed by the site terminal 40A. Of course, the center terminal 40B may display a screen different from that displayed by the site terminal 40A.

[0041] The control device 30 is an information processing device that controls the overall operation of the system 1. The control device 30 collects situation data of the highway construction site S1 and controls the operation of the system 1. For example, the control device 30 controls the display of the operation device 40 based on the collected situation data. An example of the situation data will be described below.

[0042] For example, the situation data may include information acquired by a sensor 10 installed at the highway construction site S1. Specifically, the situation data may include images of the highway construction site S1, infrared information, depth information, audio, and the like.

[0043] For example, the situation data may include information about the restricted material O that is the target of work at the highway construction site S1. The information about the restricted material O includes position information about the restricted material O, inertia information about the restricted material O (e.g., acceleration and angular velocity), and, if the restricted material O has a built-in battery for driving a sensor or the like, the remaining battery charge, etc. This information about the restricted material O is acquired by a sensor provided on the restricted material O and transmitted to the control device 30.

[0044] For example, the situation data may include information about the robot 20 working at the highway construction site S1. The information about the robot 20 may include the remaining battery level of the robot 20, images captured by a camera mounted on the robot 20, position information of the robot 20, and the operation history of the robot arm. This information about the robot 20 is acquired by a sensor provided on the robot 20 and transmitted to the control device 30.

[0045] For example, the situation data may include information about a worker W working at a highway construction site S1. The information about the worker W may include biological information such as body temperature and heart rate acquired by a vital sensor worn by the worker W, images captured by a body camera worn by the worker W, and location information about the worker W. This information about the worker W is acquired, for example, by a wearable terminal worn by the worker W and transmitted to the control device 30.

[0046] For example, the situation data may include information about the site supervisor D who manages the robot 20 and the worker W performing work at the highway construction site S1. The information about the site supervisor D may include, for example, information output from the site terminal 40A to the site supervisor D and information input by the site supervisor D to the site terminal 40A. This information about the site supervisor D is acquired by the site terminal 40A and transmitted to the control device 30.

[0047] For example, the situation data may include information about the center manager M who manages the site supervisor D. Examples of the information about the center manager M include information output by the center terminal 40B to the center manager M and information input by the center manager M to the center terminal 40B. This information about the center manager M is acquired by the center terminal 40B and transmitted to the control device 30.

[0048] An example of control performed by the control device 30 based on the situation data collected will be described below.

[0049] As shown in FIG. 1 , assume that a restricting member O3 has fallen over and moved to a position different from its original installation location. Such a problem occurring at a highway construction site S1 will hereinafter also be referred to as an event. The site supervisor D completes the event by executing an action plan, such as instructing the robot 20 or the worker W to recover. An action plan is a selection of actions that can be performed to complete the event that has occurred. Completion of an event means that the problem has been resolved; for example, in the case of the restricting member O3 having fallen over, this means that the restricting member O3 has been returned to its original position and posture. The completion of an action plan is also referred to as the completion of the action plan.

[0050] When there are multiple action plans that can be executed for an event that has occurred, the site supervisor D selects which action plan to execute. However, it may be difficult to appropriately select an action plan to execute. In particular, if the site supervisor D lacks skills, it becomes even more difficult to appropriately select an action plan to execute. Here, skills refer to the expertise to analyze and interpret situation data, knowledge of local rules established at the work site, etc. If an inappropriate action plan is selected, various inconveniences may occur, such as delays in the completion of the event, an increase in the load on the robot 20 and the worker W, and disruption of construction at the highway construction site S1.

[0051] Furthermore, not all of the situation data required to select an appropriate action plan is necessarily automatically acquired by the sensors 10, etc., and may need to be manually input by the site supervisor D. Considering the wide variety of information displayed on the on-site terminal 40A, it may be difficult for the site supervisor D to manually input appropriate situation data, especially if the site supervisor D lacks the necessary skills.

[0052] Therefore, the system 1 according to this embodiment supports the site supervisor D in selecting an appropriate action plan. Specifically, in a situation where an abnormality has occurred, the system 1 presents the site supervisor D with information necessary for decision-making and an action plan to be executed. Furthermore, the system 1 determines the skills of the site supervisor D and provides support according to the skills of the site supervisor D. With this configuration, an appropriate action plan is selected, which makes it possible to complete the event early and prevent inconveniences caused by delays in the completion of the event.

[0053] 2. Configuration Example (1) Operation Device 40 Fig. 2 is a block diagram showing an example of the configuration of the operation device 40 according to this embodiment. As shown in Fig. 2, the operation device 40 includes a communication unit 41, a display unit 42, an input unit 43, a storage unit 44, and a control unit 45.

[0054] (Communication Unit 41) The communication unit 41 has a function of performing communication conforming to any wired or wireless communication standard with other devices included in the system 1. For example, the communication unit 41 transmits situation data to the control device 30 and receives control information for controlling the operation of the operating device 40.

[0055] (Display Unit 42) The display unit 42 has a function of outputting images to the user (i.e., the site supervisor D or the center manager M) operating the operation device 40. In particular, the display unit 42 displays panels P (P1 to P8) (described later) which are screens for displaying information showing the state of the highway construction site S1 and for accepting input of situation data. In addition, the operation device 40 may have an output device for outputting sound, vibration, or the like.

[0056] (Input Unit 43) The input unit 43 has a function of accepting information input from a user operating the operation device 40. The display unit 42 and the input unit 43 may be integrally configured as, for example, a touch panel. In addition, the input unit 43 may accept information input via a keyboard or a mouse, voice input, or gesture input.

[0057] (Storage Unit 44) The storage unit 44 has a function of storing information for the operation of the operating device 40. The storage unit 44 can store, for example, control information received from the control device 30 and situation data input by the user.

[0058] (Control Unit 45) The control unit 45 has a function of controlling the overall operation of the operating device 40. In particular, the control unit 45 controls the operation of the operating device 40 in accordance with control information received from the control device 30.

[0059] As an example, the control unit 45 controls the display by the display unit 42. For example, the control unit 45 controls the display unit 42 to display a panel P generated by the control device 30, or switches the panel P to be displayed in response to a user operation. The control unit 45 also controls the storage unit 44 to store situation data input to the input unit 43 for the panel P displayed on the display unit 42, or controls the communication unit 41 to transmit the situation data to the control device 30.

[0060] As another example, the control unit 45 may control a process of capturing (i.e., recording) the screen displayed on the display unit 42 and transmitting the captured data (hereinafter, referred to as screen capture data) to the control device 30. Note that the screen capture data may be a video or a collection of still images recorded at a specific timing, such as when the user clicks a button.

[0061] (Example of Display Screen) An example of a screen displayed on the field terminal 40A will be described in detail below. Note that by sharing the screen from the field terminal 40A, the center terminal 40B may also display the same screen as the screen exemplified below.

[0062] As described above with reference to Fig. 1, the screen displayed by the on-site terminal 40A can be divided into two left and right screen blocks: a left screen block B1 and a right screen block B2. A panel P is displayed in each of the left screen block B1 and the right screen block B2. The panel P is a complete UI (User Interface) that displays predetermined information and accepts input of predetermined information.

[0063] - Display of left screen block B1 The left screen block B1 may switch between displaying one panel P among a plurality of panels P. For example, the left screen block B1 may display only the top panel P, and switch the displayed panel P in response to a scroll operation or a click operation on a link displayed at the top. Alternatively, the left screen block B1 may simultaneously display a plurality of panels P.

[0064] The left screen block B1 can display panels P (P4 to P7) for checking the status of the highway construction site S1. The site supervisor D can check the status of the highway construction site S1 by referring to the panels P4 to P7 displayed on the left screen block B1. An example of the panels P4 to P7 will be described below with reference to FIGS. 3 to 6.

[0065] 3 is a diagram showing an example of a panel P4 displayed on the on-site terminal 40A according to the present embodiment. The panel P4 displays an image of the highway construction site S1. In the example shown in FIG. 3, the panel P4 displays an image of a restricting material O3 that has fallen over. The panel P4 may display a single image as shown in FIG. 3, or may display multiple images of the highway construction site S1 captured from various angles / ranges.

[0066] 4 is a diagram showing an example of a panel P5 displayed on the on-site terminal 40A according to this embodiment. The panel P5 displays information about the robot 20. In the example shown in FIG. 4, the panel P5 displays an indicator 21 indicating the remaining battery power of the robot 20 on the left side, and an overall image of the robot 20 on the right side.

[0067] 5 is a diagram showing an example of a panel P6 displayed on the on-site terminal 40A according to the present embodiment. The panel P6 displays information about the worker W. In the example shown in FIG. 5, the panel P6 displays a graph G showing the time series transition of the biometric information of the worker W on the left side, and an overall image of the worker W on the right side.

[0068] FIG. 6 is a diagram illustrating an example of a panel P7 displayed on the on-site terminal 40A according to the present embodiment. The panel P7 displays map information in which the positions of the regulating material O, the robot 20, and the worker W are mapped on a map. In the example shown in FIG. 6, a triangle is mapped to the position of the regulating material O, a circle is mapped to the position of the robot 20, and a diamond is mapped to the position of the worker W on a map SM showing an overall image of the highway construction site S1. The on-site terminal 40A may display abnormal regulating material O and normal regulating material O in different display forms, such as highlighting only the fallen regulating material O by changing the color. Note that in FIG. 6, part of the triangle representing the regulating material O is omitted. In this manner, the display of normal regulating material O may be omitted.

[0069] - Display of Right Screen Block B2 The right screen block B2 may switch between displaying one panel P among the multiple panels P. For example, the right screen block B2 may display only the top panel P, and switch the displayed panel P in response to a scroll operation or a click operation on a link displayed at the top. Alternatively, the right screen block B2 may simultaneously display multiple panels P.

[0070] The right screen block B2 may display panels P1 to P3 and P8 that accept various inputs for managing the highway construction site S1. By referring to P1 to P3 and P8 displayed on the right screen block B2, the site supervisor D can input situation data and select an action plan to execute. Examples of panels P1 to P3 and P8 will be described below with reference to FIGS. 7 to 10.

[0071] FIG. 7 is a diagram illustrating an example of a panel P1 displayed on the on-site terminal 40A according to this embodiment. The panel P1 displays a list of situation data items that are desirable to input in order to select an appropriate action plan. In the example shown in FIG. 7, the panel P1 displays a list of situation data items defined in the action plan selection table shown in Table 2 (described later). A check box P11 for inputting YES and a check box P12 for inputting NO are displayed to the left of each situation data item. The site supervisor D can input situation data by filling in these check boxes. Note that, since item 4, “Is action plan A completed?” and item 5, “Is action plan B completed?” defined in the action plan selection table shown in Table 2 (described later) are automatically updated, the site supervisor D does not need to input these situation data. On the other hand, while items 1 to 3 are defined as items that the site supervisor D inputs situation data, the present invention is not limited to this example. For example, the control device 30 may acquire the remaining battery level of the robot 20 and automatically update item 2, “Is the remaining battery level of the robot 20 above 50%?”. The "event" in the "Is the event completed?" section of panel P1 is linked to the event name in panel P8, which will be described later.

[0072] FIG. 8 is a diagram showing an example of a panel P2 displayed on the on-site terminal 40A according to this embodiment. The panel P2 displays action plans and accepts a user operation to select an action plan to be executed. In the example shown in FIG. 8, the panel P2 displays a list of action plans defined in the action plan selection table shown in Table 2, which will be described later. The site supervisor D can execute an action plan by selecting the execute button P21 to the right of the action plan. Furthermore, if a recommended action plan has been identified by the control device 30, a message P22 indicating the recommended action plan is displayed.

[0073] FIG. 9 is a diagram showing an example of panel P3 displayed on the on-site terminal 40A according to this embodiment. Panel P3 displays a flowchart showing the flow of a series of actions that site supervisor D should take. In the example shown in FIG. 9, panel P3 displays a flowchart that lists, in a desired order of execution, the steps to determine and input situation data defined in the action plan selection table shown in Table 2 below, and the steps to select an action plan. By referring to panel P3, site supervisor D can confirm the overall flow up to the completion of the event. The "event" in "Determine whether the event is complete" on panel P3 is linked to the event name on panel P8, which will be described later.

[0074] FIG. 10 is a diagram illustrating an example of a panel P8 displayed on the on-site terminal 40A according to the present embodiment. The panel P8 displays an event list showing a list of events that have occurred so far. In the example shown in FIG. 10, it is displayed that a charging event for the robot 20 has occurred and been completed, and that a tipping event for the restricting material O has occurred and is not yet completed. Furthermore, for completed events, action plans that have been executed may also be displayed. By referring to the panel P8, the site supervisor D can understand events that have occurred in the past and events that are currently occurring. Furthermore, by checking the action plans that have been executed for events that have occurred in the past, the site supervisor D can narrow down the action plans that should be executed to complete the current event.

[0075] (2) Control Device 30 Fig. 11 is a block diagram showing an example of the configuration of the control device 30 according to this embodiment. As shown in Fig. 11, the control device 30 includes a communication unit 31, a storage unit 32, and a control unit 33.

[0076] (Communication Unit 31) The communication unit 31 has a function of communicating in accordance with any wired or wireless communication standard with other devices included in the system 1. For example, the communication unit 31 transmits information for controlling the operation of various devices such as the sensor 10, the robot 20, the field terminal 40A, the center terminal 40B, and sensors built into or attached to the restricted material O and the worker W, and receives information transmitted from the various devices.

[0077] The communication unit 31 is an example of an acquisition unit that repeatedly acquires situation data indicating the situation of the highway construction site S1. For example, the communication unit 31 receives situation data from the sensor 10, the robot 20, the on-site terminal 40A, the center terminal 40B, and sensors built into or attached to the restricted material O and the worker W.

[0078] (Storage Unit 32) The storage unit 32 has a function of storing various types of information used to control the overall operation of the system 1. An example of information stored in the storage unit 32 will be described below.

[0079] Event Determination Table The storage unit 32 stores an event determination table that defines, in association with each other, events that may occur at the highway construction site S1, conditions for determining the occurrence of the event, and action plans that may be executed if the event occurs. By referring to the event determination table, it becomes possible to appropriately determine events that have occurred at the highway construction site S1. An example of the event determination table is shown in Table 1 below.

[0080]

[0081] In the example shown in Table 1, the falling of the restricting material O, the charging of the robot 20, and the poor health of the worker W are defined as events that may occur at the highway construction site S1. For example, it is defined that the falling of the restricting material O is determined based on an email notification that the restricting material O has fallen. In addition, action plans A, B, and C are associated as action plans that may be executed when the restricting material O falls.

[0082] - Action Plan Selection Table The memory unit 32 stores an action plan selection table that defines one or more action plans that can be executed when an event occurs at the highway construction site S1, in association with one or more situation data that serve as conditions for selecting an action plan. The action plan selection table is a setting of situation data for prioritizing one or more action plans that can be executed to complete the event that has occurred. Here, the memory unit 32 stores the above-mentioned action plan selection table for multiple events that can occur at the highway construction site S1. By referring to the action plan selection table, it is possible to select an appropriate action plan that corresponds to the acquired situation data. An example of an action plan selection table corresponding to "restriction material falls" is shown in Table 2 below.

[0083]

[0084] In the example shown in Table 2, action plan A is defined as instructing the robot 20 to perform recovery, action plan B is defined as instructing the worker W to perform recovery, and action plan C is defined as contacting the center manager M. The entity that executes these action plans is typically the field terminal 40A. That is, the field terminal 40A can execute the action plan selected by the field supervisor D. Alternatively, the field supervisor D can execute the action plan. In that case, the field supervisor D can, for example, verbally issue a recovery instruction to the worker W or manually input a recovery instruction to the robot 20. Then, for each action plan, situation data items 1 to 5 are defined as one or more situation data that serve as conditions for selecting an action plan. In Table 2, for example, it is defined that if the falling position of the restriction material O is within the restriction area (item 1: YES), the remaining battery level of the robot 20 is 50% or more (item 2: YES), the event is not completed (item 3: NO), action plan A is not completed (item 4: NO), and action plan B is not completed (item 5: NO), it is desirable that action plan A be selected.

[0085] In Table 2, the action plan is defined as something to be executed, but the present invention is not limited to this example. The action plan may also be defined as not executing anything.

[0086] The related panels in Table 2 are panels displayed on the on-site terminal 40A as a reference for entering situation data. The on-site terminal 40A displays the related panels in the left screen block B1 when accepting the input of situation data items 1 to 3 in the right screen block B2. For example, the on-site terminal 40A displays panel P4 (an image of the highway construction site S1) in the left screen block B1, while accepting the input of situation data item 1 (is the location of the fallen restriction material within the restricted area?) in the right screen block B2. On the other hand, for situation data items 4 and 5, which are acquired without the intervention of the site supervisor D, the situation data is automatically updated, so no related panels are defined.

[0087] The average confirmation time in Table 2 is a reference value for the time it takes from the time the related panel is displayed until the situation data is input. The average confirmation time is set as the average value of the time it took from the time the related panel was displayed until the situation data was input for past events. According to the example shown in Table 2, the on-site terminal 40A accepts an input as to whether the fallen position of the restriction material O is within the restricted area on average 10 seconds after the start of displaying panel P4.

[0088] The action plan selection table may be generated in advance and stored in the storage unit 32. Alternatively, the action plan selection table may be updated as described below.

[0089] - Event Container The storage unit 32 temporarily stores an event container that is generated when an event occurs. An event container is a data structure that stores information about the event that occurred. For example, the event container stores information indicating the event that occurred, information indicating the action plan associated with the event that occurred, and an action plan selection table corresponding to the event that occurred. Furthermore, the storage unit 32 temporarily stores situation data acquired from the time the event occurred until the event was completed in the event container.

[0090] Event History Data The storage unit 32 stores event history data, which is data related to events that have occurred and been completed. When an event is completed, the storage unit 32 stores the temporarily stored event container as event history data (i.e., long-term storage). For example, the storage unit 32 may store the event history data related to events that have occurred and been completed in the form of a list.

[0091] Other The storage unit 32 stores recommended thresholds, which will be described later. The recommended thresholds may be stored in advance or may be updated as appropriate.

[0092] The storage unit 32 stores a skill determination threshold, which will be described later. The skill determination threshold may be stored in advance or may be updated as appropriate.

[0093] The storage unit 32 stores information indicating the skills of the site supervisor D. The skills of the site supervisor D are updated as appropriate.

[0094] (Control Unit 33) The control unit 33 has a function of controlling the overall operation of the control device 30. As shown in Fig. 11 , the control unit 33 functions as an event management unit 331, an action plan selection support unit 332, a skill management unit 333, and a definition unit 334. These components control various processes based on the situation data acquired by the communication unit 31 and the information stored in the storage unit 32. These components will be described in detail below.

[0095] (Event Management Unit 331) The event management unit 331 has a function of managing various information related to events that occur at the highway construction site S1.

[0096] The event management unit 331 determines an event that has occurred at the highway construction site S1 based on the acquired situation data and the event determination table. For example, the event management unit 331 refers to the event determination table shown in Table 1, and determines that the regulating material O has fallen over based on receiving an email informing the user that the regulating material O has fallen over.

[0097] When the event management unit 331 determines that an event has occurred, it generates an event container related to the event. The event management unit 331 then temporarily stores the event container in the storage unit 32. The event management unit 331 stores status data acquired from the time the event occurred until its completion in the event container. When the event is completed, the event management unit 331 then stores the temporarily stored event container in the storage unit 32 as event history data (i.e., long-term storage).

[0098] (Action Plan Selection Support Unit 332) The action plan selection support unit 332 has a function of supporting the site supervisor D in selecting an action plan.

[0099] The action plan selection support unit 332 controls the display by the operation device 40. More specifically, the action plan selection support unit 332 generates a screen to be displayed on the operation device 40, and transmits it to the operation device 40 for display. The action plan selection support unit 332 then updates the screen in accordance with a user input to the operation device 40, and transmits the updated screen to the operation device 40 for display. The action plan selection support unit 332 may, for example, describe the screen to be displayed on the operation device 40 using HTML (Hyper Text Markup Language) and transmit the HTML file to the operation device 40.

[0100] In particular, the action plan selection support unit 332 controls the process of generating a screen displaying information showing the status of the highway construction site S1 and presenting it to the site supervisor D. In detail, the action plan selection support unit 332 generates the above-mentioned panels P4 to P7 and displays them in the left screen block B1 of the on-site terminal 40A. This allows the site supervisor D to easily grasp the status of the highway construction site S1.

[0101] The action plan selection support unit 332 also controls the process of generating a screen for accepting input of situation data and presenting it to the site supervisor D. The action plan selection support unit 332 then acquires the situation data input on the screen. Specifically, the action plan selection support unit 332 generates the above-mentioned panels P1 to P3 and P8 and displays them on the right screen block B2 of the on-site terminal 40A. This allows the site supervisor D to input situation data and receive support from the action plan selection support unit 332 regarding the selection of an action plan.

[0102] In this case, the action plan selection support unit 332 may control a process of presenting information to the site supervisor D that the site supervisor D should refer to when entering the situation data. For example, the action plan selection support unit 332 controls the on-site terminal 40A to display a related panel for an unentered situation data item among one or more situation data items that are conditions for selecting an action plan defined in the action plan selection table. When there are multiple unentered situation data items, the action plan selection support unit 332 may extract the situation data items in order from the smallest item number. For example, when only the situation data item No. 1 in the action plan selection table shown in Table 2 has been entered, the action plan selection support unit 332 controls the on-site terminal 40A to display panel P5, which is a related panel for the situation data item No. 2, in the left screen block B1. By checking the indicator 21 indicating the remaining battery level of the robot 20 displayed on panel P5, the site supervisor D can easily enter whether the remaining battery level of the robot, which is the situation data item No. 2, is 50% or more. In this way, by dynamically switching the display of the left screen block B1 to the panel related to the item of situation data that has not been input, it becomes possible for the site supervisor D to input situation data more easily.

[0103] Displaying the related panel on the on-site terminal 40A refers to displaying the related panel within the field of view of the site supervisor D, such as displaying the related panel at the top of the left screen block B1 or displaying the related panel as a pop-up. Furthermore, the action plan selection support unit 332 may make the frame of the related panel red or blink. This configuration makes it possible to encourage attention to the related panel.

[0104] Once the situation data is acquired, the action plan selection support unit 332 controls a process for recommending an appropriate action plan based on the acquired situation data. Specifically, the action plan selection support unit 332 controls a process for recommending one or more action plans to be selected based on the acquired situation data and an action plan selection table related to the event that occurred. As an example, the action plan selection support unit 332 may control the on-site terminal 40A to display a message (e.g., message P22 on panel P2) recommending an appropriate action plan from among action plans A to C. As another example, the action plan selection support unit 332 may control the on-site terminal 40A to display the priorities of action plans A to C defined in the action plan selection table, or to sort and display action plans A to C in order of priority. This configuration makes it possible to support the decision-making of the site supervisor D.

[0105] Here, the action plan selection support unit 332 selects a recommended action plan based on the degree of match between the acquired situation data and the situation data as selection conditions defined in the action plan selection table. Specifically, the action plan selection support unit 332 may control the on-site terminal 40A to recommend an action plan among action plans A to C whose situation data match rate exceeds a predetermined threshold (hereinafter also referred to as a recommendation threshold). Furthermore, the action plan selection support unit 332 may control the on-site terminal 40A to recommend an action plan among action plans A to C whose situation data match rate is the highest. For example, assume that situation data is acquired in the action plan selection table shown in Table 2, where item number 1 is YES, item number 2 is YES, item number 3 is NO, item number 4 is NO, and item number 5 is NO. In this case, the situation data match rate for action plan A is 100%, the situation data match rate for action plan B is 60%, and the situation data match rate for action plan C is 20%. If the recommendation threshold is 50%, the action plan selection support unit 332 may recommend action plan A as the optimal action plan, action plan B as the next optimal action plan, and so on, in order of priority. Furthermore, the action plan selection support unit 332 may control the on-site terminal 40A to recommend only action plan A, which has the highest matching rate with the situation data, to the site supervisor D. This configuration makes it possible to recommend an appropriate action plan to the site supervisor D in accordance with the action plan selection table.

[0106] Here, as the situation data is input, the matching rate of the situation data may change. The action plan selection support unit 332 may change the recommended action plan in accordance with changes in the matching rate of the situation data. For example, immediately after an event occurs, the situation data for items 1 and 2 in the action plan selection table shown in Table 2 have not been input, and the situation data for items 3 to 5 are NO. In this case, the matching rate of the situation data for action plan A is 60%, the matching rate of the situation data for action plan B is 60%, and the matching rate of the situation data for action plan C is 20%. Therefore, if the recommended threshold is 50%, the action plan selection support unit 332 may recommend action plans B and A in this order. Then, assume that YES is input as the situation data for item 1. In this case, the matching rate of the situation data for action plan A is 80%, the matching rate of the situation data for action plan B is 60%, and the matching rate of the situation data for action plan C is 20%. Therefore, the action plan selection support unit 332 may recommend action plans A and B in this order.

[0107] The action plan selection support unit 332 may control a process to preferentially accept input of situation data for items of situation data defined in the action plan selection table, where the situation data associated with each action plan differs. For example, in the action plan selection table shown in Table 2, the selection conditions for the situation data items Nos. 1 and 2 between action plan A and action plan B differ (YES / NO), while the selection conditions for the situation data items Nos. 3 to 5 are the same (NO). Therefore, when the situation data items Nos. 1 and 2 are input, the recommended action plans can be narrowed down more efficiently than when the situation data items Nos. 3 to 5 are input. Therefore, in order to preferentially accept the input of the situation data items Nos. 1 and 2, the action plan selection support unit 332 may display panels P4 and P5, which are related to the situation data items Nos. 1 and 2, on the on-site terminal 40A and prompt the site supervisor D to input the situation data items Nos. 1 and 2. This configuration enables more efficient narrowing down of the recommended action plans.

[0108] The action plan selection support unit 332 may update the action plan selection table based on the event history data. As an example, the action plan selection support unit 332 analyzes the event history data to identify situation data that is frequently acquired when an event is completed by executing a specific action plan. The action plan selection support unit 332 may then add acquisition of the identified situation data to the selection conditions for the specific action plan in the action plan selection table. For example, assume that an event has been completed 10 times by executing action plan A, and 8 out of 10 times a YES answer was entered for item 1, "Is the location of the fallen restriction material within the restricted area?" Since the 8 YES answers are more frequent than the 2 NO answers, the action plan selection support unit 332 sets the more frequent YES answer to item 1, "Is the location of the fallen restriction material within the restricted area?", as the selection condition for action plan A. This configuration makes it possible to select a more appropriate action plan for future events.

[0109] (Skill management unit 333) The skill management unit 333 determines the skills of site supervisor D. Then, the skill management unit 333 controls processing according to the skills of site supervisor D. Note that the skills are quantified and the level of the skills is determined.

[0110] As an example, if the skill of site supervisor D is lower than a threshold (hereinafter also referred to as the skill judgment threshold), the skill management unit 333 may notify the center manager M who manages the site supervisor D. For example, the skill management unit 333 transmits information indicating that an event has occurred at a highway construction site S1 managed by the site supervisor D with low skills to the center terminal 40B. The center manager M can use this notification as a trigger to support the site supervisor. In this way, with this configuration, the site supervisor D with low skills can receive support from the center manager M.

[0111] As another example, if the skill of the site supervisor D is lower than the skill determination threshold, the skill management unit 333 may present the display screen presented to the site supervisor D to the center manager M. For example, the skill management unit 333 may transmit screen capture data of the site terminal 40A to the center terminal 40B, thereby causing the site terminal 40A and the center terminal 40B to share the screen. With this configuration, the center manager M can support the site supervisor D while viewing the same screen that the site supervisor D is viewing.

[0112] As another example, if the skill of the site supervisor D is lower than the skill determination threshold, the skill management unit 333 may transfer the authority to select an action plan to be executed from the site supervisor D to the center manager M. In this case, the action plan selected by the center manager M will be executed instead of the site supervisor D. This configuration makes it possible to prevent mistakes in selecting an action plan due to insufficient skills.

[0113] The skill management unit 333 may determine the skill of the site supervisor D based on the event history data. That is, the skill management unit 333 may quantify the skill of the site supervisor D based on the event history data and compare the quantified skill with a skill determination threshold to determine the level of the skill of the site supervisor D. With this configuration, it is possible to appropriately determine the skill of the site supervisor D based on the past performance of the site supervisor D.

[0114] As an example, the skill management unit 333 may determine the skill of the site supervisor D based on the number of times the site supervisor D has completed an event in the past without notifying the center manager M. Note that the site supervisor D completing an event refers to the completion of the event as a result of the execution of an action plan selected by the site supervisor D. For example, the skill management unit 333 may determine that the more times the site supervisor D has completed an event in the past without notifying the center manager M, the higher the skill, and vice versa. With this configuration, it is possible to evaluate the site supervisor D's skill based on his or her track record of completing events on his or her own.

[0115] As another example, the skill management unit 333 may determine the skill of the site supervisor D for each event based on the number of times the site supervisor D has completed the event in the past. For example, the skill management unit 333 may determine that the more times the site supervisor D has completed an event in the past, the higher the skill for that event, and vice versa. With this configuration, it is possible to evaluate the skill for each event based on the difference in experience points for each event.

[0116] As another example, the skill management unit 333 may determine the skill of the site supervisor D for each action plan based on the number of times the site supervisor D has selected the action plan in the past. For example, the skill management unit 333 may determine that the more times the site supervisor D has selected an action plan in the past, the higher the skill of the site supervisor D with respect to that action plan, and vice versa. With this configuration, it is possible to evaluate the skill of each action plan based on the experience of selecting the action plan.

[0117] Alternatively, the skill management unit 333 may determine the skill of the site supervisor D based on the real-time behavior of the site supervisor D while an event is occurring. That is, the skill management unit 333 may quantify the skill of the site supervisor D based on the real-time behavior of the site supervisor D while an event is occurring, and compare the quantified skill with a skill determination threshold to determine the level of the skill of the site supervisor D. With this configuration, it becomes possible to appropriately determine the skill even in a situation where an unprecedented accident occurs and the past performance of the site supervisor D is no longer valid.

[0118] As an example, the skill management unit 333 may determine the skill of the site supervisor D based on whether the site supervisor D selected a recommended action plan. For example, the skill management unit 333 may determine that the site supervisor D has high skills when the site supervisor D selected an action plan recommended by the action plan selection support unit 332, and may determine that the site supervisor D has low skills when the site supervisor D selected an action plan that is not recommended. With this configuration, it is possible to determine the skill of the site supervisor D in real time.

[0119] As another example, the skill management unit 333 may determine the skill of the site supervisor D based on the time it takes for the site supervisor D to input the situation data. For example, the skill management unit 333 may determine that the skill of the site supervisor D is high if the time it takes for the site supervisor D to input the situation data after the panel related to the situation data to be input is displayed is less than the average confirmation time, and may determine that the skill of the site supervisor D is low if this is not the case. With this configuration, it is possible to determine the skill of the site supervisor D in real time.

[0120] As another example, the skill management unit 333 may determine the skill of the site supervisor D based on the content of operations by the site supervisor D input to the site terminal 40A. For example, the skill management unit 333 may determine that the skill of the site supervisor D is low if unnecessary operations are repeated, such as excessively frequent mouse clicks or mouse movements, or excessively frequent scrolling of the panel P. With this configuration, it is possible to determine the skill of the site supervisor D in real time.

[0121] The skill judgment threshold is stored in the storage unit 32. The skill judgment threshold may be calculated / updated based on the event history data of multiple site supervisors D. For example, the skill management unit 333 may use the average number of times that multiple site supervisors D have completed an event without notifying the center manager M in the past as the skill judgment threshold to be compared with the average number of times.

[0122] (Definition Unit 334) The definition unit 334 has a function of defining (i.e., creating a new) an event determination table (Table 1) and an action plan selection table (Table 2), storing them in the storage unit 32, and updating them as appropriate. The definition unit 334 defines and updates these tables based on input from the center manager M or the site supervisor D via the operation device 40, for example.

[0123] 12 and 13 are flowcharts showing an example of the flow of processing executed by the system 1 according to this embodiment.

[0124] (Step S101) First, the control device 30 collects situation data related to the event that has occurred. For example, the regulating material O detects that a vehicle has collided with the regulating material O and overturned using an acceleration sensor or the like mounted on the regulating material O, and transmits the detection result to the control device 30 by email or the like. Although not specifically mentioned below, it is assumed that the control device 30 continues to collect situation data as needed even while executing each step described below.

[0125] (Step S102) Next, the control device 30 determines whether an event has occurred. For example, the control device 30 references the event determination table shown in Table 1 and determines that an event (the tipping of the restriction material O) has occurred based on the email received in step S101. Having determined that an event has occurred, the control device 30 generates an event container. For example, the control device 30 generates an event container that stores the event name "Restriction material tipping," information indicating action plans A, B, and C, which are action plans related to "Restriction material tipping," and an action plan selection table corresponding to "Restriction material tipping" shown in Table 2. Situation data acquired from the event occurrence to its completion (i.e., steps S101 to S116) is added to the event container as needed. Furthermore, having determined that an event has occurred, the control device 30 may generate a display screen displaying information indicating the occurrence of the event and display it on the on-site terminal 40A. This allows the site supervisor D to be notified that the event has occurred.

[0126] (Step S103) Next, the control device 30 transmits a screen capture start instruction to the on-site terminal 40 A. Upon receiving the screen capture start instruction, the on-site terminal 40 A starts capturing the displayed left screen block B1 and right screen block B2, and uploads the screen capture data to the control device 30 as appropriate.

[0127] (Step S104) Next, the control device 30 calculates, for each action plan, the match rate between the acquired situation data and the situation data as selection conditions defined in the action plan selection table. For example, if the situation data for items 1 and 2 in the action plan selection table shown in Table 2 are not entered and the situation data for items 3 to 5 are NO, the control device 30 calculates the match rates for action plans A to C as 60%, 60%, and 20%, respectively. The action plan selection support unit 332 then stores the calculated match rates in the event container.

[0128] If the situation data has been updated (step S111: YES) or if the event has not been completed (step S115: NO), step S104 is executed again. At that time, if situation data indicating that the falling position of the restriction material O is within the restricted area (item number 1: YES) is acquired, the control device 30 calculates the match rates of action plans A to C as 80%, 60%, and 20%, respectively. Furthermore, if situation data indicating that the remaining battery charge of the robot 20 is 50% or more (item number 2: YES) is acquired, the control device 30 calculates the match rates of action plans A to C as 100%, 60%, and 20%, respectively.

[0129] (Step S105) Next, the control device 30 determines whether the skill of the site supervisor D is equal to or greater than the skill determination threshold. As an example, the control device 30 may determine that the skill of the site supervisor D is equal to or less than the skill determination threshold if the number of times that the site supervisor D has previously completed an event without notifying the center manager M is lower than the average value of such numbers for multiple site supervisors D. As another example, the control device 30 may calculate the time taken from step S107 to step S111 from screen capture data of the on-site terminal 40A. Then, the control device 30 may determine that the skill of the site supervisor D is equal to or less than the skill determination threshold if the time taken from step S107 to step S111 is longer than the average confirmation time defined in the action plan selection table.

[0130] If it is determined that the skill of the site supervisor D is equal to or less than the skill judgment threshold (step S105: YES), the process proceeds to step S210a. On the other hand, if it is determined that the skill of the site supervisor D exceeds the skill judgment threshold (step S105: NO), the process proceeds to step S106.

[0131] (Step S106) The control device 30 extracts related panels related to unentered situation data items from among one or more situation data items that are conditions for selecting an action plan, as defined in the action plan selection table. If there are multiple unentered situation data items, the control device 30 extracts them in order, starting with the situation data items with the smallest item numbers. Regarding the example shown in Table 2, for example, if only situation data item number 1 has been entered, the control device 30 extracts panel P5, which is the related panel related to situation data item number 2.

[0132] (Step S107) Next, the control device 30 causes the on-site terminal 40A to display the related panel extracted in step S106. For example, the control device 30 transmits identification information of the extracted related panel, panel P5, to the on-site terminal 40A. Then, based on the identification information, the on-site terminal 40A displays panel P5 at the top of the left screen block B1.

[0133] (Step S108) Next, the control device 30 transmits the coincidence rate of the situation data for each action plan calculated in step S104 to the on-site terminal 40A. The on-site terminal 40A sorts the action plans in descending order of coincidence rate of the situation data on the panel P2 displayed in the right screen block B2 and presents them to the site supervisor D.

[0134] (Step S109) Next, the control device 30 determines whether the match rate of the situation data for each action plan calculated in step S104 is equal to or greater than the recommended threshold. If there is an action plan for which a match rate equal to or greater than the recommended threshold is calculated (step S109: YES), the process proceeds to step S110. On the other hand, if there is no action plan for which a match rate equal to or greater than the recommended threshold is calculated (step S109: NO), the process proceeds to step S111.

[0135] (Step S110) The control device 30 controls the on-site terminal 40A to display the recommended action plan. Specifically, the control device 30 controls the on-site terminal 40A to recommend an action plan, of the action plans A to C, whose matching rate with the situation data exceeds the recommended threshold. Furthermore, the control device 30 may control the on-site terminal 40A to recommend an action plan, of the action plans A to C, whose matching rate with the situation data is the highest. For example, the operation device 40 additionally displays a message P22 on the panel P2 recommending the action plan whose matching rate is equal to or exceeds the recommended threshold and whose matching rate is the highest.

[0136] (Step S111) As shown in FIG. 13 , the control device 30 then determines whether the situation data has been updated. Specifically, the control device 30 determines whether new situation data has been input to the panel P1 or whether new situation data has been received from the sensor 10 or the like. Alternatively, the site supervisor D may input to the on-site terminal 40A that the situation data will not be updated, and the control device 30 may determine whether the situation data has been updated based on the presence or absence of such input. If it is determined that the situation data has been updated (step S111: YES), the process returns to step S104, and the match rate of the situation data is calculated again. If it is determined that the situation data has not been updated (step S111: NO), the process proceeds to step S112.

[0137] (Step S112) Next, the on-site terminal 40A accepts a user operation to select an action plan to be executed. For example, the site supervisor D inputs an operation to the on-site terminal 40A to select the action plan recommended in step S110 as the action plan to be executed. As a result, the action plan is executed. As an example, the on-site terminal 40A transmits information instructing the robot 20 or worker W nearest to the fallen restriction material O to recover it.

[0138] (Step S113) Next, the control device 30 stores the status data at the time of the action plan execution. For example, the control device 30 stores the status data of the items defined in the action plan selection table in the event container. Specifically, the control device 30 stores the status data of items 1 to 3 entered by the site supervisor D and the status data of items 4 and 5 automatically acquired in the event container. The control device 30 also stores information indicating the panel P displayed by the site terminal 40A in the left screen block B1, information indicating the executed action plan, and screen capture data of the site terminal 40A in the event container. Furthermore, the control device 30 stores the time taken from step S107 to step S111 in the event container.

[0139] (Step S114) Next, the control device 30 determines whether the recommended action plan has been executed. If it is determined that the recommended action plan has been executed (step S114: YES), the process proceeds to step S115. If it is determined that a non-recommended action plan has been executed (step S114: NO), the process proceeds to step S210b.

[0140] (Step S115) Next, the control device 30 determines whether the event has been completed. For example, the control device 30 determines whether the check box P11 or P12 of the "Is the event completed?" checkbox is checked on the panel P1 displayed on the right screen block B2 of the operation device 40. If it is determined that the event has been completed (step S115: YES), the process proceeds to step S116. On the other hand, if it is determined that the event has not been completed (step S115: NO), the process proceeds to step S104.

[0141] (Step S116) The control device 30 transmits a screen capture end instruction to the on-site terminal 40 A. Upon receiving the screen capture end instruction, the on-site terminal 40 A ends the screen capture and the upload of the screen capture data.

[0142] (Step S117) Next, the control device 30 stores the temporarily stored event container as event history data for a long term. Note that, for any item of the situation data defined in the action plan selection table that has not yet been entered, the control device 30 may acquire the situation data after the event is completed and add it to the event history data.

[0143] (Step S118) Next, the control device 30 updates the action plan selection table based on the event history data. For example, assume that the number of times an event has been completed by executing action plan A is 10, and 8 out of 10 times a YES answer was entered for item 1, "Is the position of the fallen restriction material within the restricted area?" In this case, the control device 30 sets the more frequent YES answer to item 1, "Is the position of the fallen restriction material within the restricted area?", as the selection condition for action plan A. Then, the processing ends.

[0144] (Step S210a) The control device 30 notifies the center manager M. For example, the control device 30 transmits information indicating that the skills of site supervisor D are low and screen capture data of the field terminal 40A to the center terminal 40B. Then, upon receiving this notification, the center terminal 40B displays a pop-up message such as "Site supervisor D lacks skills" or sounds an alarm. When the center manager M confirms this message and hides the pop-up, the center terminal 40B displays the screen capture data of the field terminal 40A and urges the center manager M to monitor site supervisor D.

[0145] Thereafter, the process proceeds to step S 106. Note that step S212, which will be described later, may be executed between step S210a and step S 106, either automatically or based on the determination in step S211, which will be described later.

[0146] (Step S210b) The control device 30 notifies the center manager M. For example, the control device 30 transmits information indicating that a non-recommended action plan has been executed and screen capture data of the field terminal 40A to the center terminal 40B. Then, upon receiving this notification, the center terminal 40B displays a pop-up message such as "An action plan other than the recommended action plan has been selected" or sounds an alarm. When the center manager M confirms this message and hides the pop-up, the center terminal 40B displays the screen capture data of the field terminal 40A and urges the center manager M to monitor the site supervisor D.

[0147] (Step S211) Next, the control device 30 determines whether the judgment of the site supervisor D is appropriate. For example, the center terminal 40B displays a pop-up message such as "Please confirm whether there is any problem with the judgment of the site supervisor D" and accepts input of the confirmation result by the center manager M. The center manager M can confirm whether the judgment of the site supervisor D is appropriate by looking at the shared screen of the site terminal 40A. Then, the control device 30 determines whether the judgment of the site supervisor D is appropriate based on the input to the center terminal 40B.

[0148] If it is determined that the judgment of the site supervisor D is appropriate (step S211: YES), the process proceeds to step S115. On the other hand, if it is determined that the judgment of the site supervisor D is inappropriate (step S211: NO), the process proceeds to step S212.

[0149] (Step S212) The control device 30 transfers the authority to select an action plan to be executed from the site supervisor D to the center manager M. For example, the authority to operate the field terminal 40A is transferred to the center manager M, and the center manager M selects an action plan on the field terminal 40A through operations via the center terminal 40B. Meanwhile, the screen of the field terminal 40A is locked and cannot be operated by the site supervisor D, and the screen transitions based on operations by the center manager M via the center terminal 40B. Then, the process proceeds to step S104.

[0150] 4. Hardware Configuration Example Next, the hardware configuration of an information processing device according to this embodiment will be described with reference to Fig. 14. Fig. 14 is a block diagram showing an example of the hardware configuration of an information processing device according to this embodiment. Note that the information processing device 1000 shown in Fig. 14 may realize, for example, the operation device 40 shown in Fig. 2 or the control device 30 shown in Fig. 11. Information processing by the control device 30 or the operation device 40 according to this embodiment is realized by cooperation between software and hardware described below.

[0151] As shown in FIG. 14, the information processing device 1000 may include a processor 1001, a ROM (Read Only Memory) 1002, a RAM (Random Access Memory) 1003, an internal bus 1004, an input / output interface 1010, a display device 1011, an input device 1012, an audio output unit 1013, a storage device 1014, a drive 1015, a network interface 1016, and an external interface 1017.

[0152] The processor 1001 functions as an arithmetic processing device and a control device, and controls the overall operation of the information processing device 1000 in accordance with various programs. The processor 1001 cooperates with the ROM 1002, RAM 1003, and software described below, to realize the functions of the control unit 33 or the control unit 45, for example.

[0153] The ROM 1002 stores programs and calculation parameters used by the processor 1001. The RAM 1003 temporarily stores programs used in the execution of the processor 1001, and parameters that change as appropriate during the execution of the programs.

[0154] The processor 1001, ROM 1002, and RAM 1003 are connected to each other via an internal bus 1004, and are further connected to a display device 1011, an input device 1012, an audio output unit 1013, a storage device 1014, a drive 1015, a network interface 1016, and an external interface 1017, which will be described later, via an input / output interface 1010.

[0155] The display device 1011 is, for example, a display device such as a CRT display device, a liquid crystal display (LCD), or an OLED device, and converts video data into video and outputs the video. The display device 1011 may form, for example, the display unit 42. The input device 1012 may be composed of a mouse, a keyboard, a touch panel, buttons, a microphone, a sensor, a switch, a control circuit, and the like. The input device 1012 may form, for example, the input unit 43. The audio output unit 1013 is an audio output device such as a speaker or headphones, and converts audio data and the like into audio and outputs the audio.

[0156] The storage device 1014 is a device for storing data. The storage device 1014 may include a storage medium, a recording device for recording data on the storage medium, a reading device for reading data from the storage medium, and a deleting device for deleting data recorded on the storage medium. The storage device 1014 may be configured, for example, as a hard disk drive (HDD) or a solid state drive (SSD), or as a memory having equivalent functions. The storage device 1014 drives the storage and stores programs executed by the processor 1001 or various data. The storage device 1014 may form, for example, the storage unit 32 or the storage unit 44.

[0157] The drive 1015 is a reader / writer for a storage medium, and is built into or externally attached to the information processing device 1000. The drive 1015 reads information stored in a removable storage medium, such as an attached magnetic disk, optical disk, magneto-optical disk, or semiconductor memory, and outputs the information to the RAM 1003. The drive 1015 can also write information to the removable storage medium.

[0158] The network interface 1016 is a communication interface configured with, for example, a device for connecting to a communication network such as the Internet. The network interface 1016 may also be a communication device compatible with a wireless LAN (Local Area Network) and / or a wired LAN. The network interface 1016 may form, for example, the communication unit 31 or the communication unit 41.

[0159] The external interface 1017 is a connection interface consisting of connection ports for connecting external devices, such as a USB (Universal Serial Bus) port, an IEEE 1394 port, a SCSI (Small Computer System Interface) port, an RS-232C port, or an optical audio terminal.

[0160] The above describes an example of a hardware configuration capable of realizing the functions of the information processing device 1000 according to this embodiment. Each of the above components may be realized using general-purpose components, or may be realized by hardware specialized for the function of each component. Therefore, the hardware configuration used can be changed as appropriate depending on the technical level at the time of implementing this embodiment.

[0161] <5. Supplementary Information> Although the preferred embodiments of the present invention have been described in detail above with reference to the accompanying drawings, the present invention is not limited to such examples. It is clear that a person skilled in the art to which the present invention pertains can conceive of various modified or altered examples within the scope of the technical idea described in the claims, and it is understood that these also naturally fall within the technical scope of the present invention.

[0162] For example, in the above embodiment, an example has been described in which the present invention is applied to a highway construction site S1 where humans and robots work together and where these operations are managed by humans. However, the present invention is not limited to such an example. The present invention can be applied to any work site managed by humans, such as a building construction site or facility security. Furthermore, the present invention is not limited to work sites where humans and robots work together, but may be applied to work sites where only humans or only robots work. Furthermore, the work site is not limited to outdoors, but may also be indoors.

[0163] For example, in the above embodiment, an example has been described in which the situation data is acquired by the sensor 10 or input by the site supervisor D, but the present invention is not limited to such an example. The situation data may be estimated by machine learning based on feature amounts extracted from the screen capture data.

[0164] For example, in the above embodiment, an example has been described in which the display of the field terminal 40A is controlled by the control device 30, but the present invention is not limited to such an example. At least a portion of the display of the field terminal 40A may be controlled by the field terminal 40A itself. As an example, the panel P displayed on the field terminal 40A may be generated by the field terminal 40A itself. In this case, the control device 30 transmits information used to generate a screen by the field terminal 40A (e.g., various situation data and information indicating a recommended action plan) to the field terminal 40A, and the field terminal 40A may generate and display the panel P based on the received information. The same applies to the display of the center terminal 40B.

[0165] For example, in the above embodiment, an example was described in which panels P4 to P7 were displayed on the left screen block B1 of one monitor, and panels P1 to P3 and P8 were displayed on the right screen block B2, but the present invention is not limited to this example. These panels P may be displayed separately on multiple monitors.

[0166] Alternatively, as shown in Fig. 15, multiple panels P may be displayed on a single screen without dividing the screen. Fig. 15 is a diagram showing another example of a screen displayed on the operation device 40 according to this embodiment. In the example shown in Fig. 15, the display unit 42 of the operation device 40 displays panels P1, P2, P4, P5, P7, and P8 on a single screen, i.e., without dividing the screen, and further displays a notification PU of an abnormality that has occurred as a pop-up. In this way, the panel P displayed in the left screen block B1 and the panel P displayed in the right screen block B2 may be displayed on the same screen.

[0167] For example, in the above embodiment, an example has been described in which the definition unit 334 defines (i.e., newly creates) an event determination table and an action plan selection table, but the present invention is not limited to such an example. The definition unit 334 may acquire (e.g., download via a communication path) and reuse an event determination table and an action plan selection table defined in a device other than the control device 30.

[0168] The series of processes performed by each device described herein may be implemented using software, hardware, or a combination of software and hardware. The software programs may be stored in advance, for example, on a recording medium (more specifically, a non-transitory computer-readable storage medium) internal or external to each device. Each program is then loaded into a random access memory (RAM) and executed by a processing circuit such as a central processing unit (CPU). The recording medium may be, for example, a magnetic disk, an optical disk, a magneto-optical disk, or a flash memory. The computer program may also be distributed, for example, via a network, without using a recording medium. The computer may be, for example, an application-specific integrated circuit (ASIC), a general-purpose processor that executes functions by loading a software program, or a computer on a server used in cloud computing. The series of processes performed by each device described herein may be centrally processed by a single computer or distributed across multiple computers. Furthermore, in each of the above embodiments, two or more communication means present in a single device may be physically implemented on a single medium.

[0169] Furthermore, the processes described herein using flowcharts or sequence diagrams do not necessarily have to be performed in the order shown. Some process steps may be performed in parallel. Furthermore, additional process steps may be employed, and some process steps may be omitted.

[0170] 1 System S1 Expressway construction site 10 Sensor 20 Robot W Worker O Restricted material (O1 to O5) D Site supervisor M Center manager M 30 Control device 31 Communication unit 32 Memory unit 33 Control unit 331 Event management unit 332 Action plan selection support unit 333 Skill management unit 334 Definition unit 40 Operation device (40A: field terminal, 40B: center terminal) 41 Communication unit 42 Display unit 43 Input unit 44 Memory unit 45 Control unit B1 Left screen block B2 Right screen block P1 List display panel of status data items P11 Check box for entering YES P12 Check box for entering NO P2 Action plan execution panel P21 Action plan execution button P22 Message showing recommended action plan P3 Flowchart panel P4 Image panel P5 Robot information panel 21 Indicator showing the remaining battery level of the robot P6 Information panel relating to the worker G Graph showing the time series change in biometric information P7 Map information panel SM Map P8 Event response list panel 1000 Information processing device 1001 CPU 1002 ROM 1003 RAM 1004 Internal bus 1010 Input / output interface 1011 Display device 1012 Input device 1013 Audio output unit 1014 Storage device 1015 Drive 1016 Network interface 1017 External interface

Claims

1. An information processing device comprising: an acquisition unit that repeatedly acquires situation data indicating the situation of a work site in which a user is involved; a definition unit that defines an action plan selection table by matching action plans, which are options for one or more actions that can be executed when an event occurs at the work site, with one or more of the situation data that are conditions for selecting the action plan; a memory unit that stores the action plan selection table for a plurality of the events that can occur at the work site; and a control unit that controls a process of recommending one or more of the action plans to be selected by the user based on the situation data acquired by the acquisition unit and the action plan selection table for the event that occurred at the work site.

2. The information processing device according to claim 1, wherein the control unit selects the recommended action plan based on a degree of match between the situation data acquired by the acquisition unit and the situation data defined in the action plan selection table.

3. The information processing device according to claim 1, wherein the control unit controls a process of generating a display screen that accepts input of the situation data and presenting it to the user, and the acquisition unit acquires the situation data input on the display screen.

4. The information processing device according to claim 3, wherein the control unit controls a process of presenting to the user information that the user should refer to when inputting the situation data.

5. The information processing device according to claim 3, wherein the control unit controls a process of preferentially accepting input of situation data for items of situation data defined in the action plan selection table, the situation data corresponding to which differs between multiple action plans.

6. The information processing device according to claim 3, wherein the control unit determines the skill of the user, and if the skill of the user is lower than a threshold value, notifies a center administrator who manages the user.

7. The information processing device according to claim 6, wherein the control unit controls a process of presenting the display screen being presented to the user to the center manager when the skill of the user is lower than a threshold value.

8. The information processing device according to claim 6, wherein the control unit transfers authority to select the action plan to be executed from the user to the center administrator when the skill of the user is lower than a threshold value.

9. The information processing device of claim 6, wherein the memory unit stores historical data including the event that has occurred and the situation data acquired from the occurrence of the event to its completion, and the control unit judges the skill of the user based on the historical data.

10. The information processing device according to claim 9, wherein the control unit determines the skill of the user based on the number of times the user has completed the event in the past without notifying the center administrator.

11. The information processing device according to claim 9, wherein the control unit determines the skill of the user for each event based on the number of times the user has completed the event in the past.

12. The information processing device according to claim 9, wherein the control unit determines the skill of the user for each action plan based on the number of times the user has selected the action plan in the past.

13. The information processing device of claim 1, wherein the memory unit stores history data including the event that has occurred and the status data acquired from the occurrence of the event to its completion, and the control unit updates the action plan selection table based on the history data.

14. The information processing device according to claim 1, wherein the acquisition unit acquires the situation data acquired by a sensor disposed at the work site.

15. The information processing device according to claim 1, wherein the situation data includes information regarding a robot working at the work site.

16. The information processing device according to claim 1, wherein the situation data includes information about workers working at the work site.

17. The information processing device according to claim 1, wherein the situation data includes information relating to an item that is the subject of work at the work site.

18. An information processing method executed by a computer, comprising: repeatedly acquiring situation data indicating a situation of a work site in which a user is involved; a definition unit that defines an action plan selection table that associates action plans, which are options for one or more actions that can be executed when an event occurs at the work site, with one or more of the situation data that are conditions for selecting the action plan; storing the action plan selection table for a plurality of the events that can occur at the work site; and controlling a process that recommends one or more of the action plans to be selected by the user based on the acquired situation data and the action plan selection table for the event that occurred at the work site.

19. A non-transitory storage medium having stored therein a program for causing a computer to function as: an acquisition unit that repeatedly acquires situation data indicating the situation of a work site in which a user is involved; a definition unit that defines an action plan selection table that associates action plans, which are options for one or more actions that can be executed when an event occurs at the work site, with one or more of the situation data that are conditions for selecting the action plan; a memory unit that stores the action plan selection table for a plurality of the events that can occur at the work site; and a control unit that controls a process of recommending one or more of the action plans to be selected by the user based on the situation data acquired by the acquisition unit and the action plan selection table for the event that occurred at the work site.

Images