Information processing device, information processing method, and program

Abstract

To provide a mechanism for supporting human decision-making at a work site.SOLUTION: An information processing device includes: an acquisition unit that repeatedly acquires situation data indicating a situation at a work site in which a user participates; a storage unit that stores, for a plurality of events that can occur at the work site, an action plan selection table that defines one or more action plans that can be executed when each event occurs at the work site and one or more pieces of the situation data that are conditions for selecting the action plans so as to associate them with each other; and a control unit that controls processing for recommending the one or more action plans to be selected by the user on the basis of the situation data acquired by the acquisition unit and the table related to the event that occurs at the work site.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an information processing device, an information processing method, and a program. [Background technology]

[0002] In recent years, robotics technology has been attracting attention as one of the means to solve the labor shortage in a society with a declining population, and technologies to smoothly utilize robots in work sites have been developed. For example, the following Patent Document 1 proposes an algorithm for making decisions that eliminates personal dependency regarding robots introduced in work sites for logistics management. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 6427285 Summary of the Invention [Problem to be solved by the invention]

[0004] As disclosed in the above Patent Document 1 and the like, a robot can operate autonomously according to a predetermined algorithm. However, when some abnormality occurs, it may be difficult for the robot alone to deal with it appropriately, so humans often deal with it on behalf of the robot or by assisting the robot. However, due to factors such as difficulty in properly grasping the situation, even humans may find it difficult to deal with the abnormality that has occurred appropriately.

[0005] Therefore, the present invention has been made in consideration of the above problems, and an object of the present invention is to provide a mechanism capable of supporting human decision-making at a work site. [Means for solving the problem]

[0006] In order to solve the above problem, according to one aspect of the present invention, there is provided an information processing device comprising: an acquisition unit that repeatedly acquires situation data indicating a situation of a work site in which a user is involved; a memory unit that stores, for a plurality of events that may occur at the work site, an action plan selection table that defines a correspondence between one or more action plans that may be executed when an event occurs at the work site and one or more of the situation data that are conditions for selecting the action plan; and a control unit that controls a process of recommending one or more of the action plans to be selected by the user based on the situation data acquired by the acquisition unit and the table regarding the events that have occurred at the work site.

[0007] The control unit may select the recommended action plan based on a degree of match between the situation data acquired by the acquisition unit and the situation data defined in the action plan selection table.

[0008] The control unit may control a process of generating a display screen for accepting input of the situation data and presenting the display screen to the user, and the acquisition unit may acquire the situation data input on the display screen.

[0009] The control unit may control a process of presenting to the user information that the user should refer to when inputting the situation data.

[0010] The control unit may control a process of preferentially accepting input of situation data for items of situation data defined in the action plan selection table, the situation data corresponding to which differs between multiple action plans.

[0011] The control unit may determine the skill of the user, and if the skill of the user is lower than a threshold, notify a center administrator who manages the user.

[0012] The control unit may control a process of presenting the display screen being presented to the user to the center manager when the skill of the user is lower than a threshold value.

[0013] The control unit may transfer authority to select the action plan to be executed from the user to the center administrator when the skill of the user is lower than a threshold value.

[0014] The memory unit may store historical data including the event that has occurred and the situation data acquired from the time the event occurred to its completion, and the control unit may determine the skill of the user based on the historical data.

[0015] The control unit may determine the skill of the user based on the number of times the user has completed the event in the past without notifying the center administrator.

[0016] The control unit may determine the skill of the user for each event based on the number of times the user has completed the event in the past.

[0017] The control unit may determine the skill of the user for each action plan based on the number of times the user has selected the action plan in the past.

[0018] The memory unit may store historical data including the event that has occurred and the status data acquired from the time the event occurred to its completion, and the control unit may update the action plan selection table based on the historical data.

[0019] The acquisition unit may acquire the situation data acquired by a sensor arranged at the work site.

[0020] The situation data may include information regarding a robot working at the work site.

[0021] The situation data may include information about workers working at the work site.

[0022] The situation data may include information regarding an item being worked on at the work site.

[0023] In addition, in order to solve the above problem, according to another aspect of the present invention, there is provided an information processing method executed by a computer, which includes repeatedly acquiring situation data indicating a situation of a work site in which a user is involved, storing an action plan selection table for a plurality of events that may occur at the work site, the action plan selection table defining a correspondence between one or more action plans that may be executed when an event occurs at the work site and one or more situation data that are conditions for selecting the action plan, and controlling a process of recommending one or more of the action plans to be selected by the user based on the acquired situation data and the table related to the events that have occurred at the work site.

[0024] In addition, in order to solve the above problem, according to another aspect of the present invention, a program is provided to cause a computer to function as: an acquisition unit that repeatedly acquires situation data indicating the situation of a work site in which a user is involved; a memory unit that stores, for a plurality of events that may occur at the work site, an action plan selection table that defines a correspondence between one or more action plans that may be executed when an event occurs at the work site and one or more situation data that are conditions for selecting the action plan; and a control unit that controls a process of recommending one or more action plans to be selected by the user based on the situation data acquired by the acquisition unit and the table regarding the events that have occurred at the work site. Effect of the Invention

[0025] As described above, according to the present invention, a mechanism capable of supporting human decision-making at a workplace is provided. [Brief description of the drawings]

[0026] [Figure 1] FIG. 1 is a diagram showing a preferred application example of a system 1 according to an embodiment of the present invention. [Diagram 2] FIG. 2 is a block diagram showing an example of the configuration of a terminal device 40 according to the present embodiment. [Diagram 3] FIG. 13 is a diagram showing an example of a panel P4 displayed on a field terminal 40A according to the present embodiment. [Figure 4] FIG. 13 is a diagram showing an example of a panel P5 displayed on a field terminal 40A according to the present embodiment. [Diagram 5] FIG. 13 is a diagram showing an example of a panel P6 displayed on a field terminal 40A according to the present embodiment. [Figure 6] FIG. 13 is a diagram showing an example of a panel P7 displayed on a field terminal 40A according to the present embodiment. [Figure 7] 11 is a diagram showing an example of a panel P1 displayed on a field terminal 40A according to the present embodiment. FIG. [Figure 8] 13 is a diagram showing an example of a panel P2 displayed on a field terminal 40A according to the present embodiment. FIG. [Figure 9] FIG. 13 is a diagram showing an example of a panel P3 displayed on a field terminal 40A according to the present embodiment. [Figure 10] FIG. 13 is a diagram showing an example of a panel P8 displayed on a field terminal 40A according to the present embodiment. [Figure 11] FIG. 2 is a block diagram showing an example of the configuration of a control device 30 according to the present embodiment. [Figure 12] 4 is a flowchart showing an example of a flow of processing executed by the system 1 according to the present embodiment. [Figure 13] 4 is a flowchart showing an example of a flow of processing executed by the system 1 according to the present embodiment. [Figure 14] 1 is a block diagram showing an example of a hardware configuration of an information processing device according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0027] Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In this specification and the drawings, components having substantially the same functional configurations are designated by the same reference numerals, and duplicated explanations will be omitted.

[0028] <1. Overview> Fig. 1 is a diagram showing a suitable application example of a system 1 according to an embodiment of the present invention. As shown in Fig. 1, the system 1 according to this embodiment includes a sensor 10, a robot 20, a control device 30, and terminal devices 40 (40A and 40B). The control device 30, the sensor 10, the robot 20, and the terminal devices 40 are connected to each other via a network so as to be able to communicate with each other.

[0029] System 1 supports human decision-making at a work site where human workers and autonomously operating robots coexist. In the example shown in Figure 1, the work site to which System 1 is applied is a highway construction site S1.

[0030] Restrictive materials O (O1 to O5) are placed at the highway construction site S1. The restrictive materials O are used on highways and the like to indicate to passing vehicles that entry into restricted areas is prohibited or to instruct vehicles to change lanes. The restrictive materials O are an example of items that are the subject of work at the highway construction site S1.

[0031] A sensor 10 is placed at a construction site S1 on a highway. The sensor 10 acquires various information on the construction site S1 on the highway. The sensor 10 includes, for example, a camera, and acquires images (moving images or still images) capturing the state of the construction site S1 on the highway. For example, the sensor 10 may also include an infrared sensor, a depth sensor, a microphone, or the like.

[0032] At the highway construction site S1, workers W and robots 20 are deployed to perform predetermined tasks. Examples of predetermined tasks include monitoring whether or not problems such as tipping over have occurred in the restricted material O, and dealing with any problems that have occurred. For example, if the area of ​​responsibility is 1 km per person or vehicle, then for a restricted area of ​​10 km, the workers W and robots 20 are deployed so that the total number of workers W and robots 20 is 10.

[0033] The robot 20 has a moving means such as a motor and wheels, and can move. Furthermore, the robot 20 has a robot arm, and can grasp and move the regulating material O or change the posture of the regulating material O. The robot 20 has various sensors such as a camera, and can operate autonomously while recognizing the surrounding situation. The robot 20 has a rechargeable battery, and is powered by the battery.

[0034] The workers W and the robot 20 arranged at the highway construction site S1 are managed by one site supervisor D. For example, the site supervisor D issues instructions regarding the work to the workers W and the robot 20 while checking the status of the workers W and the robot 20. The site supervisor D is an example of a user involved in the work site.

[0035] The terminal device 40A is an information processing device used by the site supervisor D. The terminal device 40A may be implemented as, for example, a tablet terminal or a smartphone. The terminal device 40A is also referred to as a site terminal 40A below.

[0036] The on-site terminal 40A functions as an interface between the system 1 and the site manager D. As an example, the on-site terminal 40A outputs information indicating the status of the highway construction site S1 acquired by the sensor 10 or the like. As another example, the on-site terminal 40A accepts input of instructions for the worker W and the robot 20. As shown in FIG. 1, the screen displayed on the monitor (display unit 42 shown in FIG. 2) of the on-site terminal 40A can be divided into two screen blocks, a left screen block B1 and a right screen block B2, on the left and right sides. Of course, the number of divisions of the screen may be three or more, or the screen may not be divided at all. The left screen block B1 displays a panel P4 (FIG. 3) described later.

[0037] The site supervisor D is managed by a center manager M. For example, the center manager M monitors whether instructions by the site supervisor D are appropriate or not, and issues instructions to the workers W and the robot 20 on behalf of the site supervisor D. The center manager M can manage multiple site supervisors D who manage multiple highway construction sites S1.

[0038] The terminal device 40B is an information processing device used by the center manager M. The terminal device 40B is implemented as, for example, a PC (Personal Computer). The terminal device 40B is also referred to as a center terminal 40B below.

[0039] The center terminal 40B functions as an interface between the system 1 and the center manager M. As an example, the center terminal 40B outputs information (hereinafter also referred to as situation data) showing the status of the construction sites S1 of a plurality of expressways managed by the center manager M. As another example, the center terminal 40B accepts input of instructions for the workers W and the robots 20 managed by a plurality of site supervisors D managed by the center manager M. As shown in FIG. 1, the center terminal 40B may share a screen with the on-site terminal 40A, and may display the same screen as the screen displayed by the on-site terminal 40A on a monitor (display unit 42 shown in FIG. 2). This allows the center manager M to monitor whether the management by the on-site supervisor D is appropriate or not while referring to the same screen as the screen displayed by the on-site terminal 40A. Of course, the center terminal 40B may display a screen different from that of the on-site terminal 40A.

[0040] The control device 30 is an information processing device that controls the overall operation of the system 1. The control device 30 collects situation data of the highway construction site S1 and controls the operation of the system 1. For example, the control device 30 controls the display of the on-site terminal 40A and the center terminal 40B based on the collected situation data. An example of the situation data will be described below.

[0041] For example, the situation data may include information acquired by the sensor 10 installed at the highway construction site S1. In particular, the situation data may include images, infrared information, depth information, and audio of the highway construction site S1.

[0042] For example, the situation data may include information about the regulating material O that is the subject of work at the highway construction site S1. The information about the regulating material O includes position information about the regulating material O, inertial information about the regulating material O (e.g., acceleration and angular velocity), and, if the regulating material O has a built-in battery for driving a sensor or the like, the remaining battery charge. This information about the regulating material O is acquired by a sensor provided on the regulating material O and transmitted to the control device 30.

[0043] For example, the situation data may include information about the robot 20 working at a construction site S1 on a highway. The information about the robot 20 may include the remaining battery level of the robot 20, images captured by a camera mounted on the robot 20, position information of the robot 20, and the operation history of the robot arm. The information about the robot 20 is acquired by a sensor provided on the robot 20 and transmitted to the control device 30.

[0044] For example, the situation data may include information about a worker W working at a construction site S1 on a highway. The information about the worker W may include biological information such as body temperature and heart rate acquired by a vital sensor worn by the worker W, an image captured by a body camera worn by the worker W, and position information about the worker W. This information about the worker W is acquired, for example, by a wearable terminal worn by the worker W, and transmitted to the control device 30.

[0045] For example, the situation data may include information about the site supervisor D who manages the robot 20 and the worker W performing work at the construction site S1 on the highway. Examples of the information about the site supervisor D include information output by the on-site terminal 40A to the site supervisor D and information input by the site supervisor D to the on-site terminal 40A. The information about the site supervisor D is acquired by the on-site terminal 40A and transmitted to the control device 30.

[0046] For example, the situation data may include information about the center manager M who manages the site supervisor D. Examples of the information about the center manager M include information output by the center terminal 40B to the center manager M and information input by the center manager M to the center terminal 40B. This information about the center manager M is acquired by the center terminal 40B and transmitted to the control device 30.

[0047] An example of control performed by the control device 30 based on the collected situation data will be described below.

[0048] As shown in FIG. 1, it is assumed that the restricting material O3 has fallen over and moved to a position different from the original installation position. Such a problem occurring at the highway construction site S1 is also referred to as an event below. The site supervisor D completes the event by executing an action plan, such as instructing the robot 20 or the worker W to recover. An action plan is a selection of actions that can be executed to complete the event that has occurred. Completion of an event means that the problem has been resolved, and for example, in the case of the restricting material O3 having fallen over, it means that the restricting material O3 has been returned to its original position and posture. In addition, the fact that an action plan has been executed is also referred to as the completion of the action plan.

[0049] When there are multiple action plans that can be executed for an event that has occurred, the site supervisor D selects which action plan to execute. However, it may be difficult to appropriately select an action plan to execute. In particular, when the site supervisor D lacks skills, it becomes even more difficult to appropriately select an action plan to execute. The skills here refer to the expertise to analyze and interpret situation data, knowledge of local rules determined at the work site, and the like. If an inappropriate action plan is selected, various inconveniences may occur, such as delaying the completion of the event, increasing the load on the robot 20 and the worker W, and hindering construction at the highway construction site S1.

[0050] In addition, all of the situation data required to select an appropriate action plan may not be automatically acquired by the sensor 10 or the like, and may require manual input by the site supervisor D. Considering the wide variety of information displayed on the on-site terminal 40A, it may be difficult for the site supervisor D to manually input appropriate situation data, especially when the site supervisor D lacks skills.

[0051] Therefore, the system 1 according to the present embodiment supports the site supervisor D to select an appropriate action plan. Furthermore, the system 1 judges the skills of the site supervisor D and provides support according to the skills of the site supervisor D. With this configuration, an appropriate action plan is selected, so that the event can be completed early and inconveniences caused by delays in the completion of the event can be prevented.

[0052] <2. Configuration example> (1) Terminal device 40 2 is a block diagram showing an example of the configuration of the terminal device 40 according to the present embodiment. As shown in FIG. 2, the terminal device 40 includes a communication unit 41, a display unit 42, an input unit 43, a storage unit 44, and a control unit 45.

[0053] (Communications Department 41) The communication unit 41 has a function of performing communication in compliance with any wired or wireless communication standard with other devices included in the system 1. For example, the communication unit 41 transmits status data to the control device 30 and receives control information for controlling the operation of the terminal device 40.

[0054] (Display section 42) The display unit 42 has a function of outputting images to a user (i.e., the site supervisor D or the center manager M) who operates the terminal device 40. In particular, the display unit 42 displays panels P (P1 to P8) described below, which are screens for displaying information showing the state of the highway construction site S1 and for accepting input of situation data. In addition, the terminal device 40 may have an output device that outputs sound, vibration, or the like.

[0055] (Input section 43) The input unit 43 has a function of accepting information input from a user who operates the terminal device 40. The display unit 42 and the input unit 43 may be integrally configured as, for example, a touch panel. In addition, the input unit 43 may accept information input via a keyboard or a mouse, voice input, or gesture input.

[0056] (Storage unit 44) The storage unit 44 has a function of storing information for the operation of the terminal device 40. The storage unit 44 can store, for example, control information received from the control device 30 and situation data input by the user.

[0057] (Control unit 45) The control unit 45 has a function of controlling the overall operation of the terminal device 40. In particular, the control unit 45 controls the operation of the terminal device 40 in accordance with control information received from the control device 30.

[0058] As an example, the control unit 45 controls the display by the display unit 42. For example, the control unit 45 controls the display unit 42 to display the panel P generated by the control device 30, and switches the panel P to be displayed in response to a user operation. The control unit 45 also controls the storage unit 44 to store situation data input to the input unit 43 for the panel P displayed on the display unit 42, and controls the communication unit 41 to transmit the situation data to the control device 30.

[0059] As another example, the control unit 45 may control a process of capturing (i.e., recording) the screen displayed on the display unit 42 and transmitting the captured data (hereinafter, screen capture data) to the control device 30. Note that the screen capture data may be a video or a collection of still images recorded at a specific timing, such as when the user clicks a button.

[0060] (Example of display screen) An example of a screen displayed on the field terminal 40A will be described in detail below. Note that, by sharing the screen from the field terminal 40A, the center terminal 40B can also display the same screen as the screen exemplified below.

[0061] As described above with reference to Fig. 1, the screen displayed by the on-site terminal 40A can be divided into two screen blocks, a left screen block B1 and a right screen block B2, on the left and right sides. A panel P is displayed on each of the left screen block B1 and the right screen block B2. The panel P is a set of UI (User Interface) that displays predetermined information and accepts input of predetermined information.

[0062] -Display of left screen block B1 The left screen block B1 may switch between displaying one panel P among the multiple panels P. For example, the left screen block B1 may display only the top panel P, and switch the displayed panel P in response to a scroll operation or a click operation on a link displayed at the top. Alternatively, the left screen block B1 may display multiple panels P simultaneously.

[0063] The left screen block B1 may display panels P (P4 to P7) for checking the status of the highway construction site S1. The site manager D can check the status of the highway construction site S1 by referring to the panels P4 to P7 displayed in the left screen block B1. An example of the panels P4 to P7 will be described below with reference to Figs. 3 to 6.

[0064] 3 is a diagram showing an example of a panel P4 displayed on the on-site terminal 40A according to the present embodiment. The panel P4 displays an image of a highway construction site S1. In the example shown in FIG. 3, a state in which a restricting material O3 has fallen over is displayed on the panel P4. The panel P4 may display one image as shown in FIG. 3, or may display multiple images of the highway construction site S1 captured from various angles / ranges.

[0065] Fig. 4 is a diagram showing an example of a panel P5 displayed on the on-site terminal 40A according to this embodiment. The panel P5 displays information related to the robot 20. In the example shown in Fig. 4, the panel P5 displays an indicator 21 indicating the remaining battery level of the robot 20 on the left side, and displays an overall image of the robot 20 on the right side.

[0066] Fig. 5 is a diagram showing an example of a panel P6 displayed on the on-site terminal 40A according to the present embodiment. The panel P6 displays information about the worker W. In the example shown in Fig. 5, the panel P6 displays a graph G showing a time series transition of the biometric information of the worker W on the left side, and displays an overall image of the worker W on the right side.

[0067] FIG. 6 is a diagram showing an example of a panel P7 displayed on the on-site terminal 40A according to the present embodiment. The panel P7 displays map information in which the positions of the regulating material O, the robot 20, and the worker W are mapped on a map. In the example shown in FIG. 6, a triangle is mapped to the position of the regulating material O, a circle is mapped to the position of the robot 20, and a diamond is mapped to the position of the worker W on a map SM showing an overall image of a highway construction site S1. The on-site terminal 40A may display the regulating material O in which an abnormality has occurred and the normal regulating material O in different display forms, such as highlighting only the fallen regulating material O by changing the color. Note that in FIG. 6, a part of the triangle display indicating the regulating material O is omitted. In this way, the display of the normal regulating material O may be omitted.

[0068] - Display of right screen block B2 The right screen block B2 may switch between displaying one panel P among the multiple panels P. For example, the right screen block B2 may display only the top panel P, and switch the displayed panel P in response to a scroll operation or a click operation on a link displayed at the top. Alternatively, the right screen block B2 may display multiple panels P simultaneously.

[0069] The right screen block B2 may display panels P1-P3 and P8 that accept various inputs for managing the highway construction site S1. The site manager D can input situation data and select an action plan to be executed by referring to P1-P3 and P8 displayed on the right screen block B2. An example of the panels P1-P3 and P8 will be described below with reference to Figs. 7-10.

[0070] FIG. 7 is a diagram showing an example of a panel P1 displayed on the on-site terminal 40A according to the present embodiment. The panel P1 displays a list of items of situation data that are desirable to input in order to select an appropriate action plan. In the example shown in FIG. 7, the panel P1 displays a list of items of situation data defined in an action plan selection table shown in Table 2 described later. On the left side of each item of situation data, a check box P11 for inputting YES and a check box P12 for inputting NO are displayed, and the site supervisor D can input situation data by inputting in such check boxes. Note that the item No. 4 "Is action plan A completed?" and the item No. 5 "Is action plan B completed?" defined in the action plan selection table shown in Table 2 described later are automatically updated, so the site supervisor D does not need to input these situation data.

[0071] FIG. 8 is a diagram showing an example of a panel P2 displayed on the on-site terminal 40A according to this embodiment. The panel P2 displays action plans and accepts a user operation to select an action plan to be executed. In the example shown in FIG. 8, the panel P2 displays a list of action plans defined in an action plan selection table shown in Table 2, which will be described later. The site supervisor D can execute an action plan by selecting an execution button P21 to the right of the action plan. Furthermore, if a recommended action plan has been identified by the control device 30, a message P22 indicating the recommended action plan is displayed.

[0072] 9 is a diagram showing an example of a panel P3 displayed on the on-site terminal 40A according to this embodiment. The panel P3 displays a flowchart showing a series of actions to be taken by the site supervisor D. In the example shown in FIG. 9, the panel P3 displays a flowchart in which the input of situation data defined in an action plan selection table shown in Table 2, which will be described later, and the selection of an action plan are arranged in a desired execution order. By referring to the panel P3, the site supervisor D can confirm the overall flow up to the completion of an event.

[0073] FIG. 10 is a diagram showing an example of a panel P8 displayed on the on-site terminal 40A according to the present embodiment. The panel P8 displays an event list showing a list of events that have occurred so far. In the example shown in FIG. 10, it is displayed that a charging event of the robot 20 has occurred and has been completed, and that a falling event of the regulating material O has occurred and has not been completed. Furthermore, for completed events, action plans that have been executed, etc. may be displayed. By referring to the panel P8, the site supervisor D can grasp events that have occurred in the past and events that are currently occurring. In addition, the site supervisor D can narrow down the action plans to be executed to complete the currently occurring event by checking the action plans that have been executed for events that have occurred in the past.

[0074] (2) Control device 30 11 is a block diagram showing an example of the configuration of the control device 30 according to the present embodiment. As shown in FIG. 11, the control device 30 includes a communication unit 31, a storage unit 32, and a control unit 33.

[0075] (Communications Department 31) The communication unit 31 has a function of performing communication conforming to any wired or wireless communication standard with other devices included in the system 1. For example, the communication unit 31 transmits information for controlling the operation of various devices such as the sensor 10, the robot 20, the field terminal 40A, the center terminal 40B, and sensors built into or attached to the restricted material O and the worker W, and receives information transmitted from the various devices.

[0076] The communication unit 31 is an example of an acquisition unit that repeatedly acquires situation data indicating the situation of the highway construction site S1. For example, the communication unit 31 receives situation data from the sensor 10, the robot 20, the on-site terminal 40A, the center terminal 40B, and sensors built into or attached to the regulated material O and the worker W.

[0077] (Storage unit 32) The storage unit 32 has a function of storing various types of information used for controlling the overall operation of the system 1. An example of information stored in the storage unit 32 will be described below.

[0078] -Event determination table The storage unit 32 stores an event determination table that defines an event that may occur at the highway construction site S1, a condition for determining the occurrence of the event, and an action plan that may be executed when the event occurs, in association with each other. By referring to the event determination table, it becomes possible to appropriately determine an event that has occurred at the highway construction site S1. An example of the event determination table is shown in Table 1 below.

[0079] [Table 1]

[0080] In the example shown in Table 1, the falling of the restricted material O, charging of the robot 20, and poor health of the worker W are defined as events that may occur at the highway construction site S1. For example, it is defined that the falling of the restricted material O is determined based on an email notification that the restricted material O has fallen. Also, action plans A, B, and C are associated as action plans that may be executed when the restricted material O falls.

[0081] -Action plan selection table The storage unit 32 stores an action plan selection table that defines one or more action plans that may be executed when an event occurs at the highway construction site S1, in association with one or more situation data that are conditions for selecting an action plan. The action plan selection table is a setting of situation data for prioritizing one or more action plans that may be executed to complete the event that has occurred. Here, the storage unit 32 stores the above-mentioned action plan selection table for multiple events that may occur at the highway construction site S1. By referring to the action selection table, it is possible to select an appropriate action plan that corresponds to the acquired situation data. An example of an action plan selection table corresponding to "restriction material falls" is shown in Table 2 below.

[0082] [Table 2]

[0083] In the example shown in Table 2, action plan A is defined as instructing the robot 20 to recover, action plan B is defined as instructing the worker W to recover, and action plan C is defined as contacting the center manager M. The execution subject of these action plans is the on-site terminal 40A. For each action plan, one or more pieces of situation data are defined as items 1 to 5 as conditions for selecting the action plan. In Table 2, for example, it is defined that it is desirable to select action plan A when the fallen position of the restricted material O is within the restricted area (item 1: YES), the remaining battery level of the robot 20 is 50% or more (item 2: YES), the event is not completed (item 3: NO), action plan A is not completed (item 4: NO), and action plan B is not completed (item 5: NO).

[0084] In Table 2, the action plan is defined as something being executed, but the present invention is not limited to this example. The action plan may be defined as not executing anything.

[0085] The related panel in Table 2 is a panel displayed on the on-site terminal 40A as a reference for inputting situation data. The on-site terminal 40A displays the related panel in the left screen block B1 when accepting input of the situation data of items 1 to 3 in the right screen block B2. For example, the on-site terminal 40A accepts input of the situation data of item 1 (is the position of the fallen restriction material within the restricted area?) in the right screen block B2 while displaying panel P4 (an image of the construction site S1 on the expressway) in the left screen block B1. On the other hand, for the situation data of items 4 to 5, which are acquired without the intervention of the site supervisor D, the situation data is automatically updated, so no related panel is defined.

[0086] The average confirmation time in Table 2 is a reference value of the time taken from when the related panel is displayed until the situation data is input. The average confirmation time is set as the average value of the time taken from when the related panel is displayed until the situation data is input in events that have occurred in the past. According to the example shown in Table 2, the on-site terminal 40A accepts an input as to whether the fallen position of the restricting material O is within the restricted area or not, on average, 10 seconds after starting to display the panel P4.

[0087] The action plan selection table may be generated in advance and stored in the storage unit 32. Alternatively, the action plan selection table may be updated as described below.

[0088] -Event container The storage unit 32 temporarily stores an event container that is generated when an event occurs. An event container is a data structure that stores information about an event that has occurred. For example, the event container stores information indicating the event that has occurred, information indicating an action plan related to the event that has occurred, and an action plan selection table corresponding to the event that has occurred. Furthermore, the storage unit 32 temporarily stores situation data acquired from the occurrence of an event to its completion in the event container.

[0089] -Event history data The storage unit 32 stores event history data, which is data related to events that have occurred and been completed. When an event is completed, the storage unit 32 stores the temporarily stored event container as event history data (i.e., long-term storage). For example, the storage unit 32 may store the event history data related to events that have occurred and been completed in the form of a list.

[0090] -others The storage unit 32 stores a recommended threshold value, which will be described later. The recommended threshold value may be stored in advance, or may be updated as appropriate.

[0091] The storage unit 32 stores a skill judgment threshold, which will be described later. The skill judgment threshold may be stored in advance or may be updated as appropriate.

[0092] The storage unit 32 stores information indicating the skills of the site supervisor D. The skills of the site supervisor D are updated as appropriate.

[0093] (Control unit 33) The control unit 33 has a function of controlling the overall operation of the control device 30. As shown in Fig. 11, the control unit 33 functions as an event management unit 331, an action plan selection support unit 332, and a skill management unit 333. These components control various processes based on situation data acquired by the communication unit 31 and information stored in the storage unit 32. These components will be described in detail below.

[0094] (Event Management Department 331) The event management unit 331 has a function of managing various information related to events that occur at the construction site S1 on the expressway.

[0095] The event management unit 331 determines an event that has occurred at the highway construction site S1 based on the acquired situation data and the event determination table. For example, the event management unit 331 refers to the event determination table shown in Table 1, and determines that the regulating material O has fallen based on receiving an email notifying the regulating material O of the fall.

[0096] When the event management unit 331 determines that an event has occurred, it generates an event container related to the event that has occurred. Then, the event management unit 331 temporarily stores the event container in the storage unit 32. The event management unit 331 stores, in the event container, status data acquired from the occurrence of the event to its completion. Then, when the event is completed, the event management unit 331 stores the temporarily stored event container in the storage unit 32 as event history data (i.e., long-term storage).

[0097] (Action Plan Selection Support Department 332) The action plan selection support unit 332 has a function of supporting the site supervisor D in selecting an action plan.

[0098] The action plan selection support unit 332 controls the display by the terminal device 40 (the field terminal 40A and the center terminal 40B). In detail, the action plan selection support unit 332 generates a screen to be displayed on the terminal device 40, and transmits it to the terminal device 40 for display. Then, the action plan selection support unit 332 updates the screen in response to a user input to the terminal device 40, and transmits the updated screen to the terminal device 40 for display. The action plan selection support unit 332 may, for example, describe the screen to be displayed on the terminal device 40 using HTML (Hyper Text Markup Language) and transmit the HTML file to the terminal device 40.

[0099] In particular, the action plan selection support unit 332 controls the process of generating a screen displaying information showing the status of the highway construction site S1 and presenting it to the site supervisor D. In detail, the action plan selection support unit 332 generates the above-mentioned panels P4 to P7 and displays them in the left screen block B1 of the on-site terminal 40A. This allows the site supervisor D to easily grasp the status of the highway construction site S1.

[0100] The action plan selection support unit 332 also controls the process of generating a screen for accepting input of situation data and presenting it to the site supervisor D. Then, the action plan selection support unit 332 acquires the situation data input on the screen. In detail, the action plan selection support unit 332 generates the above-mentioned panels P1 to P3 and P8 and displays them on the right screen block B2 of the site terminal 40A. This allows the site supervisor D to input situation data and receive support from the action plan selection support unit 332 regarding the selection of an action plan.

[0101] At that time, the action plan selection support unit 332 may control a process of presenting to the site supervisor D information to be referred to by the site supervisor D when inputting the situation data. For example, the action plan selection support unit 332 controls the site terminal 40A to display a related panel related to an item of situation data that has not been input among one or more items of situation data that are conditions for selecting an action plan defined in the action plan selection table. When there are multiple items of situation data that have not been input, the action plan selection support unit 332 may extract the items of situation data in order from the item number that is the smallest. For example, when only the situation data of item number 1 in the action plan selection table shown in Table 2 has been input, the action plan selection support unit 332 controls the site terminal 40A to display the panel P5, which is a related panel of the item of situation data of item number 2, in the left screen block B1. The site supervisor D can easily input whether the remaining battery level of the robot, which is the situation data of item number 2, is 50% or more by checking the indicator 21 that indicates the remaining battery level of the robot 20 displayed on the panel P5. In this way, by dynamically switching the display of the left screen block B1 to the panel related to the item of situation data that has not been input, it is possible to make it easier for the site supervisor D to input situation data.

[0102] Displaying the related panel on the on-site terminal 40A refers to displaying the related panel in the range visible to the site supervisor D, such as displaying the related panel at the top of the left screen block B1 or displaying the related panel as a pop-up. Furthermore, the action plan selection support unit 332 may make the frame of the related panel red or blink. With this configuration, it is possible to encourage attention to the related panel.

[0103] When the situation data is acquired, the action plan selection support unit 332 controls a process of recommending an appropriate action plan based on the acquired situation data. More specifically, the action plan selection support unit 332 controls a process of recommending one or more action plans to be selected based on the acquired situation data and an action plan selection table related to the event that has occurred. As an example, the action plan selection support unit 332 may control the on-site terminal 40A to display a message (e.g., message P22 on the panel P2) recommending an appropriate one of the action plans A to C. As another example, the action plan selection support unit 332 may control the on-site terminal 40A to display the priority of the action plans A to C defined in the action plan selection table, or to sort and display the action plans A to C in order of priority. With this configuration, it is possible to support the decision-making of the site supervisor D.

[0104] Here, the action plan selection support unit 332 selects a recommended action plan based on the matching rate between the acquired situation data and the situation data as a selection condition defined in the action plan selection table. In detail, the action plan selection support unit 332 may control the on-site terminal 40A to recommend an action plan of which the matching rate of the situation data exceeds a predetermined threshold (hereinafter also referred to as a recommended threshold) among the action plans A to C. Furthermore, the action plan selection support unit 332 may control the on-site terminal 40A to recommend an action plan of which the matching rate of the situation data is the highest among the action plans A to C. For example, it is assumed that situation data in which item number 1 is YES, item number 2 is YES, item number 3 is NO, item number 4 is NO, and item number 5 is NO in the action plan selection table shown in Table 2 is acquired. In this case, the matching rate of the situation data of the action plan A is 100%, the matching rate of the situation data of the action plan B is 60%, and the matching rate of the situation data of the action plan C is 20%. When the recommendation threshold is 50%, the action plan selection support unit 332 may recommend action plan A as the optimal action plan, action plan B as the second optimal action plan, and so on, in order of priority. The action plan selection support unit 332 may also control the on-site terminal 40A to recommend only action plan A, which has the highest matching rate with the situation data, to the site supervisor D. With this configuration, it becomes possible to recommend an appropriate action plan to the site supervisor D in accordance with the action plan selection table.

[0105] Here, as the input of the situation data progresses, the matching rate of the situation data may change. The action plan selection support unit 332 may change the recommended action plan according to the change in the matching rate of the situation data. For example, immediately after the occurrence of an event, the situation data of items 1 to 2 in the action plan selection table shown in Table 2 is not input, and the situation data of items 3 to 5 is NO. In this case, the matching rate of the situation data of the action plan A is 60%, the matching rate of the situation data of the action plan B is 60%, and the matching rate of the situation data of the action plan C is 20%. Therefore, when the recommended threshold is 50%, the action plan selection support unit 332 may recommend the action plans B and A in that order. After that, it is assumed that YES is input as the situation data of item 1. In this case, the matching rate of the situation data of the action plan A is 80%, the matching rate of the situation data of the action plan B is 60%, and the matching rate of the situation data of the action plan C is 20%. Therefore, the action plan selection support unit 332 may recommend the action plans A and B in that order.

[0106] The action plan selection support unit 332 may control a process of preferentially accepting input of situation data of items of situation data corresponding to different situation data among the items of situation data defined in the action plan selection table. For example, in the action plan selection table shown in Table 2, the selection conditions for the items of situation data of items 1 and 2 between action plan A and action plan B are different in YES / NO, and the selection conditions for the items of situation data of items 3 to 5 are the same in NO. Therefore, when the situation data of items 1 and 2 are input, it is possible to more efficiently narrow down the action plans to be recommended compared to when the situation data of items 3 to 5 are input. Therefore, in order to preferentially accept the input of the situation data of items 1 and 2, the action plan selection support unit 332 may display panels P4 and P5, which are related panels, on the site terminal 40A and prompt the site supervisor D to input the situation data of items 1 and 2. With this configuration, it is possible to more efficiently narrow down the action plans to be recommended.

[0107] The action plan selection support unit 332 may update the action plan selection table based on the event history data. As an example, the action plan selection support unit 332 analyzes the event history data to identify situation data that is frequently acquired when an event is completed by executing a specific action plan. Then, the action plan selection support unit 332 may add acquisition of the identified situation data to the selection conditions of the specific action plan in the action plan selection table. For example, assume that the number of times an event is completed by executing action plan A is 10 times, and the number of times YES is input to item number 1 "Is the fallen position of the restricted material within the restricted area?" is 8 times out of 10 times. Since 8 times of YES is more than 2 times of NO, the action plan selection support unit 332 sets YES, which is more frequent, to item number 1 "Is the fallen position of the restricted material within the restricted area?" as the selection condition of action plan A. With this configuration, it is possible to select a more appropriate action plan for an event that occurs next time or later.

[0108] (Skills Management Department 333) The skill management unit 333 judges the skills of the site supervisor D. Then, the skill management unit 333 controls processing according to the skills of the site supervisor D. Note that the skills are quantified and the level of the skills is judged.

[0109] As an example, the skill management unit 333 may notify the center manager M who manages the site supervisor D when the skill of the site supervisor D is lower than a threshold (hereinafter also referred to as the skill judgment threshold). For example, the skill management unit 333 transmits information indicating that an event has occurred at a construction site S1 on a highway managed by the site supervisor D with low skills to the center terminal 40B. The center manager M can use this notification as a trigger to support the site supervisor. In this way, with this configuration, the site supervisor D with low skills can receive support from the center manager M.

[0110] As another example, when the skill of the site supervisor D is lower than the skill judgment threshold, the skill management unit 333 may present the display screen presented to the site supervisor D to the center manager M. For example, the skill management unit 333 may transmit screen capture data of the site terminal 40A to the center terminal 40B to share the screen between the site terminal 40A and the center terminal 40B. With this configuration, the center manager M can support the site supervisor D while viewing the same screen as the site supervisor D views.

[0111] As another example, the skill management unit 333 may transfer the authority to select an action plan to be executed from the site supervisor D to the center manager M when the skill of the site supervisor D is lower than the skill judgment threshold. In this case, the action plan selected by the center manager M will be executed instead of the site supervisor D. With this configuration, it is possible to prevent the wrong selection of an action plan due to a lack of skill.

[0112] The skill management unit 333 may determine the skill of the site supervisor D based on the event history data. That is, the skill management unit 333 may quantify the skill of the site supervisor D based on the event history data, and compare the quantified skill with a skill determination threshold to determine the level of the skill of the site supervisor D. With this configuration, it becomes possible to appropriately determine the skill of the site supervisor D based on the past performance of the site supervisor D.

[0113] As an example, the skill management unit 333 may determine the skill of the site supervisor D based on the number of times the site supervisor D has completed an event in the past without notifying the center manager M. The site supervisor D completing an event refers to the completion of the event due to the execution of an action plan selected by the site supervisor D. For example, the skill management unit 333 may determine that the more times the site supervisor D has completed an event in the past without notifying the center manager M, the higher the skill, and vice versa. With this configuration, it becomes possible to evaluate the site supervisor D's skill based on his or her track record of completing an event on his or her own.

[0114] As another example, the skill management unit 333 may determine the skill of the site supervisor D for each event based on the number of times the site supervisor D has completed the event in the past. For example, the skill management unit 333 may determine that the more times the site supervisor D has completed an event in the past, the higher the skill for that event is, and vice versa. With this configuration, it is possible to evaluate the skill for each event based on the difference in experience points for each event.

[0115] As another example, the skill management unit 333 may determine the skill of the site supervisor D for each action plan based on the number of times the site supervisor D has selected the action plan in the past. For example, the skill management unit 333 may determine that the more times the site supervisor D has selected an action plan in the past, the higher the skill of the site supervisor D for that action plan, and vice versa. With this configuration, it is possible to evaluate the skill of each action plan based on the experience of selecting the action plan.

[0116] In addition, the skill management unit 333 may determine the skill of the site supervisor D based on the real-time behavior of the site supervisor D while an event is occurring. That is, the skill management unit 333 may quantify the skill of the site supervisor D based on the real-time behavior of the site supervisor D while an event is occurring, and compare the quantified skill with a skill determination threshold to determine the level of the skill of the site supervisor D. With this configuration, it becomes possible to appropriately determine the skill even in a situation where an unprecedented accident occurs and the past performance of the site supervisor D is not valid.

[0117] As an example, the skill management unit 333 may determine the skill of the site supervisor D based on whether or not the site supervisor D selected a recommended action plan. For example, the skill management unit 333 may determine that the site supervisor D has high skills when the site supervisor D selected an action plan recommended by the action plan selection support unit 332, and may determine that the site supervisor D has low skills when the site supervisor D selected an action plan that is not recommended. With this configuration, it becomes possible to determine the skill of the site supervisor D in real time.

[0118] As another example, the skill management unit 333 may determine the skill of the site supervisor D based on the time it takes for the site supervisor D to input the situation data. For example, the skill management unit 333 may determine that the skill of the site supervisor D is high if the time it takes for the site supervisor D to input the situation data after the panel related to the situation data to be input is displayed is less than the average confirmation time, and may determine that the skill of the site supervisor D is low if not. With this configuration, it is possible to determine the skill of the site supervisor D in real time.

[0119] As another example, the skill management unit 333 may determine the skill of the site supervisor D based on the content of the operation by the site supervisor D input to the site terminal 40A. For example, the skill management unit 333 may determine that the skill of the site supervisor D is low when unnecessary operations are repeated, such as excessive mouse clicks or mouse movements, or excessive scrolling of the panel P. With this configuration, it becomes possible to determine the skill of the site supervisor D in real time.

[0120] The skill judgment threshold is stored in the storage unit 32. The skill judgment threshold may be calculated / updated based on event history data of multiple site supervisors D. For example, the skill management unit 333 may use an average value of the number of times that multiple site supervisors D have completed an event without notifying the center manager M in the past as the skill judgment threshold to be compared with the number of times.

[0121] <3. Operation processing> 12 and 13 are flowcharts showing an example of the flow of processing executed by the system 1 according to this embodiment.

[0122] (Step S101) First, the control device 30 collects situation data related to the event that has occurred. For example, the regulating material O detects that the vehicle has collided with the regulating material O and rolled over using an acceleration sensor or the like mounted on the regulating material O, and transmits the detection result to the control device 30 by email or the like. Although not specifically mentioned below, the control device 30 is assumed to be collecting situation data at all times even during the execution of each step described below.

[0123] (Step S102) Next, the control device 30 judges the occurrence of an event. For example, the control device 30 refers to the event judgment table shown in Table 1 and judges that an event (the falling of the restricting material O) has occurred based on the email received in step S101. The control device 30, which judges the occurrence of an event, generates an event container. For example, the control device 30 generates an event container that stores information indicating the event name "restricting material falling", action plans A, B, and C that are related action plans of "restricting material falling", and an action plan selection table corresponding to "restricting material falling" shown in Table 2. The situation data acquired from the occurrence of the event to its completion (i.e., steps S101 to S116) is added to the event container as needed. In addition, the control device 30, which judges the occurrence of an event, may generate a display screen that displays information indicating the occurrence of the event and display it on the site terminal 40A. This allows the site manager D to be notified that the event has occurred.

[0124] (Step S103) Next, the control device 30 transmits a screen capture start instruction to the on-site terminal 40A. Upon receiving the screen capture start instruction, the on-site terminal 40A starts capturing the left screen block B1 and the right screen block B2 being displayed, and uploads the screen capture data to the control device 30 as appropriate.

[0125] (Step S104) Next, the control device 30 calculates the matching rate between the acquired situation data and the situation data as the selection condition defined in the action plan selection table for each action plan. For example, if the situation data of items 1 to 2 in the action plan selection table shown in Table 2 is not input and the situation data of items 3 to 5 is NO, the control device 30 calculates the matching rates of action plans A to C as 60%, 60%, and 20%, respectively. Then, the action plan selection support unit 332 stores the calculated matching rates in the event container.

[0126] If the situation data is updated (step S111: YES) or the event is not completed (step S115: NO), step S104 is executed again. At that time, if situation data indicating that the falling position of the restricting material O is within the restricted area (item number 1: YES) is acquired, the control device 30 calculates the matching rates of the action plans A to C as 80%, 60%, and 20%, respectively. Furthermore, if situation data indicating that the remaining battery level of the robot 20 is 50% or more (item number 2: YES) is acquired, the control device 30 calculates the matching rates of the action plans A to C as 100%, 60%, and 20%, respectively.

[0127] (Step S105) Next, the control device 30 judges whether the skill of the site supervisor D is equal to or greater than the skill judgment threshold. As an example, the control device 30 may judge that the skill of the site supervisor D is equal to or less than the skill judgment threshold when the number of times that the site supervisor D has completed an event without notifying the center manager M in the past is lower than the average number of times for a plurality of site supervisors D. As another example, the control device 30 may calculate the time taken from step S107 to step S111 from the screen capture data of the on-site terminal 40A. Then, the control device 30 may judge that the skill of the site supervisor D is equal to or less than the skill judgment threshold when the time taken from step S107 to step S111 is longer than the average confirmation time defined in the action plan selection table.

[0128] If it is determined that the skill of the site supervisor D is equal to or less than the skill judgment threshold (step S105: YES), the process proceeds to step S210a. On the other hand, if it is determined that the skill of the site supervisor D exceeds the skill judgment threshold (step S105: NO), the process proceeds to step S106.

[0129] (Step S106) The control device 30 extracts related panels related to unentered situation data items among one or more situation data items that are conditions for selecting an action plan defined in the action plan selection table. When there are multiple unentered situation data items, the control device 30 extracts them in order starting from the situation data items with the smallest item numbers. Regarding the example shown in Table 2, for example, when only situation data item number 1 has been entered, the control device 30 extracts panel P5, which is a related panel related to situation data item number 2.

[0130] (Step S107) Next, the control device 30 causes the on-site terminal 40A to display the relevant panel extracted in step S106. For example, the control device 30 transmits identification information of the panel P5, which is the extracted relevant panel, to the on-site terminal 40A. Then, the on-site terminal 40A displays the panel P5 at the top of the left screen block B1 based on the identification information.

[0131] (Step S108) Next, the control device 30 transmits the coincidence rate of the situation data for each action plan calculated in step S104 to the on-site terminal 40A. The on-site terminal 40A rearranges the action plans in descending order of the coincidence rate of the situation data on the panel P2 displayed on the right screen block B2, and presents them to the site supervisor D.

[0132] (Step S109) Next, the control device 30 determines whether the matching rate of the situation data for each action plan calculated in step S104 is equal to or greater than the recommended threshold. If there is an action plan for which a matching rate equal to or greater than the recommended threshold is calculated (step S109: YES), the process proceeds to step S110. On the other hand, if there is no action plan for which a matching rate equal to or greater than the recommended threshold is calculated (step S109: NO), the process proceeds to step S111.

[0133] (Step S110) The control device 30 controls the on-site terminal 40A to display the recommended action plan. In particular, the control device 30 controls the on-site terminal 40A to recommend an action plan of which the matching rate of the situation data exceeds the recommended threshold, among the action plans A to C. Furthermore, the control device 30 may control the on-site terminal 40A to recommend an action plan of which the matching rate of the situation data is the highest, among the action plans A to C. For example, the terminal device 40 additionally displays a message P22 on the panel P2, recommending the action plan of which the matching rate is equal to or higher than the recommended threshold and has been calculated to be the highest.

[0134] (Step S111) As shown in FIG. 13, the control device 30 then judges whether the situation data has been updated. In detail, the control device 30 judges whether new situation data has been inputted on the panel P1, or whether new situation data has been received from the sensor 10 or the like. Alternatively, the site supervisor D may input to the site terminal 40A that the situation data will not be updated, and the control device 30 may judge whether the situation data has been updated based on the presence or absence of such input. If it is judged that the situation data has been updated (step S111: YES), the process returns to step S104, and the coincidence rate of the situation data is calculated again. If it is judged that the situation data has not been updated (step S111: NO), the process proceeds to step S112.

[0135] (Step S112) Next, the on-site terminal 40A accepts a user operation to select an action plan to be executed. For example, the site supervisor D inputs an operation to select the action plan recommended in step S110 as the action plan to be executed into the on-site terminal 40A. As a result, the action plan is executed. As an example, the on-site terminal 40A transmits information to the nearest robot 20 or worker W to the fallen restricting material O to instruct the robot 20 or worker W to recover the fallen restricting material O.

[0136] (Step S113) Next, the control device 30 stores the status data when the action plan is executed. For example, the control device 30 stores the status data of the items defined in the action plan selection table in the event container. In detail, the control device 30 stores the status data of items 1 to 3 input by the site supervisor D and the status data of items 4 to 5 automatically acquired in the event container. In addition, the control device 30 stores information indicating the panel P displayed by the field terminal 40A in the left screen block B1, information indicating the executed action plan, and screen capture data of the field terminal 40A in the event container. Furthermore, the control device 30 stores the time taken from step S107 to step S111 in the event container.

[0137] (Step S114) Next, the control device 30 determines whether the recommended action plan has been executed. If it is determined that the recommended action plan has been executed (step S114: YES), the process proceeds to step S115. If it is determined that the non-recommended action plan has been executed (step S114: NO), the process proceeds to step S210b.

[0138] (Step S115) Next, the control device 30 judges whether the event is completed. For example, the control device 30 judges whether the check box P11 or P12 of the "Is the event completed?" check box is checked in the panel P1 displayed in the right screen block B2 of the terminal device 40. If it is judged that the event is completed (step S115: YES), the process proceeds to step S116. On the other hand, if it is judged that the event is not completed (step S115: NO), the process proceeds to step S104.

[0139] (Step S116) The control device 30 transmits a screen capture end instruction to the on-site terminal 40A. Upon receiving the screen capture end instruction, the on-site terminal 40A ends the capture of the screen and the upload of the screen capture data.

[0140] (Step S117) Next, the control device 30 stores the temporarily stored event container in long-term storage as event history data. Note that, for any item of the situation data defined in the action plan selection table that has not been entered, the control device 30 may acquire the situation data after the event is completed and add it to the event history data.

[0141] (Step S118) Next, the control device 30 updates the action plan selection table based on the event history data. For example, assume that the number of times the event has been completed by executing action plan A is 10, and the number of times YES was entered for item number 1 "Is the position of the fallen restriction material within the restricted area?" was 8 out of 10. In this case, the control device 30 sets YES, which is more frequent, to item number 1 "Is the position of the fallen restriction material within the restricted area?" as the selection condition for action plan A. Then, the process ends.

[0142] (Step S210a) The control device 30 notifies the center manager M. For example, the control device 30 transmits information indicating that the skills of the site supervisor D are low and the screen capture data of the field terminal 40A to the center terminal 40B. Then, upon receiving such a notification, the center terminal 40B displays a pop-up message such as "Site supervisor D lacks skills" or sounds an alarm. When the center manager M checks such a message and hides the pop-up, the center terminal 40B displays the screen capture data of the field terminal 40A and urges the center manager M to monitor the site supervisor D.

[0143] After that, the process proceeds to step S106. Note that between step S210a and step S106, step S212, which will be described later, may be executed automatically or based on the determination in step S211, which will be described later.

[0144] (Step S210b) The control device 30 notifies the center manager M. For example, the control device 30 transmits information indicating that a non-recommended action plan has been executed and screen capture data of the field terminal 40A to the center terminal 40B. Then, upon receiving such a notification, the center terminal 40B displays a pop-up message such as "An action plan other than the recommended one has been selected" or sounds an alarm. When the center manager M checks such a message and hides the pop-up, the center terminal 40B displays the screen capture data of the field terminal 40A and urges the center manager M to monitor the site supervisor D.

[0145] (Step S211) Next, the control device 30 judges whether the judgment of the site supervisor D is appropriate or not. For example, the center terminal 40B displays a pop-up message such as "Please confirm whether there is any problem with the judgment of the site supervisor D" and accepts input of the confirmation result by the center manager M. The center manager M can check whether the judgment of the site supervisor D is appropriate or not by looking at the shared screen of the site terminal 40A. Then, the control device 30 judges whether the judgment of the site supervisor D is appropriate or not based on the input to the center terminal 40B.

[0146] If it is determined that the judgment of the site supervisor D is appropriate (step S211: YES), the process proceeds to step S115. On the other hand, if it is determined that the judgment of the site supervisor D is inappropriate (step S211: NO), the process proceeds to step S212.

[0147] (Step S212) The control device 30 transfers the authority to select an action plan to be executed from the site supervisor D to the center manager M. For example, the authority to operate the field terminal 40A is transferred to the center manager M, and the action plan is selected on the field terminal 40A by the operation by the center manager M via the center terminal 40B. On the other hand, the screen of the field terminal 40A is locked and cannot be operated by the site supervisor D, and the screen transition is based on the operation by the center manager M via the center terminal 40B. After that, the process proceeds to step S104.

[0148] <4. Hardware configuration example> Next, the hardware configuration of the information processing device according to this embodiment will be described with reference to Fig. 14. Fig. 14 is a block diagram showing an example of the hardware configuration of the information processing device according to this embodiment. Note that the information processing device 1000 shown in Fig. 14 can realize, for example, the terminal device 40 shown in Fig. 2 or the control device 30 shown in Fig. 11. Information processing by the control device 30 or the terminal device 40 according to this embodiment is realized by cooperation between software and hardware described below.

[0149] As shown in FIG. 14, the information processing device 1000 may include a processor 1001, a ROM (Read Only Memory) 1002, a RAM (Random Access Memory) 1003, an internal bus 1004, an input / output interface 1010, a display device 1011, an input device 1012, an audio output unit 1013, a storage device 1014, a drive 1015, a network interface 1016, and an external interface 1017.

[0150] The processor 1001 functions as an arithmetic processing device and a control device, and controls the overall operation of the information processing device 1000 in accordance with various programs. The processor 1001 cooperates with a ROM 1002, a RAM 1003, and software described below to realize, for example, the functions of the control unit 33 or the control unit 45.

[0151] The ROM 1002 stores programs and computation parameters used by the processor 1001. The RAM 1003 temporarily stores programs used during execution by the processor 1001, as well as parameters that change as appropriate during the execution of the programs.

[0152] The processor 1001, ROM 1002, and RAM 1003 are connected to each other by an internal bus 1004, and are further connected via an input / output interface 1010 to a display device 1011, an input device 1012, an audio output unit 1013, a storage device 1014, a drive 1015, a network interface 1016, and an external interface 1017, which will be described later.

[0153] The display device 1011 is, for example, a display device such as a CRT display device, a liquid crystal display (LCD), or an OLED device, and converts video data into video and outputs the video. The display device 1011 may form, for example, the display unit 42. The input device 1012 may be composed of a mouse, a keyboard, a touch panel, a button, a microphone, a sensor, a switch, a control circuit, and the like. The input device 1012 may form, for example, the input unit 43. The audio output unit 1013 is an audio output device such as a speaker and headphones, and converts audio data and the like into audio and outputs the audio.

[0154] The storage device 1014 is a device for storing data. The storage device 1014 may include a storage medium, a recording device for recording data in the storage medium, a reading device for reading data from the storage medium, and a deleting device for deleting data recorded in the storage medium. The storage device 1014 is configured, for example, by a hard disk drive (HDD) or a solid state drive (SSD), or a memory having an equivalent function. The storage device 1014 drives the storage and stores programs or various data executed by the processor 1001. The storage device 1014 may form, for example, the storage unit 32 or the storage unit 44.

[0155] The drive 1015 is a reader / writer for a storage medium, and is built into or externally attached to the information processing device 1000. The drive 1015 reads out information stored in a removable storage medium, such as an attached magnetic disk, optical disk, magneto-optical disk, or semiconductor memory, and outputs the information to the RAM 1003. The drive 1015 is also capable of writing information to the removable storage medium.

[0156] The network interface 1016 is, for example, a communication interface configured with a device for connecting to a communication network such as the Internet. The network interface 1016 may be a communication device compatible with a wireless LAN (Local Area Network) and / or a wired LAN. The network interface 1016 may form, for example, the communication unit 31 or the communication unit 41.

[0157] The external interface 1017 is a connection interface composed of connection ports for connecting external devices, such as a USB (Universal Serial Bus) port, an IEEE 1394 port, a SCSI (Small Computer System Interface) port, an RS-232C port, or an optical audio terminal.

[0158] An example of a hardware configuration capable of implementing the functions of the information processing device 1000 according to this embodiment has been described above. Each of the above components may be implemented using general-purpose components, or may be implemented using hardware specialized for the function of each component. Therefore, the hardware configuration to be used can be changed as appropriate depending on the technical level at the time of implementing this embodiment.

[0159] <5. Supplementary Information> Although the preferred embodiment of the present invention has been described in detail above with reference to the accompanying drawings, the present invention is not limited to such an example. It is clear that a person having ordinary knowledge in the technical field to which the present invention pertains can conceive of various modified or altered examples within the scope of the technical ideas described in the claims, and it is understood that these also naturally belong to the technical scope of the present invention.

[0160] For example, in the above embodiment, an example has been described in which the present invention is applied to a highway construction site S1 where humans and robots work together and the work is managed by humans, but the present invention is not limited to such an example. The present invention can be applied to any work site managed by humans, such as a building construction site or facility security. Furthermore, the present invention is not limited to work sites where humans and robots work together, but may be applied to work sites where only humans or only robots work. Furthermore, the work site is not limited to outdoors, but may be indoors.

[0161] For example, in the above embodiment, an example has been described in which the situation data is acquired by the sensor 10 or input by the site supervisor D, but the present invention is not limited to such an example. The situation data may be estimated by machine learning based on features extracted from the screen capture data.

[0162] For example, in the above embodiment, an example has been described in which the display of the field terminal 40A is controlled by the control device 30, but the present invention is not limited to such an example. At least a part of the display by the field terminal 40A may be controlled by the field terminal 40A itself. As an example, the panel P displayed on the field terminal 40A may be generated by the field terminal 40A itself. In that case, the control device 30 transmits information used for generating a screen by the field terminal 40A (for example, various situation data and information indicating a recommended action plan) to the field terminal 40A, and the field terminal 40A may generate and display the panel P based on the received information. The same applies to the display of the center terminal 40B.

[0163] For example, in the above embodiment, an example has been described in which the panels P4 to P7 are displayed on the left screen block B1 of one monitor, and the panels P1 to P3 and P8 are displayed on the right screen block B2 of the other monitor, but the present invention is not limited to such an example. These panels P may be displayed separately on multiple monitors.

[0164] The series of processes by each device described in this specification may be realized using any of software, hardware, and a combination of software and hardware. The programs constituting the software are stored in advance, for example, in a recording medium (more specifically, a non-transient storage medium readable by a computer) provided inside or outside each device. Then, each program is read into a random access memory (RAM) when executed by a computer that controls each device described in this specification, and executed by a processing circuit such as a central processing unit (CPU). The recording medium is, for example, a magnetic disk, an optical disk, a magneto-optical disk, a flash memory, etc. The computer program may be distributed, for example, via a network without using a recording medium. The computer may be an application specific integrated circuit such as an application specific integrated circuit (ASIC), a general-purpose processor that executes a function by reading a software program, or a computer on a server used in cloud computing. The series of processes by each device described in this specification may be processed in a centralized manner by a single computer, or may be processed in a distributed manner by multiple computers. Furthermore, in each of the above embodiments, two or more communication means present in one device may be physically realized by one medium.

[0165] In addition, the processes described in this specification using flowcharts or sequence diagrams do not necessarily have to be performed in the order shown in the drawings. Some processing steps may be performed in parallel. In addition, additional processing steps may be employed, and some processing steps may be omitted. [Explanation of symbols]

[0166] 1 System 10 Sensors 20. Robot 30 Control device 31 Communications Department 32 Storage section 33 Control Unit 331 Event Management Department 332 Action Plan Selection Support Department 333 Skills Management Department 40 Terminal device (40A: field terminal, 40B: center terminal) 41 Communications Department 42 Display section 43 Input section 44 Memory section 45 Control section

Claims

1. an acquisition unit that repeatedly acquires situation data indicating a situation of a work site in which a user is involved; a storage unit that stores, for a plurality of events that may occur at the work site, an action plan selection table that defines, in association with each other, one or more action plans that may be executed when an event occurs at the work site and one or more pieces of situation data that are conditions for selecting the action plan; a control unit that controls a process of recommending one or more of the action plans to be selected by the user based on the situation data acquired by the acquisition unit and the table related to the events that have occurred at the work site; and An information processing device comprising:

2. the control unit selects the recommended action plan based on a matching rate between the situation data acquired by the acquisition unit and the situation data defined in the action plan selection table. The information processing device according to claim 1 .

3. the control unit controls a process of generating a display screen for accepting input of the situation data and presenting the display screen to the user; The acquisition unit acquires the situation data input on the display screen. The information processing device according to claim 1 .

4. The control unit controls a process of presenting to the user information to be referred to by the user when inputting the situation data. The information processing device according to claim 3 .

5. the control unit controls a process of preferentially accepting input of situation data for an item in which the situation data associated with each of the plurality of action plans is different among the plurality of situation data items defined in the action plan selection table. The information processing device according to claim 3 .

6. The control unit determines a skill of the user, and if the skill of the user is lower than a threshold, notifies a center administrator who manages the user. The information processing device according to claim 3 .

7. the control unit controls a process of presenting the display screen presented to the user to the center manager when the skill of the user is lower than a threshold value. The information processing device according to claim 6.

8. the control unit, when the skill of the user is lower than a threshold, transfers an authority to select the action plan to be executed from the user to the center administrator. The information processing device according to claim 6.

9. the storage unit stores history data including the event that has occurred and the situation data acquired from the occurrence of the event to its completion; The control unit determines the skill of the user based on the history data. The information processing device according to claim 6.

10. The control unit determines the skill of the user based on the number of times the user has completed the event without notifying the center administrator in the past. The information processing device according to claim 9.

11. The control unit determines the skill of the user for each event based on the number of times the user has completed the event in the past. The information processing device according to claim 9.

12. The control unit determines the skill of the user for each action plan based on the number of times the user has selected the action plan in the past. The information processing device according to claim 9.

13. the storage unit stores history data including the event that has occurred and the situation data acquired from the occurrence of the event to its completion; The control unit updates the action plan selection table based on the history data. The information processing device according to claim 1 .

14. The acquisition unit acquires the situation data acquired by a sensor arranged at the work site. The information processing device according to claim 1 .

15. The situation data includes information regarding a robot working at the work site. The information processing device according to claim 1 .

16. The situation data includes information about workers working at the work site. The information processing device according to claim 1 .

17. The situation data includes information regarding an item that is the subject of work at the work site. The information processing device according to claim 1 .

18. 1. A computer-implemented information processing method, comprising: Repeatedly acquiring situation data indicative of a situation of a work site involving a user; storing an action plan selection table for a plurality of events that may occur at the work site, the action plan selection table defining, in association with each other, one or more action plans that may be executed when an event occurs at the work site and one or more pieces of situation data that are conditions for selecting the action plan; controlling a process for recommending one or more of the action plans to be selected by the user based on the acquired situation data and the table of events occurring at the work site; An information processing method comprising:

19. Computer, an acquisition unit that repeatedly acquires situation data indicating a situation of a work site in which a user is involved; a storage unit that stores, for a plurality of events that may occur at the work site, an action plan selection table that defines, in association with each other, one or more action plans that may be executed when an event occurs at the work site and one or more pieces of situation data that are conditions for selecting the action plan; a control unit that controls a process of recommending one or more of the action plans to be selected by the user based on the situation data acquired by the acquisition unit and the table related to the events that have occurred at the work site; and A program to function as a

Images