Action-based support system and program storage medium

Abstract

Disclosed is an action-based support system (ABSS) 11 for providing simple and accurate support to a person who acts. The ABSS 11 comprises a natural language system (NLS) 13 and a prediction action system (PAS) 15. For example, in an embodiment for supporting an operator 3 of a plant 1, the NLS 13 performs interactive communication with the operator 3 by using a natural language. The PAS 15 generates a future prediction 53 of the situation of the plant 1 and a recommended operation 57 for the plant 1. In accordance with the interactive communication with the operator 3, the NLS 13 provides the operator 3 with an explanatory message expressed in a natural language and based on the future prediction 53 or the recommended operation 57.

Description

Behavior-based support system and program storage medium

[0001] The present invention relates to an action-based assistance system that assists a person in deciding on an action depending on the environmental situation, or automatically decides on or executes an action on behalf of a person, for example, a system that provides assistance using artificial intelligence.

[0002] An example of behavior-based assistance is assistance for plant operators in operating a chemical plant or the like.

[0003] As artificial intelligence automatically assists in plant operations in more situations, the burden on operators will be reduced.

[0004] As a related prior art, there is a plant operation support system described in Patent Document 1.

[0005] This system includes a real-time inference system that inputs plant data generated in the plant system and performs inference, and an AI training system that inputs training data created using the plant data and trains the real-time inference system. The AI ​​training system has a know-how evaluation unit that evaluates the reliability of the inference results and selectively requests operators to participate in the creation of training data depending on the reliability evaluation result.

[0006] JP 2023-111624 A

[0007] In a plant where major operational failures are unacceptable, a behavior-based support system is expected to provide operators with the most appropriate operational recommendations possible. Furthermore, communication between the behavior-based support system and operators should be as easy to understand as possible for operators, just like a conversation between people.

[0008] The above also applies to behavioral support in various fields other than plant operation, such as driving assistance or automated driving for automobile drivers, and behavioral and thinking support for factory workers and office workers.

[0009] An object of the present invention is to provide an activity-based assistance system that provides more accurate assistance to a person.

[0010] Another object of the present invention is to make communication between an activity-based assistance system and a person easier for the person to understand.

[0011] Other objects of the present invention will become apparent in the following description.

[0012] According to one embodiment, a behavior-based support system for supporting a person in acting in accordance with an environmental situation includes a natural language system that uses natural language to engage in interactive communication with the person, and a predictive behavior system that inputs environmental data indicating the environmental situation, uses the input environmental data to generate a future prediction indicating the future situation of the environment, and generates recommended behavior using the generated future prediction.

[0013] The natural language system inputs the future prediction or the recommended action from the predictive action system, interprets or analyzes the interactive communication with the person, and, depending on the results of interpreting or analyzing the interactive communication, generates a message expressed in the natural language based on the input future prediction or the recommended action, and provides the generated message to the person.

[0014] According to one embodiment, the behavior-based support system includes a storage device storing a computer program and a processor that mechanically reads and executes the computer program. By executing the computer program, the processor engages in interactive communication with the person using natural language, inputs environmental data indicating the state of the environment, generates a future prediction indicating a future state of the environment using the input environmental data, generates a recommended action using the generated future prediction, interprets or analyzes the interactive communication with the person, generates a message expressed in the natural language based on the future prediction or the recommended action according to a result of interpreting or analyzing the interactive communication, and provides the generated message to the person.

[0015] 1 illustrates an example of the hardware configuration of a behavior-based operation support system according to one embodiment; 2 illustrates the configuration of an overall plant system to which the behavior-based operation support system according to this embodiment is applied, and the functions and operations of the plant system that operate during plant operation; 3 illustrates the configuration of the plant system, and the functions and operations of the plant system that operate when reviewing past operations; 4 illustrates the configuration of the plant system, and the functions and operations of the plant system that operate when registering operational know-how; and 5 illustrates an example of the configuration of a world model incorporated into a predictive behavior system within the behavior-based operation support system according to this embodiment.

[0016] Hereinafter, embodiments of the present invention will be described with reference to the drawings. Note that the embodiments described below do not limit the scope of the invention as claimed, and not all of the elements and combinations thereof described in the embodiments are necessarily essential to the solution of the invention.

[0017] In the drawings explaining the embodiments, parts having the same functions are given the same reference numerals, and repeated explanations thereof will be omitted.

[0018] FIG. 1 shows the physical configuration of a computer system to which an activity-based support system according to one embodiment is applied.

[0019] As shown in FIG. 1, a computer system 200 includes a plurality of (or one) physical computers 201 connected to a network 240 .

[0020] The network 240 is one or more communication networks, and may include, for example, at least one of a Fibre Channel (FC) network and an Internet Protocol (IP) network. The network 240 may exist outside the computer system 200.

[0021] Each physical computer 201 is, for example, a general-purpose computer, and has physical computer resources 350. The physical computer resources 350 include a communication interface unit 251, a storage unit 252, and a processor unit 253 connected to these.

[0022] The computer system 200 may be, for example, a cloud computing system that provides XaaS (X as a Service). Note that "XaaS" generally refers to a service that makes available some resources (e.g., hardware, lines, software execution environment, application programs, development environments, etc.) necessary for system construction or operation via a network such as the Internet. The letter (or word) used as the "X" in XaaS varies depending on the type (service model) of XaaS. Examples of XaaS include PaaS (Platform as a Service), SaaS (Software as a Service), IaaS (Infrastructure as a Service), and HaaS (Hardware as a Service).

[0023] The behavior-based support system according to one embodiment can be realized by the processor unit 253 of one or more physical computers 201 of the computer system 200 executing a computer program stored in the storage unit 252 .

[0024] As an example, the following describes an example of an activity-based support system according to an embodiment, which is applied to support the operation of a plant such as a chemical plant. Figures 2, 3, and 4 show an example of the configuration and functions of the activity-based support system according to this embodiment and an overall plant system including its related systems.

[0025] 2, 3, and 4, the plant system 300 includes a plant 1, an operator 3, and a distributed control system (hereinafter referred to as "DCS") 5. The plant system 300 further includes one or more user interface / user experience systems (hereinafter referred to as "UIS") 7 and a data management system (hereinafter referred to as "DMS") 9. The plant system 300 also includes an action-based support system (hereinafter referred to as "ABSS") 11 according to one embodiment.

[0026] A single plant 1 may have a plurality of each of the other types of components 3, 5, 7, 9, and 11. However, for the sake of convenience, the following description focuses on one component of each component type and describes its operation and function. From this description, a person skilled in the art should be able to easily understand the case where a plurality of components of each component type exist.

[0027] FIG. 2 shows the operation and function of the plant system 300 when the plant 1 is in operation.

[0028] 2, when the operator 3 inputs an operation instruction 23 to the UIS 7 (step 21), the operation instruction 23 is received by the DCS 5. The DCS 5 controls the plant 1 in accordance with the operation instruction 23 (step 25). The plant 1 operates in accordance with the control 27 (step 29).

[0029] Numerous sensors are installed at numerous locations in the plant 1 to sense various conditions at each location (operating conditions such as temperature, pressure, and flow rate, as well as the quality of materials and products). These sensors sense the conditions at each location and output sensor data indicating the location and the condition value (step 31). The DCS 5 receives the sensor data (step 35). The DCS 5 also monitors the sensor data 33 received from the plant 1 to check for abnormalities. If an abnormality is detected, the DCS 5 notifies the operator 3 via the UIS 7 (step 37). The DCS 5 outputs the received operation instructions, received sensor data, and detected abnormality data in real time. Hereinafter, the operation instructions, sensor data, and abnormality data output from the DCS 5 are collectively referred to as "plant data." The DMS 9 receives plant data 39 from the DCS 5 and stores the received plant data 39 in the plant data log base 41.

[0030] The DMS 9 also has an operation report log base 43 and an operation know-how base 45. The operation report log base 43 stores operation reports (for example, daily operation reports, handover messages, etc.) created by the operators 3 as operation report logs. In addition to the above-mentioned operation report logs, the operation report log base 43 also stores various documents related to the operation of the plant 1 (for example, operation standards, operation procedures, operation rules, etc.). The operation know-how base 45 stores various types of operation know-how created by the operators 3.

[0031] During operation of the plant 1, not only are the basic processes as described above performed, but operation assistance is also provided by the ABSS 11. The configuration and functions of the ABSS 11 will be described below.

[0032] 2 , the ABSS 11 can communicate with other components of the plant system 300, such as the DCS 5, the DMS 9, and the UIS 7, and provides various support services to the operator 3 by using the communications with these other components. The ABSS 11 has a Natural Language System (hereinafter referred to as "NLS") 13 and a Predictive Action System (hereinafter referred to as "PAS") 15.

[0033] In the following description, the two systems, NLS 13 and PAS 15, are assumed to be included as subsystems in the ABSS 11, but this is merely a convenient example. It may also be understood that not only the NLS 13 and PAS 15, but also all or part of the DCS 5, DMS 9, and UIS 7 are subsystems of the ABSS 11.

[0034] The NLS 13 of the ABSS 11 has the function of interpreting, analyzing, and generating messages using natural language that is commonly used and understandable by humans (operators 3). Here, natural language may include not only characters, sentences, or text that are commonly used by humans, but also other methods of expression that are commonly used and understandable by humans, such as diagrams, tables, and pictures.

[0035] The NLS 13 can input a message in a natural language, interpret or analyze it, and, depending on the results of the interpretation or analysis, generate and output a new message expressed in an artificial language (e.g., a language used by a machine such as the DCS5, DMS9, and / or PAS15) and / or a new message expressed in a natural language. The NLS 13 can also input a message in an artificial language, interpret or analyze it, and, depending on the results of the interpretation or analysis, create and output a new message in a natural language. To achieve such natural language processing functionality, the NLS 13 can have a deep neural network of a type called a Large Language Model (hereinafter referred to as "LLM"), such as a Generative Pre-trained Transformer (GPT).

[0036] The NLS 13 is capable of communicating with components of the plant system 300, such as the PAS 15, the DMS 9, and the UIS 7.

[0037] The PAS 15 of the ABSS 11 has the role of generating information related to the future operation of the plant 1 that is useful to the operator 3. The PAS 15 has a function of predicting the future behavior and state of the plant 1, and a function of generating an action recommended for the operator 3 to take (particularly, an operation command recommended to be given to the plant 1) based on the predicted future behavior and state. To realize this predictive action function, the PAS 15 can have, for example, a deep neural network of a type called a World Model (hereinafter referred to as "WM").

[0038] The PAS 15 is capable of communicating with components of the plant system 300, such as the NLS 13, the DCS 5, the DMS 9, and the UIS 7.

[0039] When the plant 1 is in operation, the ABSS 11 can perform the following processes.

[0040] 2, the PAS 15 of the ABSS 11 receives plant data output from the DCS 5 in real time, and generates and outputs a future prediction 53 of the behavior and state of the plant 1 using the received plant data (step 51). The PAS 15 further generates and outputs a recommended operation (operational instruction recommended to be provided to the plant 1 in the future) 57 based on the generated future prediction 53.

[0041] The NLS 13 receives the future prediction 53 and the recommended operation 57 output from the PAS 15. Using the received future prediction 53 and the recommended operation 57, the NLS 13 creates explanatory messages, i.e., a future prediction message 61 and a recommended operation message 63, that express the future prediction 53 and the recommended operation 57 in natural language in a format that is easier for the operator 3 to understand or use (step 59). The NLS 13 then displays the created explanatory messages 61 and 63 on the UIS 7, respectively, to communicate them to the operator 3.

[0042] When creating each of the above messages 61 and 63, the NLS 13 can search for data related to each of the input future predictions 53 and recommended operations 57 (e.g., plant data logs, operation report logs, operating standards, operating rules, or operating know-how) from the plant data log base 41, the operation report log base 43, and / or the operation know-how base 45. The NLS 13 can then add the searched related data to the future predictions 53 or the recommended operations 57 or edit the data to create the explanatory future prediction message 61 and the recommended operation message 63. Furthermore, the NLS 13 communicates interactively with the operator 3 using natural language and interprets or analyzes the interactive communication. Then, depending on the results of the interpretation or analysis, the NLS 13 can determine what type of relevant data the operator 3 wants and / or what format (e.g., text, table, diagram, formula, or a combination thereof) is appropriate to express the messages 61, 63 in, and create the messages 61, 63 according to that determination.

[0043] The operator 3 can use the messages 61 and 63 as reference or assistance information for proper operation of the plant 1 (steps 65, 67).

[0044] Furthermore, the operator 3 can input any query for additional information in free text in natural language to the UIS 7 (step 71), and the query message 73 is received by the NLS 13 of the ABSS 11 via the UIS 7. The query message 73 may inquire about the result that would be caused if a specified operation were performed, such as "If an operation 'xx' is performed, what will the future prediction be?" Alternatively, the query message 73 may inquire about the operation that should be performed to achieve a specified result, such as "To make the production volume of monomethylamine 'xx', what temperature should 'yy' be set to?" The specific content of the query message 73 can be freely determined by the operator 3.

[0045] The NLS 13 of the ABSS 11 interprets or analyzes the received inquiry message 73, and thereby determines a processing method to be performed in order to respond to the inquiry from the operator 3 (step 75). As the processing method, there are several possible examples as follows:

[0046] [First Method] This method can be applied when a question is asked about the likely outcome of performing a specified operation, such as the above example, "If an operation 'xx' is performed, what will the future prediction be?" In this method, the NLS 13 issues an instruction 77 to the PAS 15 to change part of the plant data 39 used in the future prediction. Then, the PAS 15 uses the changed plant data to perform a tentative future prediction (step 79).

[0047] For example, only the parameter values ​​of the relevant operation instructions (e.g., set values ​​such as temperature or pressure at a specific location in the plant 1) in the actual plant data 39 input from the DCS 5 to the PAS 15 are changed to the parameter values ​​of the operation instructions specified in the query message 73. The PAS 15 uses the changed operation instructions combined with other data in the actual plant data 39 as provisional plant data to generate a provisional future forecast 81 indicating the inquired results (step 79). The NLS 13 receives the generated provisional future forecast 81 and uses the provisional future forecast 81 to generate a reply message 89 to the query message 73 (step 87). The operator 3 receives the reply message 89 via the UIS 7 and can use it as reference information.

[0048] [Second Method] This method can be applied when a question is asked about the operation to be performed to obtain a specified result, such as the above-mentioned example, "What temperature should be set to 'yy' to achieve 'xx' production of monomethylamine?" In this method, the NLS 13 analyzes at least a portion of various data (e.g., plant data logs, operation report logs, or operational know-how) managed in the databases 41, 43, and 45 of the DMS 9, and selects one or more operation instructions that are likely to affect the result specified in the query message 73 (i.e., have a high correlation with the specified result). Then, the NLS 13 issues an instruction 77 to the PAS 15 to perform a tentative future forecast while changing the parameter values ​​of the selected operation instructions to various values.

[0049] In accordance with this instruction 77, the PAS 15 changes only the parameter values ​​of the selected operation instructions in the actual plant data 39 input from the DCS 5 to various values, for example, by trial and error, and generates a tentative future forecast 81 for each changed parameter value (step 79). Furthermore, the PAS 15 generates a tentative recommended operation 85 for each changed parameter value using the tentative future forecast 81 (step 83).

[0050] The NLS 13 receives from the PAS 15 tentative future forecasts 81 and tentative recommended operations 85 corresponding to various changed parameter values. Then, the NLS 13 selects, from the various received tentative future forecasts 81, the one that is closest to the result specified in the query message 73, and selects a tentative recommended operation 85 corresponding to the selected tentative future forecast 81. The NLS 13 generates a reply message 89 to the query message 73 using the selected tentative future forecast 81 and tentative recommended operation 85 (step 87). The operator 3 receives the reply message 89 via the UIS 7 and can use it as reference information.

[0051] [Third Method] Like the second method, this method can also be applied when a query is made about the operation to be performed to obtain a specified result. This method is a modified combination of the first and second methods. The NLS 13 analyzes at least a portion of various data (e.g., plant data logs, operation report logs, or operational know-how) managed in the databases 41, 43, and 45 of the DMS 9 to identify one type of parameter value of one or more operational instructions that are effective for obtaining the result specified in the query message 73. The NLS 13 then issues an instruction 77 to the PAS 15 to change the corresponding portion of the plant data 39 to the parameter value of the identified operational instruction and perform a hypothetical future forecast using the changed plant data 39 as hypothetical plant data. The subsequent process is similar to that of the first method.

[0052] In any of the first to third methods described above, in step 87 of creating a reply message, the NLS 13 interactively communicates with the operator 3 and interprets or analyzes the communication, similar to step 59 described above. Then, depending on the results of the interpretation or analysis, the NLS 13 can read data related to the tentative future prediction 81 or the tentative recommended operation 85 from the database 41, 43, or 45 of the DMS 9, and generate an explanatory reply message 89 including the related data, so as to meet the requests of the operator 3 as well as possible.

[0053] FIG. 3 shows the operation and function of a plant system 300 for reviewing past plant operations and relearning the ABSS 11.

[0054] 3 , when an operator 3 uses the UIS 7 to create an operation report (e.g., a daily operation report) 103 (step 101), the operation report 103 is registered in the operation report log base 43 by the DMS 9. When the operator 3 inputs a request 107 for an operation log that meets certain conditions (e.g., time, location within the plant, type of log, etc.) into the UIS 7 (step 105), the NLS 13 of the ABSS 11 reads the plant data log and / or operation report log that meets the conditions from the plant data log base 41 and / or the operation report log base 43, interprets or analyzes the log data, and generates an operation log message 111 that expresses the results of the interpretation or analysis using free text in natural language, graphs, etc. (step 109). In this step 109, similar to step 59 shown in FIG. 2, the NLS 13 communicates interactively with the operator 3, interprets or analyzes the interactive communication, and, depending on the results of the interpretation or analysis (i.e., to meet the needs of the operator 3 as best as possible), selects and reads data related to the requested driving log from the database 41, 43, or 45 of the DMS 9, and generates an explanatory driving log message 111 including the related data.

[0055] The driving log message 111 is presented to the operator 3 via the UIS 7. The operator 3 can refer to the driving log message 111 to review and discuss past driving (step 113).

[0056] When the operator 3 writes, for example, the above-mentioned review or discussion of past driving (hereinafter referred to as driving review) 117 in free text and inputs it to the UIS 7, the NLS 13 of the ABSS 11 interprets or analyzes the free text input content 117 to generate learning data (e.g., teacher data for supervised learning of the PAS 15) 121 in a format that can be used for relearning the PAS 15 (step 119). At this time, the NLS 13 reads the driving log message 111 presented to the operator 3 or data related to the driving review 117 input by the operator 3 from the database 41, 43, or 45 of the DMS 9, and can generate effective learning data 121 using the driving review 117 and its related data. For example, if the input operation review 117 is, "The operation that set the temperature of object "yy" to "ww" degrees at date and time "xx" was "vv" minutes too early," the NLS13 uses, in addition to this operation review, for example, the transition of plant data during a certain time period before and after date and time "xx," the operating standards, operating rules, and / or operating know-how related to the temperature operation of object "yy," to create learning data that allows the PAS15 to learn when and how to operate the temperature of object "yy" if the plant data changes in the same way.

[0057] The created learning data 121 is saved and accumulated in a learning database 123. The PAS 15 uses the learning data 121 to re-learn the future prediction method and / or the recommended operation generation method (step 125). This re-learning may be performed automatically online at a timing depending on the amount of unlearned learning data in the learning database 123, the elapsed time since the previous re-learning, etc. Alternatively, the re-learning may be performed manually by an engineer at an appropriate time. During the latter re-learning, the engineer may also manually improve or adjust the logic and / or learning method of the computer program of the PAS 15.

[0058] FIG. 4 shows the operation and function of the plant system 300 for registering operational know-how.

[0059] 4 , the operator 3 can use the UIS 7 to create a know-how message 133 expressing driving know-how (a guide or instruction showing specific driving operations to be performed under specific circumstances) in free text using natural language, for example, based on the results of the above-mentioned review and discussion of driving (step 113). The NLS 13 of the ABSS 11 interprets or analyzes the created free text know-how message 133, and based on the results of the interpretation or analysis, generates driving know-how in a predetermined data format for database registration, and registers the driving know-how in the driving know-how base 45.

[0060] The NLS 13 of the ABSS 11 receives the plant data 39 output from the DCS 5 in real time. Alternatively, the NLS 13 acquires the past plant data 39 from the plant data log base 41. Then, the NLS 13 searches the operation know-how base 45 for operating know-how that matches the operating conditions included in the acquired plant data. Furthermore, the NLS 13 searches the database 41, 43, or 45 for data related to the searched operation know-how. Then, the NLS 13 uses the searched operation know-how and related data to generate an explanatory operation guide 139 that expresses the operation know-how together with the related data in an easy-to-understand natural language (step 137). In this case, similar to step 59 shown in FIG. 2, the NLS 13 can interactively communicate with the operator 3, interpret or analyze the interactive communication, and, depending on the results of the interpretation or analysis (i.e., to meet the needs of the operator 3 as well as possible), select and read relevant data desired by the operator 3 from the database 41, 43, or 45 of the DMS 9, and generate an explanatory driving guide 139 including the relevant data.

[0061] The driving guide 139 generated by the NLS 13 is presented to the operator 3 via the UIS 7, and the operator 3 can use it as reference information (step 141).

[0062] The operator 3 can also input a request 145 for a driving guide that matches any driving condition to the UIS 7 (step 143). In this case, the NLS 13 also reads the driving know-how that matches the conditions specified by the request 145 and data related to the driving know-how from the database 41, 43, or 45, generates a driving guide 139 in the same manner as described above, and provides the driving guide 139 to the driver through the UIS 7 (step 137).

[0063] FIG. 5 shows an example of the configuration of the PAS 15 of the ABSS 11 according to this embodiment.

[0064] The PAS 15 has a world model, and the world model is configured to learn the dynamics (how things move and change) of the plant 1, use that knowledge to predict the future behavior and state of the plant 1, and use the predictions to plan operations on the plant 1 by the operator 3. For example, as shown in FIG. 5 , the PAS 15 has an encoder 151, a predictor 153, and a controller 155. The PAS 15 may further have a decoder 157.

[0065] The encoder 151 receives the plant data 39, which is a high-dimensional vector, in real time and encodes it into a low-dimensional latent vector 161 that includes highly important information related to the behavior and state of the plant 1. The encoder 151 can be configured using, for example, a variational autoencoder (VAE).

[0066] The predictor 153 receives the latent vector 161 from the encoder 151 and predicts a future image (hereafter referred to as a future vector) 163 of the latent vector 161 at a certain near future point in time based on the history of the input latent vector 161. The future vector 163 is fed back to the predictor 153, which then uses the fed-back future vector 163 to predict a future vector 163 at a more distant future point in time. The predictor 153 can be configured using, for example, a recurrent neural network (RNN) with a mixture density network (MDN) output layer.

[0067] The future vector 163 output from the predictor 153 may be used as the future prediction 53 indicating the future state of the plant 1. Alternatively, the decoder 157 may decode the future vector 163 into a high-dimensional vector similar to the plant data 39, and the decoded high-dimensional vector may be used as the future prediction 53. The decoder 157 may be configured using, for example, a variational autoencoder (VAE) similar to the encoder 151.

[0068] Controller 155 receives output vectors 161 and 163 from encoder 151 and predictor 153 and uses output vectors 161 and 163 to generate recommended actions 57 for plant 1 .

[0069] The future prediction 53 and recommended operation 57 output from the PAS 15 are converted by the NLS 13 into explanatory messages 61, 63 expressed in natural language, as explained in relation to step 59 in Fig. 2, and are provided to the operator 3. The operator 3 can refer to the future prediction 53 and recommended operation 57 to determine actual operation instructions 23 and apply them to the plant 1.

[0070] 2 can be implemented by modifying the corresponding portion of the plant data 39 input to the encoder 151 in FIG. 5 using any of the first to third methods described above. As a result, the predictor 153 or the decoder 157 generates a tentative future prediction 81, and the controller 155 generates a tentative recommended operation 85. The tentative future prediction 53 and the tentative recommended operation 57 are converted by the NLS 13 into an explanatory message 89 expressed in natural language, as described with respect to step 87 in FIG. 2, and provided to the operator 3.

[0071] 5, as a result of an actual operation instruction 23 by the operator 3, plant data 39 indicating the actual situation of the plant 1 is input to the encoder 151. The PAS 15 can automatically perform reinforcement learning by using the plant data 39 input to the encoder 151 as experience.

[0072] Also, as shown in FIG. 5 (and as described with respect to step 119 of FIG. 3), based on the operation review 117 from the operator 3, the NLS 13 generates training data 121, which can then be used to retrain the PAS 15.

[0073] 2 to 5, the ABSS 11 includes the NLS 13 and the PAS 15. The PAS 15 is interposed between the plant 1 and the NLS 13. The PAS 15 receives plant data 39 output from the plant 1, uses the received plant data 39 to generate future predictions of the behavior and state of the plant 1, and generates recommended operations for the plant 1 using the future predictions.

[0074] The NLS 13 is also interposed between the operator 3 (UIS 7), the PAS 15, and the DMS 9 (databases 41, 43, 45). The NLS 13 receives a future prediction or a recommended operation from the PAS 15, converts the future prediction or the recommended operation into a message expressed in natural language, and provides the message to the operator 3 via the UIS 7. In this case, the NLS 13 obtains data related to the future prediction or the recommended operation from the DMS 9 while interacting with the operator 3, and can use the obtained related data to make the content of the message provided to the operator 3 more explanatory or specific.

[0075] The NLS 13 can also receive free-text messages in natural language created by the operator 3, interpret or analyze the received messages, and, based on the results of the interpretation or analysis, search for data requested by the operator 3 from the databases 41, 43, or 45, and / or instruct the PAS 15 to generate a future prediction or a recommended operation requested by the operator 3 and return it to the NLS 13. The NLS 13 can then convert the searched data and / or the returned future prediction or recommended operation into a message expressed in natural language and provide it to the operator 3 via the UIS 7. In this case, the NLS 13 can also obtain related data from the DMS 9 while interacting with the operator 3, and use the related data to make the content of the message to be provided to the operator 3 more explanatory or specific.

[0076] Therefore, the ABSS 11 can provide useful and accurate support to the plant operators, and can also provide friendly and easy-to-understand communication to the operators.

[0077] The above-described embodiments have been described in detail to clearly explain the present invention, and are not necessarily limited to those including all of the described configurations. Furthermore, some of the configurations of each embodiment can be added to, deleted from, or replaced with other configurations.

[0078] Furthermore, the above-described embodiment is merely an example intended to assist plant operators. The present invention can be implemented in various different forms applied to other purposes. For example, with regard to an activity-based support system according to one embodiment of the present invention applied to the purpose of assisting a person to act in accordance with the circumstances of a certain environment, those skilled in the art can easily understand the configuration, functions, and operation of the activity-based support system by replacing the combination of "plant" and "DCS" with "the environment," "operator" with "the person," "plant data" with "environmental data indicating the circumstances of the environment," and "operation" and "driving" with "action" in the above description with reference to FIGS. 2 to 5 .

[0079] In the operation of the plant described above, major operational errors are not tolerated. In contrast, there may be fields in which minor errors or inappropriate actions are not considered a major problem. In the case of an activity-based assistance system applied to such fields, the activity-based assistance system may be able to execute the generated recommended actions on behalf of a person.

[0080] Furthermore, the above-described configurations, functions, processing units, processing means, etc. may be partially or entirely implemented in hardware, for example, by designing them as integrated circuits. The present invention can also be realized by software program code that implements the functions of the embodiments. In this case, a storage medium on which the program code is recorded is provided to a computer, and a processor included in the computer reads the program code stored in the storage medium. In this case, the program code itself read from the storage medium implements the functions of the above-described embodiments, and the program code itself and the storage medium on which it is stored constitute the present invention. Examples of storage media for providing such program code include flexible disks, CD-ROMs, DVD-ROMs, hard disks, solid-state drives (SSDs), optical disks, magneto-optical disks, CD-Rs, magnetic tape, non-volatile memory cards, and ROMs.

[0081] Furthermore, the program code that realizes the functions described in the above embodiments can be implemented in a wide range of program or script languages, such as assembler, C / C++, perl, Shell, PHP, Java (registered trademark), and Python.

[0082] Furthermore, all or part of the program code of the software that realizes the functions of the above embodiments may be stored in the computer's storage resources in advance, or, if necessary, may be stored in the computer's storage resources from a non-temporary storage device of another device connected to the network, or from a non-temporary storage medium via an external I / F (not shown) provided on the computer.

[0083] Furthermore, the program code of the software that realizes the functions of the above embodiments may be distributed via a network and stored in a storage means such as a computer's hard disk or memory, or in a storage medium such as a CD-RW or CD-R, and the processor of the computer may read and execute the program code stored in the storage means or the storage medium.

[0084] 1: Plant, 3: Operator, 5...DCS (Distributed Control System), 7: UIS (User Interface / User Experience System), 9: DMS (Data Management System), 11: ABSS (Behavior-Based Support System), 13: NLS (Natural Language System), 15: PAS (Predictive Action System), 41: Plant Data Log Base, 43: Operation Report Log Base, 45: Operation Know-How Base, 151: Encoder, 153: Predictor, 155: Controller, 157: Decoder

Claims

1. In an action-based support system for assisting a person who acts according to the environmental situation, a natural language system that conducts interactive communication with the person using natural language; a predictive action system that inputs environmental data indicating the situation of the environment, generates a future prediction indicating the future situation of the environment using the input environmental data, and generates a recommended action using the generated future prediction; The natural language system inputs the future prediction or the recommended action from the predictive action system, interprets or analyzes the interactive communication with the person, and according to the result of interpreting or analyzing the interactive communication, generates a message expressed in the natural language based on the input future prediction or the recommended action, and provides the generated message to the person. An action-based support system.

2. In the action-based support system according to claim 1, the natural language system can access a data management system that manages data related to the environment and the person's actions; the natural language system obtains first related data related to the input recommended action from the data management system, creates a recommended action message expressed in the natural language using the input recommended action and the obtained first related data, and provides the created recommended action message to the person. An action-based support system.

3. In the action-based support system according to any one of claims 1 or 2, the natural language system can access a data management system that manages data related to the environment and the person's actions; the natural language system obtains second related data related to the input future prediction from the data management system, creates a future prediction message expressed in the natural language using the input recommended action and the obtained second related data, and provides the created future prediction message to the person. An action-based support system.

4. In the action-based support system according to any one of claims 1 or 2, the natural language system inputs an inquiry message expressed in a free text of the natural language created by the person, interprets or analyzes the input inquiry message, and according to the result of interpreting or analyzing the inquiry message, inputs provisional environmental data obtained by partially changing the environmental data to the prediction action system; the prediction action system uses the input provisional environmental data to generate a provisional future prediction, and uses the generated provisional future prediction to generate a provisional recommended action; the natural language system inputs the provisional future prediction or the provisional recommended action from the prediction action system, generates a response message expressed in the natural language based on the input provisional future prediction or the provisional recommended action, and provides the generated response message to the person. An action-based support system.

5. In the action-based support system according to claim 4, the natural language system interprets or analyzes the interactive communication with the person, and generates the response message based on the provisional future prediction or the provisional recommended action according to the result of interpreting or analyzing the interactive communication. An action-based support system.

6. In the action-based support system according to claim 4, the natural language system can access a data management system that manages data related to the environment and the person's actions; the natural language system obtains third related data related to the input provisional future prediction from the data management system, and creates the response message using the input provisional future prediction and the obtained third related data. An action-based support system.

7. In the action-based support system according to claim 4, the natural language system can access a data management system that manages data related to the environment and the person's actions; the natural language system obtains fourth related data related to the input provisional recommended action from the data management system, and creates the response message using the input provisional recommended action and the obtained fourth related data. An action-based support system.

8. In the action-based support system according to any one of claims 1 or 2, the natural language system can access a data management system that manages data related to the environment and the person's actions. The natural language system inputs a log request created by the person, interprets or analyzes the interactive communication with the person, and according to the result of interpreting or analyzing the interactive communication, obtains fifth related data related to the input log request from the data management system, creates a log message using the obtained fifth related data, and provides the created log message to the person. An action-based support system.

9. In the action-based support system according to any one of claims 1 or 2, the prediction action system has a deep neural network that has pre-learned a method for generating the future prediction or the recommended action. The natural language system can access a data management system that manages data related to the environment and the person's actions. The natural language system inputs an action review created by the person, obtains sixth related data related to the input action review from the data management system, and creates learning data for the prediction action system using the input action review and the obtained sixth related data. The prediction action system re-learns the deep neural network using the created learning data. An action-based support system.

10. In the action-based support system according to any one of claims 1 or 2, the natural language system inputs a know-how message related to an action wh-question created by the person using a free text, interprets or analyzes the input know-how message, generates action know-how according to the result of interpreting or analyzing the know-how message, and registers the generated action know-how in the data management system. An action-based support system.

11. In the behavior-based support system according to any one of claims 1 and 2, the natural language system can access a data management system that manages data related to the environment and the person's behavior. The data management system manages one or more behavior know-hows. The natural language system inputs the environmental data, obtains relevant behavior know-hows related to the input environmental data from the data management system, obtains seventh related data related to the input environmental data or related to the obtained relevant behavior know-hows from the data management system, creates a behavior guide using the obtained relevant behavior know-hows and the seventh related data, and provides the created behavior guide to the person. A behavior-based support system.

12. In the behavior-based support system according to any one of claims 1 and 2, the natural language system can access a data management system that manages data related to the environment and the person's behavior. The data management system manages one or more behavior know-hows. The natural language system inputs a behavior ID request created by the person, obtains relevant behavior know-hows related to the input behavior guide request from the data management system, obtains eighth related data related to the input behavior guide request or related to the obtained relevant behavior know-hows from the data management system, creates a behavior guide using the obtained relevant behavior know-hows and the eighth related data, and provides the created behavior guide to the person. A behavior-based support system.

13. In the behavior-based support system according to any one of claims 1 and 2, the predictive behavior system has a deep neural network that has pre-learned a method for generating the future prediction or the recommended behavior. The predictive behavior system performs reinforcement learning of the deep neural network using the input environmental data. A behavior-based support system.

14. In the action-based support system according to any one of claims 1 or 2, the prediction action system has a world model, and the world model includes an encoder that encodes the input environmental data into the lower-dimensional latent vector, a predictor that inputs the latent vector from the encoder and generates a future vector indicating a future image of the latent vector, and a controller that inputs the latent vector from the encoder and inputs the future vector from the predictor to generate the recommended action. An action-based support system.

15. In an action-based support system for assisting a person who acts according to the situation of the environment, it includes a storage device that stores a computer program and a processor that mechanically reads and executes the computer program. By executing the computer program, the processor conducts interactive communication with the person using natural language, inputs environmental data indicating the situation of the environment, generates a future prediction indicating the future situation of the environment using the input environmental data, generates a recommended action using the generated future prediction, interprets or analyzes the interactive communication with the person, generates a message expressed in the natural language based on the future prediction or the recommended action according to the result of interpreting or analyzing the interactive communication, and provides the generated message to the person. An action-based support system.

16. A program storage medium storing a machine-readable computer program, wherein the program includes instruction codes for causing a processor to perform an action-based support method, and the action-based support method includes: performing interactive communication with the person using natural language; inputting environment data indicating the situation of the environment; generating a future prediction indicating a future situation of the environment using the input environment data; generating a recommended action using the generated future prediction; interpreting or analyzing the interactive communication with the person; generating a message expressed in the natural language based on the future prediction or the recommended action according to the result of interpreting or analyzing the interactive communication; and providing the generated message to the person. A program recording medium.

17. In the action-based support system according to any one of claims 1 or 2, the prediction action system executes the generated recommended action. An action-based support system.

Images