Action-based support system and program storage medium

Abstract

Disclosed is an action-based support system (ABSS) 11 for providing support that is accurate and easy to understand for a person performing an action. The ABSS 11 is provided with a natural language system (NLS) 13 and a predictive action system (PAS) 15. For example, in an embodiment for supporting an operator 3 of a plant 1, the NLS 13 uses natural language to perform interactive communication with the operator 3. The PAS 15 generates a future prediction 53 of circumstances of the plant 1 and a recommended operation 57 with respect to the plant 1. The NLS 13 provides, to the operator 3, an explanatory message represented in natural language based on the future prediction 53 or recommended operation 57 in accordance with the interactive communication with the operator 3.

Description

Behavior-based support system and program storage medium

[0001] The present invention relates to an action-based assistance system that assists a person in deciding on an action depending on the environmental situation, or automatically decides on or executes an action on behalf of a person, for example, a system that provides assistance using artificial intelligence.

[0002] An example of behavior-based assistance is assistance for plant operators in operating a chemical plant or the like.

[0003] As artificial intelligence automatically assists in plant operations in more situations, the burden on operators will be reduced.

[0004] As a related prior art, there is a plant operation support system described in Patent Document 1.

[0005] This system includes a real-time inference system that inputs plant data generated in the plant system and performs inference, and an AI training system that inputs training data created using the plant data and trains the real-time inference system. The AI ​​training system has a know-how evaluation unit that evaluates the reliability of the inference results and selectively requests operators to participate in the creation of training data depending on the reliability evaluation result.

[0006] JP 2023-111624 A

[0007] In a plant where major operational failures are unacceptable, a behavior-based support system is expected to provide operators with the most appropriate operational recommendations possible. Furthermore, communication between the behavior-based support system and operators should be as easy to understand as possible for operators, just like a conversation between people.

[0008] The above also applies to behavioral support in various fields other than plant operation, such as driving assistance or automated driving for automobile drivers, and behavioral and thinking support for factory workers and office workers.

[0009] An object of the present invention is to provide an activity-based assistance system that provides more accurate assistance to a person.

[0010] Another object of the present invention is to make communication between an activity-based assistance system and a person easier for the person to understand.

[0011] Other objects of the present invention will become apparent in the following description.

[0012] According to one embodiment, a behavior-based support system for supporting a person in acting in accordance with an environmental situation includes a natural language system that uses natural language to engage in interactive communication with the person, and a predictive behavior system that inputs environmental data indicating the environmental situation, uses the input environmental data to generate a future prediction indicating the future situation of the environment, and generates recommended behavior using the generated future prediction.

[0013] The natural language system inputs the future prediction or the recommended action from the predictive action system, interprets or analyzes the interactive communication with the person, and, depending on the results of interpreting or analyzing the interactive communication, generates a message expressed in the natural language based on the input future prediction or the recommended action, and provides the generated message to the person.

[0014] According to one embodiment, the behavior-based support system includes a storage device storing a computer program and a processor that mechanically reads and executes the computer program. By executing the computer program, the processor engages in interactive communication with the person using natural language, inputs environmental data indicating the state of the environment, generates a future prediction indicating a future state of the environment using the input environmental data, generates a recommended action using the generated future prediction, interprets or analyzes the interactive communication with the person, generates a message expressed in the natural language based on the future prediction or the recommended action according to a result of interpreting or analyzing the interactive communication, and provides the generated message to the person.

[0015] 1 illustrates an example of the hardware configuration of a behavior-based operation support system according to one embodiment. 2 illustrates an example of the configuration of an overall plant system to which the behavior-based operation support system according to this embodiment is applied, and the functions and operations of the plant system that operate during plant operation. 3 illustrates an example of the configuration of the plant system, and the functions and operations of the plant system that operate when reviewing past operations. 4 illustrates an example of the configuration of the plant system, and the functions and operations of the plant system that operate when registering operational know-how. 5 illustrates an example of the configuration of a world model incorporated into a predictive behavior system in the behavior-based operation support system according to this embodiment. 6 illustrates an example of the configuration of a recommended behavior generation unit in the plant operation support system according to this embodiment. 7 illustrates an example of the configuration of a causal logic learning unit in the plant operation support system according to this embodiment. 8 illustrates an example of the configuration of a causal analysis logic unit in the plant operation support system according to this embodiment.

[0016] Hereinafter, embodiments of the present invention will be described with reference to the drawings. Note that the embodiments described below do not limit the scope of the invention as claimed, and not all of the elements and combinations thereof described in the embodiments are necessarily essential to the solution of the invention.

[0017] In the drawings explaining the embodiments, parts having the same functions are given the same reference numerals, and repeated explanations thereof will be omitted.

[0018] FIG. 1 shows the physical configuration of a computer system to which an activity-based support system according to one embodiment is applied.

[0019] As shown in FIG. 1, a computer system 200 includes a plurality of (or one) physical computers 201 connected to a network 240 .

[0020] The network 240 is one or more communication networks, and may include, for example, at least one of a Fibre Channel (FC) network and an Internet Protocol (IP) network. The network 240 may exist outside the computer system 200.

[0021] Each physical computer 201 is, for example, a general-purpose computer, and has physical computer resources 350. The physical computer resources 350 include a communication interface unit 251, a storage unit 252, and a processor unit 253 connected to these.

[0022] The computer system 200 may be, for example, a cloud computing system that provides XaaS (X as a Service). Note that "XaaS" generally refers to a service that makes available some resources (e.g., hardware, lines, software execution environment, application programs, development environments, etc.) necessary for system construction or operation via a network such as the Internet. The letter (or word) used as the "X" in XaaS varies depending on the type (service model) of XaaS. Examples of XaaS include PaaS (Platform as a Service), SaaS (Software as a Service), IaaS (Infrastructure as a Service), and HaaS (Hardware as a Service).

[0023] The behavior-based support system according to one embodiment can be realized by the processor unit 253 of one or more physical computers 201 of the computer system 200 executing a computer program stored in the storage unit 252 .

[0024] As an example, the following describes an example of an activity-based support system according to an embodiment, which is applied to support the operation of a plant such as a chemical plant. Figures 2, 3, and 4 show an example of the configuration and functions of the activity-based support system according to this embodiment and an overall plant system including its related systems.

[0025] 2, 3, and 4, the plant system 300 includes a plant 1, an operator 3, and a distributed control system (hereinafter referred to as "DCS") 5. The plant system 300 further includes one or more user interface / user experience systems (hereinafter referred to as "UIS") 7 and a data management system (hereinafter referred to as "DMS") 9. The plant system 300 also includes an action-based support system (hereinafter referred to as "ABSS") 11 according to one embodiment.

[0026] A single plant 1 may have a plurality of each of the other types of components 3, 5, 7, 9, and 11. However, for the sake of convenience, the following description focuses on one component of each component type and describes its operation and function. From this description, a person skilled in the art should be able to easily understand the case where a plurality of components of each component type exist.

[0027] FIG. 2 shows the operation and function of the plant system 300 when the plant 1 is in operation.

[0028] 2, when the operator 3 inputs an operation instruction 23 to the UIS 7 (step 21), the operation instruction 23 is received by the DCS 5. The DCS 5 controls the plant 1 in accordance with the operation instruction 23 (step 25). The plant 1 operates in accordance with the control 27 (step 29).

[0029] Numerous sensors are installed at numerous locations in the plant 1 to sense various conditions at each location (operating conditions such as temperature, pressure, and flow rate, as well as the quality of materials and products). These sensors sense the conditions at each location and output sensor data indicating the location and the condition value (step 31). The DCS 5 receives the sensor data (step 35). The DCS 5 also monitors the sensor data 33 received from the plant 1 to check for abnormalities. If an abnormality is detected, the DCS 5 notifies the operator 3 via the UIS 7 (step 37). The DCS 5 outputs the received operation instructions, received sensor data, and detected abnormality data in real time. Hereinafter, the operation instructions, sensor data, and abnormality data output from the DCS 5 are collectively referred to as "plant data." The DMS 9 receives plant data 39 from the DCS 5 and stores the received plant data 39 in the plant data log base 41.

[0030] The DMS 9 also has an operation report log base 43 and an operation know-how base 45. The operation report log base 43 stores operation reports (for example, daily operation reports, handover messages, etc.) created by the operators 3 as operation report logs. In addition to the above-mentioned operation report logs, the operation report log base 43 also stores various documents related to the operation of the plant 1 (for example, operation standards, operation procedures, operation rules, etc.). The operation know-how base 45 stores various types of operation know-how created by the operators 3.

[0031] During operation of the plant 1, not only are the basic processes as described above performed, but operation assistance is also provided by the ABSS 11. The configuration and functions of the ABSS 11 will be described below.

[0032] 2 , the ABSS 11 can communicate with other components of the plant system 300, such as the DCS 5, the DMS 9, and the UIS 7, and provides various support services to the operator 3 by using the communications with these other components. The ABSS 11 has a Natural Language System (hereinafter referred to as "NLS") 13 and a Predictive Action System (hereinafter referred to as "PAS") 15.

[0033] In the following description, the two systems, NLS 13 and PAS 15, are assumed to be included as subsystems in the ABSS 11, but this is merely a convenient example. It may also be understood that not only the NLS 13 and PAS 15, but also all or part of the DCS 5, DMS 9, and UIS 7 are subsystems of the ABSS 11.

[0034] The NLS 13 of the ABSS 11 has the function of interpreting, analyzing, and generating messages using natural language that is commonly used and understandable by humans (operators 3). Here, natural language may include not only characters, sentences, or text that are commonly used by humans, but also other methods of expression that are commonly used and understandable by humans, such as diagrams, tables, and pictures.

[0035] The NLS 13 can input a message in a natural language, interpret or analyze it, and, depending on the results of the interpretation or analysis, generate and output a new message expressed in an artificial language (e.g., a language used by a machine such as the DCS5, DMS9, and / or PAS15) and / or a new message expressed in a natural language. The NLS 13 can also input a message in an artificial language, interpret or analyze it, and, depending on the results of the interpretation or analysis, create and output a new message in a natural language. To achieve such natural language processing functionality, the NLS 13 can have a deep neural network of a type called a Large Language Model (hereinafter referred to as "LLM"), such as a Generative Pre-trained Transformer (GPT).

[0036] The NLS 13 is capable of communicating with components of the plant system 300, such as the PAS 15, the DMS 9, and the UIS 7.

[0037] The PAS 15 of the ABSS 11 has the role of generating information related to the future operation of the plant 1 that is useful to the operator 3. The PAS 15 has a function of predicting the future behavior and state of the plant 1, and a function of generating an action recommended for the operator 3 to take (particularly, an operation command recommended to be given to the plant 1) based on the predicted future behavior and state. To realize this predictive action function, the PAS 15 can have, for example, a deep neural network of a type called a World Model (hereinafter referred to as "WM").

[0038] The PAS 15 is capable of communicating with components of the plant system 300, such as the NLS 13, the DCS 5, the DMS 9, and the UIS 7.

[0039] When the plant 1 is in operation, the ABSS 11 can perform the following processes.

[0040] 2, the PAS 15 of the ABSS 11 receives plant data output from the DCS 5 in real time, and generates and outputs a future prediction 53 of the behavior and state of the plant 1 using the received plant data (step 51). The PAS 15 further generates and outputs a recommended operation (operational instruction recommended to be provided to the plant 1 in the future) 57 based on the generated future prediction 53.

[0041] The NLS 13 receives the future prediction 53 and the recommended operation 57 output from the PAS 15. Using the received future prediction 53 and the recommended operation 57, the NLS 13 creates explanatory messages, i.e., a future prediction message 61 and a recommended operation message 63, that express the future prediction 53 and the recommended operation 57 in natural language in a format that is easier for the operator 3 to understand or use (step 59). The NLS 13 then displays the created explanatory messages 61 and 63 on the UIS 7, respectively, to communicate them to the operator 3.

[0042] When creating each of the above messages 61 and 63, the NLS 13 can search for data related to each of the input future predictions 53 and recommended operations 57 (e.g., plant data logs, operation report logs, operating standards, operating rules, or operating know-how) from the plant data log base 41, the operation report log base 43, and / or the operation know-how base 45. The NLS 13 can then add the searched related data to the future predictions 53 or the recommended operations 57 or edit the data to create the explanatory future prediction message 61 and the recommended operation message 63. Furthermore, the NLS 13 communicates interactively with the operator 3 using natural language and interprets or analyzes the interactive communication. Then, depending on the results of the interpretation or analysis, the NLS 13 can determine what type of relevant data the operator 3 wants and / or what format (e.g., text, table, diagram, formula, or a combination thereof) is appropriate to express the messages 61, 63 in, and create the messages 61, 63 according to that determination.

[0043] The operator 3 can use the messages 61 and 63 as reference or assistance information for proper operation of the plant 1 (steps 65, 67).

[0044] Furthermore, the operator 3 can input any query for additional information in free text in natural language to the UIS 7 (step 71), and the query message 73 is received by the NLS 13 of the ABSS 11 via the UIS 7. The query message 73 may inquire about the result that would be caused if a specified operation were performed, such as "If an operation 'xx' is performed, what will the future prediction be?" Alternatively, the query message 73 may inquire about the operation that should be performed to achieve a specified result, such as "To make the production volume of monomethylamine 'xx', what temperature should 'yy' be set to?" The specific content of the query message 73 can be freely determined by the operator 3.

[0045] The NLS 13 of the ABSS 11 interprets or analyzes the received inquiry message 73, and thereby determines a processing method to be performed in order to respond to the inquiry from the operator 3 (step 75). As the processing method, there are several possible examples as follows:

[0046] [First Method] This method can be applied when a question is asked about the likely outcome of performing a specified operation, such as the above example, "If an operation 'xx' is performed, what will the future prediction be?" In this method, the NLS 13 issues an instruction 77 to the PAS 15 to change part of the plant data 39 used in the future prediction. Then, the PAS 15 uses the changed plant data to perform a tentative future prediction (step 79).

[0047] For example, only the parameter values ​​of the relevant operation instructions (e.g., set values ​​such as temperature or pressure at a specific location in the plant 1) in the actual plant data 39 input from the DCS 5 to the PAS 15 are changed to the parameter values ​​of the operation instructions specified in the query message 73. The PAS 15 uses the changed operation instructions combined with other data in the actual plant data 39 as provisional plant data to generate a provisional future forecast 81 indicating the inquired results (step 79). The NLS 13 receives the generated provisional future forecast 81 and uses the provisional future forecast 81 to generate a reply message 89 to the query message 73 (step 87). The operator 3 receives the reply message 89 via the UIS 7 and can use it as reference information.

[0048] [Second Method] This method can be applied when a question is asked about the operation to be performed to obtain a specified result, such as the above-mentioned example, "What temperature should be set to 'yy' to achieve 'xx' production of monomethylamine?" In this method, the NLS 13 analyzes at least a portion of various data (e.g., plant data logs, operation report logs, or operational know-how) managed in the databases 41, 43, and 45 of the DMS 9, and selects one or more operation instructions that are likely to affect the result specified in the query message 73 (i.e., have a high correlation with the specified result). Then, the NLS 13 issues an instruction 77 to the PAS 15 to perform a tentative future forecast while changing the parameter values ​​of the selected operation instructions to various values.

[0049] In accordance with this instruction 77, the PAS 15 changes only the parameter values ​​of the selected operation instructions in the actual plant data 39 input from the DCS 5 to various values, for example, by trial and error, and generates a tentative future forecast 81 for each changed parameter value (step 79). Furthermore, the PAS 15 generates a tentative recommended operation 85 for each changed parameter value using the tentative future forecast 81 (step 83).

[0050] The NLS 13 receives from the PAS 15 tentative future forecasts 81 and tentative recommended operations 85 corresponding to various changed parameter values. Then, the NLS 13 selects, from the various received tentative future forecasts 81, the one that is closest to the result specified in the query message 73, and selects a tentative recommended operation 85 corresponding to the selected tentative future forecast 81. The NLS 13 generates a reply message 89 to the query message 73 using the selected tentative future forecast 81 and tentative recommended operation 85 (step 87). The operator 3 receives the reply message 89 via the UIS 7 and can use it as reference information.

[0051] [Third Method] Like the second method, this method can also be applied when a query is made about the operation to be performed to obtain a specified result. This method is a modified combination of the first and second methods. The NLS 13 analyzes at least a portion of various data (e.g., plant data logs, operation report logs, or operational know-how) managed in the databases 41, 43, and 45 of the DMS 9 to identify one or more parameter values ​​of one or more operational instructions that are effective for obtaining the result specified in the query message 73. The NLS 13 then issues an instruction 77 to the PAS 15 to change the corresponding portion of the plant data 39 to the parameter values ​​of the identified operational instructions and to perform a tentative future forecast using the changed plant data 39 as tentative plant data. The subsequent process is similar to the first method. In addition, as in the second method, the parameter values ​​of the operational instructions are changed in multiple places, and tentative future forecasts are generated for each parameter value. Then, based on the plurality of tentative future predictions corresponding to the different parameter values, a tentative recommended operation corresponding to the tentative future prediction that is closest to the specified result is selected.

[0052] In any of the first to third methods described above, in step 87 of creating a reply message, the NLS 13 interactively communicates with the operator 3 and interprets or analyzes the communication, similar to step 59 described above. Then, depending on the results of the interpretation or analysis, the NLS 13 can read data related to the tentative future prediction 81 or the tentative recommended operation 85 from the database 41, 43, or 45 of the DMS 9, and generate an explanatory reply message 89 including the related data, so as to meet the requests of the operator 3 as well as possible.

[0053] FIG. 3 shows the operation and function of a plant system 300 for reviewing past plant operations and relearning the ABSS 11.

[0054] 3 , when an operator 3 uses the UIS 7 to create an operation report (e.g., a daily operation report) 103 (step 101), the operation report 103 is registered in the operation report log base 43 by the DMS 9. When the operator 3 inputs a request 107 for an operation log that meets certain conditions (e.g., time, location within the plant, type of log, etc.) into the UIS 7 (step 105), the NLS 13 of the ABSS 11 reads the plant data log and / or operation report log that meets the conditions from the plant data log base 41 and / or the operation report log base 43, interprets or analyzes the log data, and generates an operation log message 111 that expresses the results of the interpretation or analysis using free text in natural language, graphs, etc. (step 109). In this step 109, similar to step 59 shown in FIG. 2, the NLS 13 communicates interactively with the operator 3, interprets or analyzes the interactive communication, and, depending on the results of the interpretation or analysis (i.e., to meet the needs of the operator 3 as best as possible), selects and reads data related to the requested driving log from the database 41, 43, or 45 of the DMS 9, and generates an explanatory driving log message 111 including the related data.

[0055] The driving log message 111 is presented to the operator 3 via the UIS 7. The operator 3 can refer to the driving log message 111 to review and discuss past driving (step 113).

[0056] When the operator 3 writes, for example, the above-mentioned review or discussion of past driving (hereinafter referred to as driving review) 117 in free text and inputs it to the UIS 7, the NLS 13 of the ABSS 11 interprets or analyzes the free text input content 117 to generate learning data (e.g., teacher data for supervised learning of the PAS 15) 121 in a format that can be used for relearning the PAS 15 (step 119). At this time, the NLS 13 reads the driving log message 111 presented to the operator 3 or data related to the driving review 117 input by the operator 3 from the database 41, 43, or 45 of the DMS 9, and can generate effective learning data 121 using the driving review 117 and its related data. For example, if the input operation review 117 is, "The operation that set the temperature of object "yy" to "ww" degrees at date and time "xx" was "vv" minutes too early," the NLS13 uses, in addition to this operation review, for example, the transition of plant data during a certain time period before and after date and time "xx," the operating standards, operating rules, and / or operating know-how related to the temperature operation of object "yy," to create learning data that allows the PAS15 to learn when and how to operate the temperature of object "yy" if the plant data changes in the same way.

[0057] The created learning data 121 is saved and accumulated in a learning database 123. The PAS 15 uses the learning data 121 to re-learn the future prediction method and / or the recommended operation generation method (step 125). This re-learning may be performed automatically online at a timing depending on the amount of unlearned learning data in the learning database 123, the elapsed time since the previous re-learning, etc. Alternatively, the re-learning may be performed manually by an engineer at an appropriate time. During the latter re-learning, the engineer may also manually improve or adjust the logic and / or learning method of the computer program of the PAS 15.

[0058] FIG. 4 shows the operation and function of the plant system 300 for registering operational know-how.

[0059] 4 , the operator 3 can use the UIS 7 to create a know-how message 133 expressing driving know-how (a guide or instruction showing specific driving operations to be performed under specific circumstances) in free text using natural language, for example, based on the results of the above-mentioned review and discussion of driving (step 113). The NLS 13 of the ABSS 11 interprets or analyzes the created free text know-how message 133, and based on the results of the interpretation or analysis, generates driving know-how in a predetermined data format for database registration, and registers the driving know-how in the driving know-how base 45.

[0060] The NLS 13 of the ABSS 11 receives the plant data 39 output from the DCS 5 in real time. Alternatively, the NLS 13 acquires the past plant data 39 from the plant data log base 41. Then, the NLS 13 searches the operation know-how base 45 for operating know-how that matches the operating conditions included in the acquired plant data. Furthermore, the NLS 13 searches the database 41, 43, or 45 for data related to the searched operation know-how. Then, the NLS 13 uses the searched operation know-how and related data to generate an explanatory operation guide 139 that expresses the operation know-how together with the related data in an easy-to-understand natural language (step 137). In this case, similar to step 59 shown in FIG. 2, the NLS 13 can interactively communicate with the operator 3, interpret or analyze the interactive communication, and, depending on the results of the interpretation or analysis (i.e., to meet the needs of the operator 3 as well as possible), select and read relevant data desired by the operator 3 from the database 41, 43, or 45 of the DMS 9, and generate an explanatory driving guide 139 including the relevant data.

[0061] The driving guide 139 generated by the NLS 13 is presented to the operator 3 via the UIS 7, and the operator 3 can use it as reference information (step 141).

[0062] The operator 3 can also input a request 145 for a driving guide that matches any driving condition to the UIS 7 (step 143). In this case, the NLS 13 also reads the driving know-how that matches the conditions specified by the request 145 and data related to the driving know-how from the database 41, 43, or 45, generates a driving guide 139 in the same manner as described above, and provides the driving guide 139 to the driver through the UIS 7 (step 137).

[0063] FIG. 5 shows an example of the configuration of the PAS 15 of the ABSS 11 according to this embodiment.

[0064] The PAS 15 has a world model, and the world model is configured to learn the dynamics (how things move and change) of the plant 1, use that knowledge to predict the future behavior and state of the plant 1, and use the predictions to plan operations on the plant 1 by the operator 3. For example, as shown in FIG. 5 , the PAS 15 has an encoder 151, a predictor 153, and a controller 155. The PAS 15 may further have a decoder 157.

[0065] The encoder 151 receives the plant data 39, which is a high-dimensional vector, in real time and encodes it into a low-dimensional latent vector 161 that includes highly important information related to the behavior and state of the plant 1. The encoder 151 can be configured using, for example, a variational autoencoder (VAE).

[0066] The predictor 153 receives the latent vector 161 from the encoder 151 and predicts a future image (hereafter referred to as a future vector) 163 of the latent vector 161 at a certain near future point in time based on the history of the input latent vector 161. The future vector 163 is fed back to the predictor 153, which then uses the fed-back future vector 163 to predict a future vector 163 at a more distant future point in time. The predictor 153 can be configured using, for example, a recurrent neural network (RNN) with a mixture density network (MDN) output layer.

[0067] The future vector 163 output from the predictor 153 may be used as the future prediction 53 indicating the future state of the plant 1. Alternatively, the decoder 157 may decode the future vector 163 into a high-dimensional vector similar to the plant data 39, and the decoded high-dimensional vector may be used as the future prediction 53. The decoder 157 may be configured using, for example, a variational autoencoder (VAE) similar to the encoder 151.

[0068] Controller 155 receives output vectors 161 and 163 from encoder 151 and predictor 153 and uses output vectors 161 and 163 to generate recommended actions 57 for plant 1 .

[0069] The future prediction 53 and recommended operation 57 output from the PAS 15 are converted by the NLS 13 into explanatory messages 61, 63 expressed in natural language, as explained in relation to step 59 in Fig. 2, and are provided to the operator 3. The operator 3 can refer to the future prediction 53 and recommended operation 57 to determine actual operation instructions 23 and apply them to the plant 1.

[0070] 2 can be implemented by modifying the corresponding portion of the plant data 39 input to the encoder 151 in FIG. 5 using any of the first to third methods described above. As a result, the predictor 153 or the decoder 157 generates a tentative future prediction 81, and the controller 155 generates a tentative recommended operation 85. The tentative future prediction 53 and the tentative recommended operation 57 are converted by the NLS 13 into an explanatory message 89 expressed in natural language, as described with respect to step 87 in FIG. 2, and provided to the operator 3.

[0071] 5, as a result of an actual operation instruction 23 by the operator 3, plant data 39 indicating the actual situation of the plant 1 is input to the encoder 151. The PAS 15 can automatically perform reinforcement learning by using the plant data 39 input to the encoder 151 as experience.

[0072] Also, as shown in FIG. 5 (and as described with respect to step 119 of FIG. 3), based on the operation review 117 from the operator 3, the NLS 13 generates training data 121, which can then be used to retrain the PAS 15.

[0073] 2 to 5, the ABSS 11 includes the NLS 13 and the PAS 15. The PAS 15 is interposed between the plant 1 and the NLS 13. The PAS 15 receives plant data 39 output from the plant 1, uses the received plant data 39 to generate future predictions of the behavior and state of the plant 1, and generates recommended operations for the plant 1 using the future predictions.

[0074] The NLS 13 is also interposed between the operator 3 (UIS 7), the PAS 15, and the DMS 9 (databases 41, 43, 45). The NLS 13 receives a future prediction or a recommended operation from the PAS 15, converts the future prediction or the recommended operation into a message expressed in natural language, and provides the message to the operator 3 via the UIS 7. In this case, the NLS 13 obtains data related to the future prediction or the recommended operation from the DMS 9 while interacting with the operator 3, and can use the obtained related data to make the content of the message provided to the operator 3 more explanatory or specific.

[0075] The NLS 13 can also receive free-text messages in natural language created by the operator 3, interpret or analyze the received messages, and, based on the results of the interpretation or analysis, search for data requested by the operator 3 from the databases 41, 43, or 45, and / or instruct the PAS 15 to generate a future prediction or a recommended operation requested by the operator 3 and return it to the NLS 13. The NLS 13 can then convert the searched data and / or the returned future prediction or recommended operation into a message expressed in natural language and provide it to the operator 3 via the UIS 7. In this case, the NLS 13 can also obtain related data from the DMS 9 while interacting with the operator 3, and use the related data to make the content of the message to be provided to the operator 3 more explanatory or specific.

[0076] Therefore, the ABSS 11 can provide useful and accurate support to the plant operators, and can also provide friendly and easy-to-understand communication to the operators.

[0077] 6 is a diagram for explaining in more detail the generation of the recommended operation 57 based on the future prediction 53 by the PAS 15. The recommended action generator 400 shown in FIG. 6 includes a causal logic unit 301, an operation history DB 302, and a causal logic learning unit 303.

[0078] The causal logic unit 301 receives a future prediction 53 of the behavior and state of the plant 1 based on the plant data 39 from the world model 158 in the PAS 15. The causal logic unit 301 also holds logic 96 that indicates the relationship between the future prediction 53 and a recommended operation 57 based on the future prediction 53. The causal logic unit 301 generates the recommended operation 57 using the future prediction 53 and the logic 96.

[0079] The causal logic unit 301 outputs recommended information 91 including a future prediction 53 and a recommended operation 57 to the UIS 7, and outputs prediction information 92 including an ideal state 97 (see Figure 7), which is the behavior or state of the plant 1 that should be achieved by performing the future prediction 53 and the recommended operation 57, to the operation history DB 302.

[0080] The UIS 7 provides the recommended information 91 from the causal logic unit 301 to the operator 3 by displaying it. At this time, the recommended information 91 may be converted by the NLS 13 into explanatory recommended messages (such as the future prediction message 61 and the recommended operation message 63) expressed in natural language in a format that is easier for the operator 3 to understand or use, and then displayed.

[0081] The operator 3 who has received the recommendation information 91 determines an actual operation instruction 23 by referring to the recommendation information 91 and inputs the actual operation instruction 23 to the UIS 7. The UIS 7 outputs the input operation instruction 23 to the DCS 5, and the DCS 5 performs control 27 of the plant 1 in accordance with the operation instruction 23. The UIS 7 also outputs an actual operation 93 indicating the input operation instruction 23 to the operation history DB 302.

[0082] The operation history DB 302 receives prediction information 92 from the causal logic unit 301, receives actual operations 93 from the UIS 7, and receives plant data 39 from the DCS 5 after control 27 based on operation instructions 23 has been performed as an actual state 94 indicating the behavior and state of the plant 1 actually achieved by the actual operations 93.

[0083] The operation history DB 302 stores an operation history 95, which is a history related to the actual operation 93, based on the prediction information 92, the actual operation 93, and the actual state 94. The operation history 95 indicates, for example, the actual operation 93, the actual state 94, and the ideal state 97 for each future prediction 53.

[0084] The causal logic learning unit 303 acquires the operation history 95 from the operation history DB 302, and updates the logic 96 of the causal logic unit 301 based on the operation history 95. There is no particular limitation on the timing for updating the logic 96. For example, the logic 96 may be updated at regular intervals, every time an operation instruction 23 is input, or when an instruction is received from the operator 3 or the like.

[0085] 7 is a diagram showing a detailed configuration of the causal logic learning unit 303. The causal logic learning unit 303 shown in FIG.

[0086] The clustering unit 331 clusters each of the actual operations 93 included in the operation history 95 into a plurality of similar groups in which the pairs of the future predictions 53 and the ideal states 97 included in the operation history 95 are similar to each other, and outputs the clustering results 98. The method (algorithm) for performing the clustering is not particularly limited.

[0087] The prioritization unit 332 assigns a priority to each of one or more actual operations 93 corresponding to a combination of a future prediction 53 and an ideal state 97 included in each similarity group based on the operation history 95 and the clustering result 98. Specifically, the prioritization unit 332 evaluates the actual operations 93 based on an actual state 94 after the operation instruction 23 indicated by the actual operations 93 is input, and assigns a priority to each actual operation 93 based on the evaluation result. For example, the prioritization unit 332 calculates the difference between the ideal state 97 and the actual state 94 (the difference between the values ​​of sensor data indicating the ideal state 97 and the actual state 94) as the evaluation result, and assigns a priority to each actual operation 93 so that the higher the evaluation result (the smaller the difference between the ideal state 97 and the actual state 94), the higher the priority. Note that instead of being calculated by the prioritization unit 332, the evaluation value may be calculated in advance and stored in the operation history DB 302.

[0088] The prioritization unit 332 sets the highest priority actual operation 93 for each similar group as recommended operation information. Furthermore, the prioritization unit 332 outputs, for each similar group, the representative value of the combination of the future prediction 53 and the ideal state 97 in that similar group and the recommended operation information as logic 96 to the causal logic unit 301, thereby updating the logic 96 held in the causal logic unit 301.

[0089] 8 is a diagram showing a detailed configuration of the causal logic unit 301. The causal logic unit 301 shown in FIG.

[0090] The logic DB 311 holds the logic 96 , and when new logic 96 is received from the causal logic unit 301 , the logic DB 311 updates the logic 96 held therein with the new logic 96 .

[0091] The cause input unit 312 generates and outputs cause information 100 related to the causes of the future prediction 53 of the behavior or state of the plant 1, based on the input data 99 input to the cause input assisting device 401. For each recommended operation state, which is the future prediction 53 for which input of an operation instruction 23 is recommended, the cause information 100 includes, for example, at least one of the factors that caused the recommended operation state, an ideal state, one or more candidate operations that are candidates for the operation instruction 23 that are recommended to be input in order to return the recommended operation state to the ideal state, and a reason for inputting each candidate operation.

[0092] The cause input assisting device 401 may be a human interface device (hereinafter referred to as "HID") into which text data is input, such as a keyboard, or an image reading device, such as a scanner. For example, when text data indicating the cause information 100 is directly input as input data 99 to the cause input assisting device 401, the input data 99 may be used as the cause information 100. Alternatively, the cause input assisting device 401 may use an AI model or the like to check whether the input data 99 is appropriate as the cause information 100, and use appropriate data from the input data 99 as the cause information 100. Furthermore, when the cause input assisting device 401 reads specifications for the plant 1 or a previously created operation log as an image, it may recognize characters in the image and generate the cause information 100.

[0093] The cause DB 313 stores the cause information 100 from the cause input unit 812 .

[0094] The linking unit 314 uses the logic 96 in the logic DB 311 and the cause information 100 in the cause DB 313 to generate a recommended operation 57 based on the future prediction 53, and outputs recommended information 91 including the future prediction 53 and the recommended operation 57 to the LLM 315.

[0095] For example, the linking unit 314 determines whether the future prediction 53 indicates a recommended operation state in which it is recommended to input the operation instruction 23. The recommended operation state is, for example, a state in which the sensor data indicates an abnormal value, and the conditions (such as a threshold value for the sensor data) for determining whether the future prediction 53 is a recommended operation state may be included in the cause information 100, or may be stored separately from the cause information 100, for example.

[0096] When the future prediction 53 is a recommended operation state, the linking unit 314 performs pattern matching between the future prediction 53 and the recommended operation state in the cause information 100, and acquires an ideal state corresponding to the matching recommended operation state. The linking unit 314 performs pattern matching between the combination of the future prediction 53 and the ideal state and the representative value of that combination in the logic 96, and sets the recommended operation corresponding to the matching representative value as the recommended operation 57, and outputs recommendation information 91 including the future prediction 53 and the recommended operation 57 to the LLM 315.

[0097] The recommended information 91 may include various supplemental information in addition to the future prediction 53 and the recommended operation 57. The supplemental information is generated by the linking unit 314 based on the cause information 100. The supplemental information may include, for example, at least one of the factors that cause the recommended operation state that matches the future prediction 53, an ideal state corresponding to the recommended operation state, a candidate operation that is recommended to be input as a candidate operation instruction for returning the recommended operation state to the ideal state, and a reason for inputting the candidate operation. The candidate operation is used as a second best option for the recommended operation 57.

[0098] The LLM 315 converts the recommendation information 91 from the linking unit 314 into a recommendation message 71B and outputs it to the UIS 7 .

[0099] The above-described configuration of the recommended action generator 400 is merely an example, and is not limited to this. For example, the operation history DB 302, the logic DB 311, the cause DB 313, and other DBs may be stored in the DMS 9.

[0100] In the recommended action generation unit 400 described above, the causal logic unit 301 outputs the recommended information 91 including the generated recommended operation 57 to the UIS 7, and the operator 3, upon reviewing the recommended information 91, determines the actual operation instruction 23 and inputs it to the UIS 7. However, the recommended operation 57 generated by the causal logic unit 301 may be directly input to the DCS 5 as the operation instruction 23. In this case, it is possible to automatically perform control 27 on the plant 1 according to the future prediction 53 without human intervention. The operator 3 may also be able to select whether to review the recommended information 91 and determine the operation instruction 23 or to automatically input the recommended operation 57 as the operation instruction 23 to the DCS 5. For example, when the operator 3 has not yet sufficiently learned the logic 96, the operator 3 may review the recommended information 91, and when the operator 3 has sufficiently learned the logic 96 and there is no problem with automatically setting the recommended operation 57 as the operation instruction 23, the recommended operation 57 may automatically be set as the operation instruction 23. Whether the logic 96 has been sufficiently learned may be determined by the operator 3, or may be determined based on whether a predetermined condition is met (for example, the recommended operation 57 and the actual operation 93 match for a predetermined number of consecutive times or more).

[0101] In the embodiment described above with reference to Fig. 6 to Fig. 8 , the causal logic unit 301 included in the PAS 15 generates the recommended operation 57 by further using logic indicating the relationship between the future prediction 53 of the behavior and state of the plant 1 and the recommended operation 57. In addition, the causal logic unit 301 updates the logic based on the actual operation performed by the operator 3 on the plant 1. This makes it possible to appropriately generate the recommended operation 57.

[0102] The above-described embodiments have been described in detail to clearly explain the present invention, and are not necessarily limited to those including all of the described configurations. Furthermore, some of the configurations of each embodiment can be added to, deleted from, or replaced with other configurations.

[0103] For example, while the POSS 11 includes the NLS 13 as an interface system for communicating with the operator 3, exchanging information, the interface system is not limited to the NLS 13. Instead of or in addition to providing the above-described messages expressed in natural language as messages based on the future prediction 53 or the recommended operation 57, the interface system may provide other messages. Here, the message refers to notification information provided to the operator 3 and may include information indicating options for the operator 3 to select, interface components such as buttons and check boxes for inputting the selection results, and the like. The options may include, for example, whether or not to execute the recommended operation 57, candidates for the operation instruction 23 (the recommended operation 57 and its candidate operations), and the type and content of an inquiry to be entered into the POSS 11. The notification information is not limited to this example, and may be, for example, information that is friendly and easy to understand for the operator. Furthermore, the inquiry message entered into the POSS 11 is not limited to a message expressed in natural language and may be, for example, information (message) indicating an option selected by the operator 3.

[0104] Furthermore, the above-described embodiment is merely an example intended to assist plant operators. The present invention can be implemented in various different forms applied to other purposes. For example, with regard to an activity-based support system according to one embodiment of the present invention applied to the purpose of assisting a person to act in accordance with the circumstances of a certain environment, those skilled in the art can easily understand the configuration, functions, and operation of the activity-based support system by replacing the combination of "plant" and "DCS" with "the environment," "operator" with "the person," "plant data" with "environmental data indicating the circumstances of the environment," and "operation" and "driving" with "action" in the above description with reference to FIGS. 2 to 5 .

[0105] In the operation of the plant described above, major operational errors are not tolerated. In contrast, there may be fields in which minor errors or inappropriate actions are not considered a major problem. In the case of an activity-based assistance system applied to such fields, the activity-based assistance system may be able to execute the generated recommended actions on behalf of a person.

[0106] Furthermore, the above-described configurations, functions, processing units, processing means, etc. may be partially or entirely implemented in hardware, for example, by designing them as integrated circuits. The present invention can also be realized by software program code that implements the functions of the embodiments. In this case, a storage medium on which the program code is recorded is provided to a computer, and a processor included in the computer reads the program code stored in the storage medium. In this case, the program code itself read from the storage medium implements the functions of the above-described embodiments, and the program code itself and the storage medium on which it is stored constitute the present invention. Examples of storage media for providing such program code include flexible disks, CD-ROMs, DVD-ROMs, hard disks, solid-state drives (SSDs), optical disks, magneto-optical disks, CD-Rs, magnetic tape, non-volatile memory cards, and ROMs.

[0107] Furthermore, the program code that realizes the functions described in the above embodiments can be implemented in a wide range of program or script languages, such as assembler, C / C++, perl, Shell, PHP, Java (registered trademark), and Python.

[0108] Furthermore, all or part of the program code of the software that realizes the functions of the above embodiments may be stored in the computer's storage resources in advance, or, if necessary, may be stored in the computer's storage resources from a non-temporary storage device of another device connected to the network, or from a non-temporary storage medium via an external I / F (not shown) provided on the computer.

[0109] Furthermore, the program code of the software that realizes the functions of the above embodiments may be distributed via a network and stored in a storage means such as a computer's hard disk or memory, or in a storage medium such as a CD-RW or CD-R, and the processor of the computer may read and execute the program code stored in the storage means or the storage medium.

[0110] 1: Plant, 3: Operator, 5...DCS (Distributed Control System), 7: UIS (User Interface / User Experience System), 9: DMS (Data Management System), 11: ABSS (Behavior-Based Support System), 13: NLS (Natural Language System), 15: PAS (Predictive Action System), 41: Plant Data Log Base, 43: Operation Report Log Base, 45: Operation Know-How Base, 151: Encoder, 153: Predictor, 155: Controller, 157: Decoder

Claims

1. In a behavior-based support system for assisting a person who acts according to the environmental situation, an interface system that communicates with the person, an environmental data indicating the environmental situation is input, a future prediction indicating a future situation of the environment is generated using the input environmental data, and a prediction behavior system that generates a recommended behavior using the generated future prediction, and the interface system inputs the future prediction or the recommended behavior from the prediction behavior system, interprets or analyzes the communication with the person, and generates a message based on the input future prediction or the recommended behavior according to the result of interpreting or analyzing the communication, and provides the generated message to the person. Behavior-based support system.

2. In the behavior-based support system according to claim 1, the interface system can access a data management system that manages data related to the environment and the person's behavior, the interface system acquires first related data related to the input recommended behavior from the data management system, creates a message using the input recommended behavior and the acquired first related data, and provides the created recommended behavior message to the person. Behavior-based support system.

3. In the behavior-based support system according to any one of claims 1 or 2, the interface system can access a data management system that manages data related to the environment and the person's behavior, the interface system acquires second related data related to the input future prediction from the data management system, creates a future prediction message using the input recommended behavior and the acquired second related data, and provides the created future prediction message to the person. Behavior-based support system.

4. In the action-based support system according to any one of claims 1 or 2, the interface system inputs an inquiry message created by the person, interprets or analyzes the input inquiry message, and inputs provisional environmental data obtained by partially changing the environmental data according to the result of interpreting or analyzing the inquiry message into the prediction action system; the prediction action system generates a provisional future prediction using the input provisional environmental data, and generates a provisional recommended action using the generated provisional future prediction; the interface system inputs the provisional future prediction or the provisional recommended action from the prediction action system, generates a response message based on the input provisional future prediction or the provisional recommended action, and provides the generated response message to the person. An action-based support system.

5. In the action-based support system according to claim 4, the interface system interprets or analyzes the communication with the person, and generates the response message based on the provisional future prediction or the provisional recommended action according to the result of interpreting or analyzing the communication. An action-based support system.

6. In the action-based support system according to claim 4, the interface system can access a data management system that manages data related to the environment and the person's actions; the interface system obtains third related data related to the input provisional future prediction from the data management system, and creates the response message using the input provisional future prediction and the obtained third related data. An action-based support system.

7. In the action-based support system according to claim 4, the interface system can access a data management system that manages data related to the environment and the person's actions; the interface system obtains fourth related data related to the input provisional recommended action from the data management system, and creates the response message using the input provisional recommended action and the obtained fourth related data. An action-based support system.

8. In the behavior-based support system according to any one of claims 1 or 2, the interface system can access a data management system that manages data related to the environment and the person's behavior. The interface system inputs a log request created by the person, interprets or analyzes the interactive communication with the person, and according to the result of interpreting or analyzing the interactive communication, obtains fifth related data related to the input log request from the data management system, creates a log message using the obtained fifth related data, and provides the created log message to the person. A behavior-based support system.

9. In the behavior-based support system according to any one of claims 1 or 2, the prediction behavior system has a deep neural network that has pre-learned a method for generating the future prediction or the recommended behavior. The interface system can access a data management system that manages data related to the environment and the person's behavior. The interface system inputs a behavior review created by the person, obtains sixth related data related to the input behavior review from the data management system, and creates learning data for the prediction behavior system using the input behavior review and the obtained sixth related data. The prediction behavior system re-learns the deep neural network using the created learning data. A behavior-based support system.

10. In the behavior-based support system according to any one of claims 1 or 2, the interface system inputs a know-how message regarding a behavior how-to created by the person, interprets or analyzes the input know-how message, generates behavior know-how according to the result of interpreting or analyzing the know-how message, and registers the generated behavior know-how in the data management system. A behavior-based support system.

11. In the action-based support system according to any one of claims 1 and 2, the interface system can access a data management system that manages data related to the environment and the actions of the person, the data management system manages one or more action know-hows, the interface system inputs the environmental data, obtains relevant action know-hows related to the input environmental data from the data management system, obtains seventh relevant data related to the input environmental data or related to the obtained relevant action know-hows from the data management system, creates an action guide using the obtained relevant action know-hows and the seventh relevant data, and provides the created action guide to the person. An action-based support system.

12. In the action-based support system according to any one of claims 1 and 2, the interface system can access a data management system that manages data related to the environment and the actions of the person, the data management system manages one or more action know-hows, the interface system inputs an action ID request created by the person, obtains relevant action know-hows related to the input action guide request from the data management system, obtains eighth relevant data related to the input action guide request or related to the obtained relevant action know-hows from the data management system, creates an action guide using the obtained relevant action know-hows and the eighth relevant data, and provides the created action guide to the person. An action-based support system.

13. In the action-based support system according to any one of claims 1 and 2, the prediction action system has a deep neural network that has pre-learned a method for generating the future prediction or the recommended action, and the prediction action system performs reinforcement learning of the deep neural network using the input environmental data. An action-based support system.

14. In the action-based support system according to any one of claims 1 or 2, the prediction action system has a world model, and the world model includes an encoder that encodes the input environmental data into the lower-dimensional latent vector, a predictor that inputs the latent vector from the encoder and generates a future vector indicating a future image of the latent vector, and a controller that inputs the latent vector from the encoder and inputs the future vector from the predictor to generate the recommended action. An action-based support system.

15. In an action-based support system for assisting a person who acts according to the situation of the environment, it includes a storage device that stores a computer program and a processor that mechanically reads and executes the computer program. By executing the computer program, the processor communicates with the person, inputs environmental data indicating the situation of the environment, generates a future prediction indicating the future situation of the environment using the input environmental data, generates a recommended action using the generated future prediction, interprets or analyzes the communication with the person, generates a message based on the future prediction or the recommended action according to the result of interpreting or analyzing the communication, and provides the generated message to the person. An action-based support system.

16. In a program storage medium storing a machine-readable computer program, the program includes instruction codes for causing a processor to perform an action-based support method, and the action-based support method includes communicating with the person, inputting environmental data indicating the situation of the environment, generating a future prediction indicating the future situation of the environment using the input environmental data, generating a recommended action using the generated future prediction, interpreting or analyzing the communication with the person, generating a message based on the future prediction or the recommended action according to the result of interpreting or analyzing the communication, and providing the generated message to the person. A program recording medium.

17. In the action-based support system according to any one of claims 1 or 2, the prediction action system executes the generated recommended action, the action-based support system.

18. In the action-based support system according to claim 1 or 2, the prediction action system further uses logic indicating the relationship between the future prediction and the recommended action to generate the recommended action, obtains action information indicating the execution action, which is the action of the person after the message is provided, and updates the logic based on the action information, the action-based support system.

19. In the action-based support system according to claim 18, the prediction action system evaluates the action information based on the actual situation indicating the situation of the environment after the execution action, and further updates the logic based on the evaluation result, the action-based support system.

20. In the action-based support system according to claim 19, the prediction action system clusters the action information into a plurality of groups in which the future predictions are similar to each other, and for each group, updates the logic based on the action information with the highest evaluation result within the group, the action-based support system.

21. In the action-based support system according to claim 20, the logic indicates the relationship between the combination of the future prediction and the ideal state indicating the situation of the environment to be realized by the person performing the recommended action and the recommended action, and the prediction action system clusters the action information into a plurality of groups in which the combination of the future prediction and the ideal state are similar to each other, the action-based support system.

22. In the action-based support system according to claim 18, the prediction action system uses cause information related to the cause of the occurrence of the future prediction to generate supplementary information regarding the recommended action, and the interface system further inputs the supplementary information from the prediction action system and generates the message based on the input supplementary information, the action-based support system.

23. In the action-based support system according to claim 22, the cause information includes at least one of the cause, a candidate action that is a candidate for the recommended action corresponding to the cause, a reason for performing the candidate action, and the situation of the environment whose realization is predicted by performing the candidate action. An action-based support system.

24. In the action-based support system according to claim 1, the interface system is a natural language system, and the message is expressed in natural language. A plant operation support system.

25. In an action-based support system for assisting a person who acts according to the situation of the environment, a prediction action system that inputs environment data indicating the situation of the environment, generates a future prediction indicating the future situation of the environment using the input environment data, and generates a recommended action using the generated future prediction; And a control system that controls the environment according to the recommended action. A plant operation support system.

Images