Control apparatus and control method

MYPI2025005173A0Undetermined Publication Date: 2024-09-30HITACHI LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
MYPI2025005173
Authority / Receiving Office
MY · MY
Patent Type
Applications
Current Assignee / Owner
Priority Date
2023-03-30
Filing Date
2023-11-20
Publication Date
2024-09-30

AI Technical Summary

Technical Problem

Existing AI control technologies struggle to handle controlled objects whose target values change over time, as they are optimized for static target values and fail to determine appropriate operation amounts in dynamic environments.

Method used

A control device with a state determination unit, target state setting unit, state transition estimation unit, and optimal operation determination unit, utilizing a state transition model and control model to estimate and adjust operations based on changing target values, ensuring optimal transitions and operation amounts in real-time.

Benefits of technology

Enables accurate and timely determination of operation amounts for controlled objects with changing target values, improving control precision and adaptability in dynamic processes like temperature control in batch plants.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure PI2025005173_DRAWING_001
    Figure PI2025005173_DRAWING_001
  • Figure PI2025005173_DRAWING_002
    Figure PI2025005173_DRAWING_002
  • Figure PI2025005173_DRAWING_003
    Figure PI2025005173_DRAWING_003
Patent Text Reader

Abstract

State determination processing of determining a state of the control target from a measurement value of a control target, target state setting processing of using a control model including information on a second state to which a transition occurs in a case where first operation is performed on a control target in a first state to set a plurality of candidates for a target value for making the determined state of the control target closer to the second state that is a target state of the control target controlled by the target value that changes over time from the first state by performing the first operation, state transition estimation processing of estimating a state to which a transition occurs by the first operation for setting the control target to the target value for each of a plurality of the candidates based on the control model, and optimal operation determination processing of determining an operation amount of the first operation based on information regarding the estimated state to which a transition occurs by the first operation are performed. Figure 1
Need to check novelty before this filing date? Find Prior Art

Description

Control device and control method

[0001] The present invention relates to a plant control device and a plant control method.

[0002] In recent years, there has been a growing need to apply AI technology to the industrial sector, and the number of cases in which AI-based control technology is being applied is also increasing in the field of process control.

[0003] For example, Patent Document 1 discloses an example in which a future state estimation device is extended to the optimization of model-based control, the future state estimation device comprising: a model storage unit that stores a model that simulates a simulated object and its surrounding environment; a future state prediction result storage unit that stores information that estimates, in the form of a probability density distribution, the future state of the simulated object and its surrounding environment at an infinite time or time step ahead in a finite space; and a future state prediction calculation unit that performs calculations equivalent to a series using a model that simulates the future state of the simulated object and its surrounding environment in the form of a probability density distribution.

[0004] Japanese Patent Application Laid-Open No. 2019-159876

[0005] According to the above-mentioned Patent Document 1, the future state of a control object and its surrounding environment in an infinite time future can be calculated in the form of a probability density distribution, regardless of the time until the future state to be predicted, and by using this calculation result, an optimal control law can be calculated taking into account the future state in an infinite time future. Therefore, for example, in process control, if it is desired to switch from a certain state A (e.g., temperature TA) to another state B (temperature TB) early, the optimal operating conditions for transitioning from state A to state B can be found in a short time.

[0006] However, for example, in the temperature increase process of a batch plant, it is necessary to operate the plant so as to follow as closely as possible a target temperature that changes over time. The AI ​​control technology disclosed in Patent Document 1 is capable of control to quickly reach a target value, such as a certain target temperature, but it is difficult to deal with a controlled object whose target value changes from moment to moment.

[0007] An object of the present invention is to provide a technique that can determine an appropriate manipulated variable for a controlled object even when the target value of the controlled object changes over time.

[0008] The control device according to the present invention is a control device that controls a control object using a computer having a processor and a memory, and the processor is configured as a control device that performs the following operations: a state determination process that determines the state of the control object from a measurement value of the control object; a target state setting process that uses a control model including information on a second state to which the control object transitions when a first operation is performed on the control object in a first state to perform the first operation to set a plurality of candidates for the target value to bring the determined state of the control object closer to the second state, which is a target state of the control object controlled by a target value that changes over time; a state transition estimation process that estimates, based on the control model, a state to which the control object transitions by the first operation to set the control object to the target value, for each of the plurality of candidates; and an optimal operation determination process that determines the operation amount of the first operation based on information on the state to which the estimated first operation transitions.

[0009] According to the present invention, even when the target value of the controlled object changes over time, it is possible to determine an appropriate manipulated variable for the controlled object.

[0010] 1 is a diagram showing a functional configuration of a plant control system according to an embodiment of the present invention (Embodiment 1). FIG. 2 is a diagram showing an example of a schematic diagram of a computer. FIG. 3 is a diagram showing an overview of a plant to be controlled in this embodiment. FIG. 4 is a diagram showing an example of target information shown in FIG. 1. FIG. 5 is a diagram for explaining a state transition model. FIG. 6 is a diagram showing an example of data of a state transition matrix for creating a state transition model. FIG. 7 is a diagram for explaining a control model. FIG. 8 is a diagram for explaining a control model. FIG. 9 is a flowchart showing an example of a processing procedure for determining an manipulated variable in an operation phase. FIG. 10 is a diagram for explaining a method for selecting a target state. FIG. 11 is a flowchart showing an example of a selection processing procedure for selecting multiple target states in S82. FIG. 12 is a diagram for explaining estimation of state transitions when each target state is set for selected candidates. FIG. 13 is a diagram showing an example of a state transition when each target state is set for selected candidates. FIG. 14 is a diagram showing a functional configuration of a plant control system according to an embodiment of the present invention (Embodiment 2). FIG. 15 is a diagram for explaining a method of defining a state when adaptive resonance theory is used. FIG. 16 is a flowchart showing an example of the procedure of an evaluation processing for evaluating a state transition taking into account probabilistic state transitions when estimating the state transition shown in S83. FIG. 17 is an explanatory diagram for determining a state transition destination when an optimal action is taken from a current state, together with the probability, from a state transition model.

[0011] Hereinafter, embodiments of the present invention will be described with reference to the drawings. The examples are illustrative for explaining the present invention, and appropriate omissions and simplifications have been made for clarity of explanation. The present invention can be implemented in various other forms. Unless otherwise specified, each component may be singular or plural. The position, size, shape, range, etc. of each component shown in the drawings may not represent the actual position, size, shape, range, etc., in order to facilitate understanding of the invention. Therefore, the present invention is not necessarily limited to the position, size, shape, range, etc. disclosed in the drawings.

[0012] Although various types of information may be described using expressions such as "table," "list," and "queue" as examples, the various types of information may also be expressed using data structures other than these. For example, various types of information such as "XX table," "XX list," and "XX queue" may also be expressed as "XX information." When describing identification information, expressions such as "identification information," "identifier," "name," "ID," and "number" are used, but these are interchangeable.

[0013] When there are multiple components with the same or similar functions, they may be described using the same reference numeral with different subscripts. When there is no need to distinguish between these multiple components, the subscripts may be omitted.

[0014] In the embodiments, processing performed by executing a program may be described. Here, a computer executes the program using a processor (e.g., a CPU or a GPU) and performs processing defined by the program using storage resources (e.g., memory) and interface devices (e.g., communication ports). Therefore, the entity performing the processing by executing the program may be the processor. Similarly, the entity performing the processing by executing the program may be a controller, device, system, computer, or node having a processor. The entity performing the processing by executing the program may be any computing unit, and may include a dedicated circuit that performs specific processing. Here, the dedicated circuit may be, for example, an FPGA (Field Programmable Gate Array), an ASIC (Application Specific Integrated Circuit), or a CPLD (Complex Programmable Logic Device).

[0015] A program may be installed on a computer from a program source. The program source may be, for example, a program distribution server or a computer-readable storage medium. When the program source is a program distribution server, the program distribution server may include a processor and storage resources for storing the program to be distributed, and the processor of the program distribution server may distribute the program to be distributed to other computers. In addition, in the embodiments, two or more programs may be realized as one program, or one program may be realized as two or more programs.

[0016] FIG. 1 is a diagram showing the functional configuration of a plant control system 1000 according to one embodiment of the present invention. The plant control system 1000 according to this embodiment is configured to include a plant 1, target information 2, and a control device 3. The plant 1 is an object to be controlled, and in this embodiment, it is a batch plant. The target information 2 is information indicating a target state of the plant 1, including target values ​​that change over time, such as the temperature of a reaction vessel in the plant 1. The control device 3 is a device that determines an optimal manipulated variable for the plant 1 based on the measurement values ​​measured in the plant 1 and the target information 2.

[0017] The control device 3 has a state determination unit 31, a target state setting unit 32, a state transition estimation unit 33, a model 34, and an optimal operation determination unit 35. The state determination unit 31 determines the current state of the plant 1 from measured values ​​of the plant 1. The state is defined separately by values ​​of state quantities such as temperature and pressure. The target state setting unit 32 sets a target state, which is a state that is a target for control of the plant 1, from the plant state determined by the state determination unit 31, target information 2, and a model 34 that can simulate the characteristics of the plant 1.

[0018] The model 34 is a model created from operational data stored in the plant 1 and includes a state transition model 341 and a control model 342. The state transition model 341 is a model that defines, using a probability matrix, the states to which a certain action will lead from a certain state. The control model 342 is a model that defines the optimal action for finally reaching a certain state B from a certain state A. The state transition estimation unit 33 uses the model 34 to estimate the optimal transition method from the current state to the target state set by the target state setting unit 32. The optimal operation determination unit 35 determines the manipulated variable to achieve the optimal transition based on the results obtained by the state transition estimation unit 33. The manipulated variable is, for example, an MV value such as a valve opening or an SV value (set value) for PID control. The operational data is, for example, data including measurement values ​​accumulated in a time series obtained during plant operation, such as manipulated variables for the plant and plant temperatures.

[0019] The control device 3 shown in Figure 1 can be realized, for example, by a general computer 1600, as shown in Figure 2 (schematic computer diagram), which includes a CPU 1601, memory 1602, an external storage device 1603 such as an HDD (Hard Disk Drive), a reading / writing device 1607 that reads and writes information from a portable storage medium 1608 such as a CD (Compact Disk) or USB memory, an input device 1606 that accepts input of various information such as a keyboard and a mouse, an output device 1605 such as a display that outputs various information that has been input and is used for processing, a communication device 1604 such as a NIC (Network Interface Card) for connecting to a communication network, and an internal communication line (referred to as a system bus) 1609 such as a system bus that connects these.

[0020] Furthermore, various data (e.g., model 34) stored in the control device 3 or used for processing can be realized by the CPU 1601 reading and using the data from the memory 1602 or the external storage device 1603. Furthermore, each unit of the control device 3 (e.g., state identification unit 31, target state setting unit 32, state transition estimation unit 33, and optimal operation determination unit 35) can be realized by the CPU 1601 loading a predetermined program stored in the external storage device 1603 into the memory 1602 and executing it.

[0021] The above-mentioned predetermined programs and data may be stored (downloaded) into the external storage device 1603 from the storage medium 1608 via the reading / writing device 1607 or from the network via the communication device 1604, and then loaded onto the memory 1602 and executed by the CPU 1601. Alternatively, the programs and data may be directly loaded onto the memory 1602 from the storage medium 1608 via the reading / writing device 1607 or from the network via the communication device 1604, and then executed by the CPU 1601.

[0022] In the following, we will explain an example in which the control device 3 is configured by a single computer, but all or part of these functions may be distributed across one or more computers, such as a cloud, and similar functions may be realized by communicating with each other via a network.

[0023] Examples of the present invention will be described in detail below.

[0024] FIG. 3 is a diagram showing an overview of the plant 1 to be controlled in this example. The plant 1 is a batch plant that produces polymers by polymerization reaction. Its main components are a reaction vessel 11, a jacket 12, an agitator 13, a pump 14, a valve 15, and a controller 16. Monomers, which are raw materials, are charged into the reaction vessel 11 together with a solvent, and a polymerization reaction of the monomers is initiated by an initiator. During operation, the charged monomers and other materials are agitated by the agitator 13 so as to be as uniform as possible. In addition, temperature-adjusted water is sent by a pump 14 to a temperature-adjusting jacket 12 attached to the reaction vessel 11, and the temperature Tr measured in the reaction vessel is controlled to a target temperature.

[0025] Next, a method for adjusting the temperature Tr of the reaction vessel will be described in detail. In this embodiment, the temperature Tr in the reaction vessel is controlled by changing the inlet temperature Tc of the jacket 12. Specifically, the measured temperature Tr is input to the controller 16a, which compares the temperature Tr with a target value SV1 for the temperature Tr and provides a set value (SV2) for the jacket inlet temperature Tc to the controller 16b. For example, if the temperature Tr is higher than the target value SV1, the controller 16a adjusts the set value SV2 of the jacket inlet temperature Tc downward, and if the temperature Tr is lower than the target value SV1, the controller 16a adjusts the set value SV2 of the jacket inlet temperature Tc upward. The controller 16b compares the set value SV2 of the jacket inlet temperature Tc with the actual value of the jacket inlet temperature Tc measured by a temperature sensor and adjusts the openings of the valves 15a and 15b using the manipulated variable MV so that the actual value of the jacket inlet temperature Tc approaches the set value SV2 of the jacket inlet temperature Tc.

[0026] 4 is a diagram showing an example of the target information shown in FIG. 1. In this embodiment, the target is to raise the temperature Tr1 of the reaction vessel to a temperature Tr2 between time t1 and time t2, and then to control the temperature Tr2 at a constant value. Therefore, the temporal change in the target value of the temperature Tr measured in the reaction vessel is stored as target information 2.

[0027] As described above, in this plant, temperature control is performed by the controllers 16a and 16b even without the control device 3, but by adding the control device 3, temperature can be controlled more accurately.

[0028] Next, the control device 3 will be described. The control device 3 inputs the temperature Tr of the reaction vessel and the target value SV1 of the temperature Tr set as target information, and provides a target value SV1' to the controller 16a. In this case, the target value SV1 is not provided to the controller 16a.

[0029] As described above, the model 34 is created from the operation data of the plant 1. Therefore, the present control device 3 has a learning phase in which the model 34 is created from the operation data, and an operation phase in which the created model 34 is utilized to control the plant 1. First, a method by which the control device 3 creates the model 34 in the learning phase will be described.

[0030] The model 34 has a state transition model 341 and a control model 342. The state transition model is a model that defines, with probability, the state to which a certain action is taken from a certain state. Therefore, in order to create a state transition model, it is first necessary to define three things: the state, the action, and the transition time.

[0031] A state, for example, represents the current state of a controlled object. In other words, a state represents the current state or behavior of a controlled object. For example, the temperature Tr of a reactor at a certain point in time is an example of the state. A target state represents the state of a controlled object controlled by target information. For example, the temperature Tr of a reactor in plant 1 reaches a target value SV1 or SV1'. In this example, as an example, the state is expressed as a combination of the reactor temperature Tr and its change amount dTr. Specifically, as shown in FIG. 5 , 400 states were defined, each divided into 20 parts by the boundaries of the minimum value Tr_min and maximum value Tr_max of the reactor temperature Tr, and the minimum value dTr_min and maximum value dTr_max of the change amount dTr of the reactor temperature Tr. As an example, the behavior is the transition of the reactor temperature Tr to a set value discretized in 1°C increments, with a transition time of 10 seconds.

[0032] 6 shows a portion of an example of state transition matrix data for creating a state transition model. The example in FIG. 6 shows that when action a1 is taken from state s190, there is a 20% probability of transitioning to state s211 after 10 seconds, a 70% probability of transitioning to state s212, and a 10% probability of transitioning to state s213. Furthermore, when action a2 is taken from the same state s190, there is a 55% probability of transitioning to state s233 and a 45% probability of transitioning to state s234. This state transition model can be obtained by discretizing past operational data into "states" and then statistically processing the relationship between the action (the set value of the reactor temperature Tr) and the state after 10 seconds, which is the transition time.

[0033] Next, the control model will be described. The control model is a model that represents the optimal action to reach the final goal state (denoted as Sg) in the shortest time from a certain state. The control model can be obtained from a state transition model. In this embodiment, the control model is obtained by applying a reinforcement learning algorithm, as an example. That is, if taking a certain action from a certain state makes it easier to approach the final goal state, a large reward is given. The control model was determined by determining the optimal action in this way. Note that the optimal route for transitioning from a certain state to the final goal state may also be obtained by dynamic programming.

[0034] Some examples of the control model are shown in FIGS. 7A and 7B . FIG. 7A shows an example of a database constituting the control model. In FIG. 7A , when a certain target state is set, information on a second state to which a control target in a certain first state transitions when a first operation is performed is stored. For example, the example shown in the first row of FIG. 7A indicates that taking action a2 is the optimal action for state s190, resulting in a transition to state s233. The state transition model of FIG. 6 represents multiple actions that have been taken in the past for a certain state and the transition probabilities when those actions are taken. However, the control model represents only the optimal action for ultimately transitioning to the target state. As described above, in the learning phase, a state transition model is created from past operating data, and a control model is created from the state transition model. The control model may also store information required for transitioning from a certain initial state to a final target state for each combination of the initial state and the final state.

[0035] Fig. 7B shows an image of the control model when transitioning from state s190, which is the initial state, to state s355, which is the final goal state. Using information in the database that constitutes the control model shown in Fig. 7A, one or more actions, i.e., operations, required for transitioning from state s190, which is the initial state, to state s355, which is the final goal state, are uniquely identified, as shown in Fig. 7B.

[0036] Next, the operation phase will be described. In the operation phase, the manipulated variable is determined by the processing flow shown in Fig. 8. Fig. 8 is a flowchart showing an example of the processing procedure for determining the manipulated variable in the operation phase.

[0037] First, the state determination unit 31 of the control device 3 determines and determines the current state of the reactor by converting the temperature Tr of the reactor measured in the plant 1 into a state number in accordance with the state definition shown in Figure 5 (S81).

[0038] In this embodiment, the control state determination unit 31 converted the temperature Tr of the reaction vessel into a state number defined by the state definition. However, as a modification of this process, the control state determination unit 31 may perform the following process. That is, the control state determination unit 31 may determine whether information about the state determined from the measurement value of the temperature Tr of the reaction vessel measured in the plant 1 is included in the state definition, and if it determines that the information about the state is not included in the state definition, it may convert the temperature Tr of the reaction vessel into the state number of the state closest to the determined state, among the states included in the control model shown in Figures 7A and 7B. In this way, the temperature Tr of the reaction vessel can be converted into a state number even when the state is not defined.

[0039] Next, the target state setting unit 32 selects multiple target states on a target curve TC that represents the temporal progression of the target value defined as target information (S82). A specific method will be described with reference to FIG. 9. FIG. 9 is a diagram illustrating a method for selecting a target state when, at the current time tc, the temperature of the reactor deviates from the target value Trc' and becomes temperature Trc. In this case, multiple candidate points A, B, C, D, etc., indicating times after time tc, become target values. The number of these candidate points can be determined depending on the characteristics, environment, etc. of the plant.

[0040] The target state setting unit 32 selects an appropriate candidate from these candidates. For example, if the target state setting unit 32 sets a target state indicated by point A at the current time as a target value, the control model takes action to approach the target state of point A. However, by the time the target state indicated by point A transitions, the target state at the time of the transition has also changed, and the deviation from the target state is not eliminated. On the other hand, if the target state setting unit 32 selects a target value that is a certain distance from the current time (for example, a target state at point D), there is a possibility that the target state of point D will be reached earlier than the time that the target state of point D should be reached. This is because, in this control model, the target state setting unit 32 takes action to reach the set target state more quickly. Therefore, the target state setting unit 32 needs to appropriately select the target state itself.

[0041] In this embodiment, the target state setting unit 32 calculates a gradient K, which is the rate of change from the temperature at the current time to the temperature in the target state, and selects a candidate target state from the gradient K and the maximum rate of change of the state of the controlled object, thereby making it possible to select a candidate target state taking into consideration the characteristics and environment of the plant. Specifically, the gradient K is calculated in the following steps shown in FIG. 10 .

[0042] FIG. 10 is a flowchart illustrating an example of the procedure for the selection process for selecting multiple target states in S82. As shown in FIG. 10, the target state setting unit 32 first obtains the maximum temperature change rate from the current state as a base point from the state transition model (S101). As shown in FIG. 9, the maximum temperature change rate can be expressed as the slope Kmax at which the temperature change is maximum when the target state is reached after a predetermined transition time has elapsed from the current state. Here, the state transition model described in FIG. 6 is used. However, instead of using such modeled data, past driving data may be discretized into states, and data obtained by statistically processing the relationship between actions and states after the transition time may be directly used.

[0043] Next, the target state setting unit 32 calculates a slope K at a time corresponding to the transition time from the current time, and determines candidates for the target value in the reference target state (S102). Specifically, the target state setting unit 32 calculates a slope K to the target value in each target state, starting from the target value in the target state at a time further in transition time from the current time (i.e., a time further in time from the current time) to the target value in the target state at a time closer in transition time from the current time (i.e., a time closer in time to the current time), and sets the target value that is closest to the rate of change calculated in S101 as a candidate for the target value in the reference target state. In Fig. 9, the target state setting unit 32 calculates the slopes in the following order: slope K (KD) between point P at the current time and point D that is farthest from the current time; slope K (KC) between point P at the current time and point C that is next farthest from the current time; slope K (KB) between point P at the current time and point B that is further farthest from the current time; and slope K (KA) between point P at the current time and point A that is closest to the current time. Then, the target state setting unit 32 sets the point having the closest slope to the slope determined in S101 as a candidate for the target value in the target state. In Fig. 9, for example, the target state setting unit 32 sets point B as a candidate for the target value in the target state.

[0044] Furthermore, the target state setting unit 32 uses the time of the candidate target values ​​in the target state obtained in S102 as a reference, and selects n1 target values ​​in the target state at times before the reference time and n2 target values ​​in the target state at times after the reference time (S103). For example, the target state setting unit 32 selects point A before point B obtained in S102 and points C and D after point B. In this example, n1 = 1 and n2 = 2. The reason for selecting these points other than the reference point is to take into account errors in the measurement values. The reason for selecting more points at times after the reference time than points before the reference time is because there is a history of operation and there is a high possibility that operation will be appropriate.

[0045] By selecting in this manner, it is possible to use the case with the fastest rate of change in the past as a reference, set a first candidate for the target state that is closest to the reference (for example, point B), and select target states around the first candidate (for example, points A, C, and D). Also, by selecting more candidates at times after the current time than at times before the current time, it is possible to select highly stable candidates that take past operating performance into consideration.

[0046] Returning to FIG. 8 , the state transition estimation unit 33 then estimates state transitions when each target state is set for the selected candidate (S83). In this example, the state transition estimation unit 33 estimates state transitions using the control model of the model 43. Specifically, as shown in FIGS. 11 and 12 , the state transition estimation unit 33 determines, from the control model, an optimal action ac1 for a current state (sc) when the target state is sg1, and a state (sc2) resulting from the action ac1. Next, the state transition estimation unit 33 sequentially determines an optimal action ac2 that results in the target state sg1 for the state sc2, and a state sc3 resulting from the action ac2, thereby determining a route R1 to the target state sg1. The state transition estimation unit 33 also performs this route determination for the other target state candidates sg2, sg3, and sg4, and determines the respective routes.

[0047] Next, the optimal operation determiner 35 selects an optimal route from the routes calculated for each target state candidate by the state transition estimator 33, and determines the manipulated variable (S84). Specifically, the optimal operation determiner 35 calculates, for each candidate target state, the deviation between the route calculated for each target state candidate by the state transition estimator 33 and a target curve TC, which represents the temporal transition of a target value defined as target information. The optimal operation determiner 35 selects the route of the target state candidate with the smallest deviation as the optimal route, and determines the selected route as the optimal operation for controlling the controlled object with the optimal manipulated variable to be applied in the current state. The optimal operation determiner 35 performs this process of determining the optimal operation for each control period represented by the transition time (e.g., every 10 seconds). This allows the optimal manipulated variable to be determined for each control period, even if actual data deviates from the target.

[0048] In this embodiment, the output of the control device 3 is SV1' which is input to the controller 16a, but it may also be SV2' which is input to the controller 16b or an opening command value for the valve 15.

[0049] Next, a second embodiment of the present invention will be described. The configuration of the second embodiment of the present invention is shown in Fig. 13. The second embodiment differs from the first embodiment in that the control device 3 in the plant control system 2000 according to this embodiment has a state determination unit 31b different from that in the first embodiment and a state transition estimation unit 33b different from that in the first embodiment. The following mainly describes the differences from the first embodiment.

[0050] The state determination unit 31b performs state determination using a data clustering technique. In this embodiment, adaptive resonance theory is used as an example of a data clustering technique. FIG. 14 illustrates a method for defining states using adaptive resonance theory. Similar to FIG. 5, FIG. 14 plots past operating data as black circles, with the horizontal axis representing the reactor temperature Tr and the vertical axis representing the change in reactor temperature Tr, dTr. As shown in FIG. 14, in the process of increasing the reactor temperature Tr from near the minimum value Tr_min of the reactor temperature Tr to near the maximum value Tr_max of the reactor temperature Tr, when the change in temperature Tr dTr is small, only operating data near the minimum value Tr_min or the minimum value Tr_max of the temperature Tr is available. Therefore, in the region between these values, the change in temperature Tr dTr is relatively large, resulting in a region RN where no operating data exists. Using adaptive resonance theory, the state determination unit 31b generates categories that classify the operating data into certain groups, as indicated by the dashed circle CC. Therefore, a category is not generated in an area where there is no driving data. Therefore, the state determination unit 31b defines categories as states, so that it is possible to define as many states as necessary.

[0051] The state transition estimation unit 33b estimates state transitions using a state transition model such as that described with reference to FIG. 6 in addition to the control model. In the control model, a certain action results in a transition to one state, but in reality, a transition to another state occurs probabilistically. Since information on this probabilistic state transition is included in the state transition model, in the second embodiment, the state transition is evaluated taking into account the probabilistic state transition when estimating the state transition shown in S83 of FIG. 8. In this embodiment, the evaluation was performed using the following steps.

[0052] 15 is a flowchart showing an example of the procedure of the evaluation process for evaluating the state transition in consideration of the probabilistic state transition when estimating the state transition shown in S83. As shown in Fig. 15, the state transition estimation unit 33b first obtains, from the control model, an optimal action ac1 for transitioning from the current state sc to the final target state sg1 (S151).

[0053] Next, the state transition estimation unit 33b calculates, from the state transition model, the state transition destination when the above-mentioned optimal action ac1 is taken from the current state sc, along with the probability (S152). For example, the state transition estimation unit 33b calculates states sc2 and sc4, which are state transition destinations when action ac1 is taken in sc shown in Figure 16, and their respective probabilities. As a result, states s211, s212, and s213, which are state transition destinations when action a1 is taken when the current state is state s190 as shown in Figure 6, and their probabilities of 20%, 70%, and 10% are calculated.

[0054] Next, the state transition estimation unit 33b calculates an optimal action from the control model for the state calculated in S152 (S153). For example, the state transition estimation unit 33b calculates an action ac1 that becomes the action a1 when the state transitions to state sc2, which is the state after the state transition with the highest probability calculated in S152 (s212, 70%).

[0055] Furthermore, the state transition estimation unit 33b repeats the same process as S153 for the state transition destinations (e.g., states sc2, sc3, etc.) leading up to the goal state sg1 (S154). That is, the state transition estimation unit 33b determines whether the goal state sg1 has been reached by the optimal action calculated from the control model in S153. If the state transition estimation unit 33b determines that the goal state sg1 has not been reached by the optimal action (S154; No), the process returns to S152 and repeats the subsequent processes. On the other hand, if the state transition estimation unit 33b determines that the goal state sg1 has been reached by the optimal action (S154; Yes), the process proceeds to S155.

[0056] Finally, the state transition estimation unit 33b calculates the deviation Ei between the transition probability p and the target curve for each of the i routes Ri leading to the target state sg1 obtained by the processing up to S154, calculates the deviation when the target state sg1 is set as the target value as Σp*Ei, and selects the optimal route from each route (S155).

[0057] In this way, by calculating the deviation for all possible routes, it becomes possible to evaluate state transitions more accurately, and the optimal route can be selected from among the routes with the highest average accuracy. In the above example, all routes Ri are evaluated, but routes with a certain degree of low probability may be excluded from the evaluation. This makes it possible to obtain the above-mentioned effects while reducing the processing load.

[0058] As described above, by determining the manipulated variable for the controlled object as in this control device, it is possible to determine an appropriate manipulated variable even in a process in which the target value changes over time.

[0059] Specifically, as described in the first embodiment, FIGS. 1 and 8, etc., in the control device 3 that controls a control target (for example, a plant 1) by a computer 1600 having a processor (CPU 1602) and a memory (memory 1602), The processor performs a state determination process (processing of the state determination unit 31) that determines the state of the control object from the measured value of the control object; a target state setting process (processing of the target state setting unit 32) that uses a control model 342 including information on a second state (temperature Tr at a later time, state s233) to which the control object transitions when a first operation (a certain action a2) is performed on the control object in a first state (temperature Tr at the current time, state s190); the target state setting process (processing of the target state setting unit 32) that sets a plurality of candidates for the target value for performing the first operation to bring the determined state of the control object closer to the second state, which is a target state of the control object controlled by a target value (target information 2) that changes over time; a state transition estimation process (processing of the state transition estimation unit 33) that estimates, based on the control model, a state to which the control object transitions by the first operation to set the control object to the target value (for example, a route R1 to the target state sg1), for each of the plurality of candidates; and an optimal operation determination process (processing of the optimal operation determination processing unit 35) that determines the amount of operation of the first operation based on information on the state to which the control object transitions due to the estimated first operation. This makes it possible to determine an appropriate manipulated variable for the controlled object even when the target value of the controlled object changes over time, and to appropriately control the plant based on the determined manipulated variable.

[0060] 5, 6, 14, etc. as the state transition models of Examples 1 and 2, in the state transition estimation process, the processor estimates, for each of the plurality of candidates, information on the operation amount of the first operation for setting the control object to the target value and information on the transition state when an operation based on the operation amount of the first operation is performed (e.g., state s211 after taking action a1 in state 190) based on a state transition model 341 including information on one or more states to which the control object may transition when the first operation is performed on the control object in the first state and the probability of transition to each of the one or more states of the control object, and the control model 342, and then determines the first operation based on the estimated information in the optimal operation determination process. This makes it possible to determine the operation amount for the control object in consideration of the transition probability.

[0061] 8, when there is no information on the state determined from the measurement value of the controlled object in the control state determination process, the processor determines the state closest to the determined state among the states included in the control model as the state of the controlled object. This makes it possible to determine the state to transition to even when the state is not defined.

[0062] 14 and the like in the second embodiment, the processor classifies the state of the controlled object into a plurality of categories by predetermined data clustering (e.g., clustering based on adaptive resonance theory) in the state determination process, and estimates a state to be transitioned by the first operation using the control model and the state transition model obtained by the classification in the state transition estimation process. This defines a state excluding a region where no operational data exists, making it possible to estimate a state to be transitioned while reducing the processing load.

[0063] 9 and the like, in the target state setting process, the processor selects, as the candidate target value, the target value at which the maximum rate of change and the rate of change satisfy a predetermined relationship (closest gradient) based on the maximum rate of change of the state of the controlled object when the target state is reached after a predetermined transition time has elapsed from the current state (e.g., gradient Kmax at which the temperature change when the target state is reached) and the rate of change between the current state and the candidate target value (e.g., gradient K at a time corresponding to the transition time from the current time). This makes it possible to select the candidate target state taking into account the characteristics of the plant.

[0064] 9 and the like, in the target state setting process, the processor selects more candidates for the target value at a time point after the current time point than at a time point before the current time point, thereby making it possible to appropriately select candidates for the target state with high stability in consideration of past operating results.

[0065] The present invention is not limited to the above-described embodiments, and in the implementation stage, the components can be modified and embodied within the scope of the gist of the present invention, or multiple components disclosed in the above-described embodiments can be appropriately combined. For example, in the above-described embodiment, multiple target values ​​for the target state are selected, and one is chosen from the selected target states. However, the target value for the selected target state may be one.

[0066] REFERENCE SIGNS LIST 1 Plant 2 Target information 3 Control device 31, 31b State determination unit 32 Target state setting unit 33, 33b State transition estimation unit 34 Model 341 State transition model 342 Control model 35 Optimal operation determination unit

Claims

1. A control device that controls a control object using a computer having a processor and a memory, wherein the processor performs the following operations: a state determination process that determines the state of the control object from a measurement value of the control object; a target state setting process that uses a control model including information of a second state to which the control object transitions when a first operation is performed on a control object in a first state to set a plurality of candidates for the target value for performing the first operation to bring the determined state of the control object closer to the second state, which is a target state of the control object controlled by a target value that changes over time; a state transition estimation process that estimates, based on the control model, the state to which the control object transitions by the first operation to set the control object to the target value, for each of the plurality of candidates; and an optimal operation determination process that determines the amount of operation of the first operation based on information regarding the state to which the estimated first operation transitions.

2. A control device as described in claim 1, wherein the processor, in the state transition estimation process, estimates, for each of a plurality of candidates, information on the transition state when an operation based on the operation amount of the first operation is performed for the controlled object to set the controlled object to the target value and information on the transition state when an operation based on the operation amount of the first operation is performed, based on a state transition model including information on one or more states of the controlled object to which the controlled object in the first state may transition when the first operation is performed and information on the probability of transitioning to each of the states of the one or more controlled objects, and the control model; and, in the optimal operation determination process, determines the first operation based on the estimated information.

3. A control device as described in claim 1, characterized in that, when there is no information regarding the state determined from the measurement value of the controlled object in the control state determination process, the processor determines the state closest to the determined state among the states included in the control model as the state of the controlled object.

4. A control device as described in claim 2, characterized in that, in the state discrimination process, the processor classifies the state of the controlled object into a plurality of categories by a predetermined data clustering, and, in the state transition estimation process, estimates the state to which the transition will occur due to the first operation using the control model and the state transition model obtained by the classification.

5. A control device as described in claim 2, characterized in that, in the target state setting process, the processor selects, based on the maximum rate of change of the state of the controlled object when a predetermined transition time has elapsed from the current state to the target state and the rate of change between the current state and the candidate target value, the target value where the maximum rate of change and the rate of change satisfy a predetermined relationship.

6. A control device according to claim 5, characterized in that, in the target state setting process, the processor selects more candidates for the target value at a time point after the current time than at a time point before the current time.

7. A control method for controlling a control object using a computer having a processor and a memory, comprising: determining a state of the control object from a measurement value of the control object; using a control model including information of a second state to which the control object transitions when a first operation is performed on the control object in a first state, performing the first operation to change the determined state of the control object from the first state to the second state which is a target state of the control object controlled by a target value that changes over time, setting multiple candidates for the target value for the first operation to approximate the determined state of the control object to the second state which is a target state of the control object controlled by a target value that changes over time, based on the control model, estimating, for each of the multiple candidates, the state to which the control object transitions due to the first operation to set the control object to the target value, and determining the amount of operation of the first operation based on information regarding the estimated state to which the control object transitions due to the first operation.

8. A control method as described in claim 7, characterized in that, in the estimation, based on a state transition model including information on one or more states of the control object to which the control object in the first state may transition when the first operation is performed on the control object and the probability of transitioning to each of the one or more states of the control object, and based on the control model, the operation amount of the first operation for setting the control object to the target value and information on the transition state when an operation based on the operation amount of the first operation is performed are estimated for each of the multiple candidates, and, in the determination, the first operation is determined based on the estimated information.

9. A control method as claimed in claim 7, characterized in that, if there is no information regarding the state determined from the measurement value of the controlled object in said determination, a state closest to the determined state among the states included in said control model is determined as the state of the controlled object.

10. A control method according to claim 8, characterized in that, in the discrimination, the state of the controlled object is classified into a plurality of categories by a predetermined data clustering, and, in the estimation, the state to which the transition occurs due to the first operation is estimated using the control model and the state transition model obtained by the classification.

11. A control method as described in claim 8, characterized in that, in the setting, the target value at which the maximum rate of change in the state of the controlled object when a predetermined transition time has elapsed from the current state to the target state and the rate of change between the current state and the candidate target value satisfy a predetermined relationship between the maximum rate of change and the rate of change are selected as the candidate target value.

12. A control method according to claim 11, characterized in that in said setting, more candidates for the target value at a time point after the current time point are selected than at a time point before the current time point.