Intelligent control device, intelligent control method, and intelligent control program

Abstract

To enable sufficiency of learning data related to control performance to be evaluated while identifying a shortage range of the learning data related to the control performance.SOLUTION: An intelligent control device 100 comprises: a model generating unit 147 for generating a control object model that simulates a behavior of a control object by learning learning data obtained by an operation of the control object; a range estimating unit 144 for estimating a range of a condition of the control object for which there is insufficient learning data by using the control object model; a transition probability calculating unit 145 for calculating a transition probability for the control object model to transition from an initial condition to a goal condition; and an output control unit 142 for outputting the range estimated by the range estimating unit 144 and the transition probability calculated by the transition probability calculating unit 145.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an intelligent control device, an intelligent control method and an intelligent control program for evaluating data sufficiency of artificial intelligence technology for controlling an industrial plant. [Background technology]

[0002] In the fields of plants (power generation and industry), automobiles, and distribution, artificial intelligence (AI) technologies, including machine learning, are being used. In particular, in industrial plants, AI technologies are used to optimize control rules from operational data, improving flexible operation performance such as high-speed load changes and fast startup in addition to improving operational efficiency. The control performance of these AI technologies for industrial plants is strongly dependent on the sufficiency of operational data.

[0003] Therefore, in conventional artificial intelligence technology, a simulator of a controlled object is developed to evaluate data sufficiency and control performance. However, there are problems in that the development time and development cost of a controlled object is long. Furthermore, even when attempting to improve control performance by fine-tuning the parameters of artificial intelligence technology, the calculation time is long in the simulator, and adjustments may take several days to several weeks. Therefore, the following devices and methods have been developed to evaluate the sufficiency of operating data and control performance from the control rules calculated by artificial intelligence technology.

[0004] Patent Document 1 discloses a technology that defines a space in which the process value signals used for control are discretized, and evaluates the reliability of AI control based on the maximum and minimum values ​​of the learning data in each space and the number of data. If the data transitions to a range with low reliability, the safety of the control is guaranteed by switching to conventional control.

[0005] Patent Document 2 discloses a technology in which, when movement history data is supplied as learning data, it is determined whether the movement history data is for a known route or an unknown route, and if it is known, the parameters of an existing model are updated, and if it is unknown, a new model is generated and an updated model is generated by combining it with the existing model. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Patent Publication No. 2022-093878 [Patent Document 2] JP 2012-008659 A Summary of the Invention [Problem to be solved by the invention]

[0007] The device and method of Patent Document 1 disclose means for evaluating the reliability of AI control based on the maximum value, minimum value, number of data, and data density of learning data for each space. These methods are useful for identifying spaces with insufficient data, and are excellent for identifying the range of data replenishment. However, there are many cases where the range related to control performance is only a small part of the entire area of ​​the defined space. For example, 90% or more of the area of ​​the defined space is a transition prohibited space, and even if data for that space does not exist, it may not have a significant impact on control performance. For this reason, Patent Document 1 does not explicitly state a means for evaluating the sufficiency of data related to control performance.

[0008] The device and method of Patent Document 2 discloses a means for improving the learning efficiency of artificial intelligence technology when new data is added by determining whether the added data is known data or unknown data. However, like Patent Document 1, it does not clearly state a means for supplementing with data suitable for improving control performance.

[0009] Therefore, an object of the present invention is to enable evaluation of the sufficiency of learning data related to control performance while identifying the range of insufficiency of learning data related to control performance. [Means for solving the problem]

[0010] In order to solve the above-mentioned problems, the intelligent control device of the present invention is characterized by comprising: a model generation unit that learns learning data obtained by operating a controlled object and generates a controlled object model that simulates the behavior of the controlled object; a range estimation unit that uses the controlled object model to estimate a range of the state of the controlled object for which learning data is lacking; a transition probability calculation unit that calculates a transition probability of the controlled object model transitioning from an initial state to a target state; and an output control unit that outputs the range estimated by the range estimation unit and the transition probability calculated by the transition probability calculation unit.

[0011] The intelligent control method of the present invention is characterized by comprising the steps of: a model generation unit learning learning data obtained by operating a controlled object and generating a controlled object model that simulates the behavior of the controlled object; a range estimation unit using the controlled object model to estimate a range of the state of the controlled object for which learning data is lacking; a transition probability calculation unit calculating a transition probability for the controlled object model to transition from an initial state to a target state; and an output control unit outputting the range estimated by the range estimation unit and the transition probability calculated by the transition probability calculation unit.

[0012] The intelligent control program of the present invention is intended to execute the steps of: learning learning data obtained by operating a controlled object, and generating a controlled object model that simulates the behavior of the controlled object; using the controlled object model to estimate the range of the state of the controlled object for which learning data is lacking; calculating the transition probability for the controlled object model to transition from an initial state to a target state; and outputting the range of the state of the controlled object for which learning data is lacking, and the transition probability. Other means will be described in the description of the embodiment of the invention. Effect of the Invention

[0013] According to the present invention, it is possible to evaluate the sufficiency of learning data related to control performance while identifying the range of insufficiency of learning data related to control performance. [Brief description of the drawings]

[0014] [Figure 1] 1 is a diagram illustrating the configuration of an intelligent control device according to a first embodiment; [Diagram 2] FIG. 2 is a diagram showing a configuration of a control target. [Diagram 3] 1 is a graph showing steam temperature versus elapsed time. [Figure 4] FIG. 11 is a diagram showing the correspondence between states and amounts of operation. [Diagram 5] FIG. 13 is a diagram showing the correspondence between state, temperature, and rate of change. [Figure 6] FIG. 4 is a diagram showing the correspondence between an operation amount and a valve opening adjustment amount. [Figure 7] 1 is a flowchart of data processing. [Figure 8] FIG. 4 is a mode transition diagram relating to control. [Figure 9] FIG. 11 is a diagram showing the minimum number of control steps in each state according to the second embodiment. [Figure 10] FIG. 13 is a diagram showing the standard deviation of the minimum number of control steps for each state. [Figure 11] FIG. 13 is a diagram showing the standard deviation of the minimum number of control steps for each state when learning data is missing. [Figure 12] FIG. 13 is a diagram showing the minimum number of control steps for each state. [Figure 13] FIG. 13 is a diagram illustrating a configuration of a data processing device according to a third embodiment. [Figure 14] 1 is a flowchart of data processing. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0015] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. First Embodiment 1 is a block diagram illustrating an example of an intelligent controlling device 100 according to a first embodiment of the present invention. The intelligent controlling device 100 is a computer including an input device 110, a data reading device 116, a temperature sensor 209, an output device 120, a storage device 130, and a computing device 140 as main components.

[0016] Of these, the input device 110 is a section that accepts instructions from an operator, and is composed of buttons, a touch panel, and the like. The data reading device 116 is a part that receives data from outside the intelligent controlling device 100, and is composed of an optical drive, a USB (Universal Serial Bus) terminal, a LAN (Local Area Network) cable terminal, a communication device, and so on. The temperature sensor 209 is a sensor that measures the temperature of the controlled object. However, it is not limited to this, and a pressure sensor, a humidity sensor, a flow rate sensor, or the like that measures the controlled object may be provided, and is not limited thereto.

[0017] The output device 120 is a device that outputs instruction information, scanned images, scanning results, etc. to an operator, and is composed of a display and a communication device. The above configurations are standard, and any or all of the input device 110, data reading device 116, and output device 120 may be connected externally to the intelligent controlling device 100.

[0018] The storage device 130 is a part for storing various data, and includes an operation data storage unit 131, a model storage unit 132, a control law storage unit 133, a data shortage range storage unit 134, and an intelligent control program 135. The operation data storage unit 131 is a part for storing data related to the behavior of the controlled object of the intelligent control device 100 inputted from the data reading device 116. The model storage unit 132 is a part for storing a model that simulates the behavior of the controlled object of the intelligent control device 100. The control law storage unit 133 is a part for storing the calculation result of the operation optimization unit 143 described later. The data shortage range storage unit 134 is a part for storing the calculation result of the range estimation unit 144 described later. The intelligent control program 135 is executed by the arithmetic device 140 to embody various processing units, such as an input control unit 141, an output control unit 142, an operation optimization unit 143, a range estimation unit 144, a transition probability calculation unit 145, a state division number adjustment unit 146, a model generation unit 147, a transition calculation unit 148, a learning data estimation unit 149, and an operation amount determination unit 150. The storage device 130 is formed of, for example, a hard disk or a semiconductor disk. The details of the storage device 130 will be described later, and only the general function will be described here.

[0019] The arithmetic device 140 processes data input from the input device 110 and the data reading device 116 and data stored in the storage device 130, and outputs the results to the output device 120 or records them in the storage device 130. The arithmetic device 140 is configured to include the following processing units (an input control unit 141, an output control unit 142, an operation optimization unit 143, a range estimation unit 144, a transition probability calculation unit 145, a state division number adjustment unit 146, a model generation unit 147, a transition calculation unit 148, a learning data estimation unit 149, and an operation amount determination unit 150). The arithmetic device 140 is, for example, a CPU (Central Processing Unit).

[0020] The input control unit 141 classifies data input from the input device 110, data reading device 116, or temperature sensor 209 into commands, models, etc., and transfers them to the storage device 130 and each part of the arithmetic device 140.

[0021] The output control unit 142 is a part that outputs the data stored in the storage device 130 to the output device 120. The output control unit 142 outputs the average value to the output device 120 as an evaluation result of the controlled model together with the range of insufficient learning data estimated by the range estimation unit 144. When the output destination is a screen or the like, it is preferable that the result is output each time a reading operation is performed. When the output destination is a communication destination or the like, the output process may be performed each time some or all of the calculations of the calculation device 140 are performed, or the data may be processed by aggregating several times of data, aggregating at predetermined intervals, or the like.

[0022] The operation optimization unit 143 estimates a control law for achieving the purpose of the controlled object of the intelligent control device 100 from the operation data storage unit 131, and stores the estimated control law in the control law storage unit 133. This enables the model generation unit 147 to easily generate a controlled object model that simulates the behavior of the controlled object.

[0023] The model generation unit 147 generates a control object model that simulates the behavior of the control object from the driving data stored in the driving data storage unit 131 or the control law stored in the control law storage unit 133, and stores this control object model in the model storage unit 132. In other words, the model generation unit 147 learns the driving data obtained by driving the control object as learning data, and generates a control object model that simulates the movement of the control object.

[0024] The range estimation unit 144 estimates a range in the state space of the controlled object where learning data is insufficient, using the controlled object model. Here, the range of the operating data refers to a predetermined range of the state space indicated by the state parameters or control parameters of the controlled object. The range estimation unit 144 stores the range where the learning data is insufficient, which is the result of its own estimation, in the data insufficient range storage unit 134.

[0025] The transition probability calculation unit 145 calculates the probability that the control object of the intelligent control device 100 will transition from an initial state to a target state and achieve the objective, based on the information stored in the data shortage range storage unit 134 and the information on the behavior of the control object stored in one or more of the operation data storage unit 131, the model storage unit 132, and the control law storage unit 133. The probability that the control object will achieve the objective indicates the control performance of the control object model and indicates the sufficiency of the learning data related to the control performance. This makes it possible to evaluate the sufficiency of the learning data related to the control performance.

[0026] The transition calculation unit 148 uses the transition probability calculated by the transition probability calculation unit 145 to calculate at least one of the transition time from the initial state to the target state, the number of transitions of the state of the controlled object, and the probability of transitioning to the target state.

[0027] The learning data estimation unit 149 estimates the conditions of the learning data to be additionally learned by the model generation unit 147 based on the calculation result of the transition calculation unit 148. This makes it possible to identify the range of insufficient learning data regarding the control performance.

[0028] The manipulated variable determination unit 150 divides the state space of the controlled object model into discrete spaces, and determines the minimum number of control steps for making the state of the controlled object into a target state for each divided space as the manipulated variable for each space. The range estimation unit 144 calculates the standard deviation of the manipulated variables around each space in the space obtained by discretely dividing the state of the controlled object model. In this way, the range estimation unit 144 estimates the space with insufficient learning data.

[0029] 2 is a diagram showing the configuration of a controlled object. In order to explain the details of the processing executed by the intelligent control device 100, the example of steam temperature control of a thermal power plant is taken here. Note that the scope of application of the present invention includes industrial plants, mobile robots, self-driving vehicles, etc., but the controlled object of the present invention is not limited to these examples.

[0030] 2, the steam temperature of a thermal power plant 200 is controlled. The thermal power plant 200 to be controlled is composed of a boiler drum 201, steam pipes 202, 210, and 211, a first superheater 203, a second superheater 204, a steam turbine 205, a spray 206, a control valve 207, a spray pipe 208, and a temperature sensor 209. The superheated steam flows from the boiler drum 201 through the steam pipe 202 into the first superheater 203. The steam is superheated as it passes through the inside of the first superheater 203, and the temperature of the steam increases.

[0031] The steam then flows into the second superheater 204 through a steam pipe 210. The steam is further superheated as it passes through the inside of the second superheater 204, and the temperature of the steam increases. The steam then flows into the steam turbine 205 through a steam pipe 211, and rotates the steam turbine 205, causing a generator (not shown) connected to the steam turbine 205 to generate electricity.

[0032] When the steam passes through a steam pipe 210 between the first superheater 203 and the second superheater 204, compressed water may be injected from a spray 206. The injected compressed water is mixed with the steam, thereby reducing the temperature. The amount of temperature reduction depends on the amount of compressed water injected, which varies depending on the valve opening of a control valve 207. The control valve 207 is installed in a spray pipe 208 through which the compressed water passes. The steam temperature is measured by a temperature sensor 209 installed in a steam pipe 211 at the outlet of the second superheater 204. In the steam temperature control, the valve opening of the control valve 207 is manipulated to control the temperature of the steam measured by the temperature sensor 209 to a desired temperature.

[0033] Figure 3 is a graph of steam temperature versus elapsed time. The horizontal axis of the graph is elapsed time. The vertical axis of the graph is steam temperature. Here, an example of the behavior of the steam temperature of the thermal power plant 200 in the example of Fig. 2 will be described. The goal in this example is to transition to state s4 where the steam temperature is 500°C and the temperature change rate is approximately 0°C / control step. The initial temperature of the steam is 495°C and the temperature change rate is approximately 3°C / control step (state s1). Note that the control step is the time until the manipulated variable is updated using the control law stored in the control law storage unit 133, and may be set separately from the control cycle.

[0034] Hereinafter, the operation data of the thermal power plant 200, which is the object of control, will be described with reference to Figs. FIG. 4 is a diagram showing the correspondence between states and amounts of operation. The control law in Figure 4 is a vector that stores the manipulated variable for each plant state.

[0035] FIG. 5 is a diagram showing the correspondence between state, temperature, and rate of change. In the example of state definition in Figure 5, the state is defined by a combination of the steam temperature and the temperature change rate. For example, if the steam temperature is 499°C and changes by 3°C for each control step, it indicates state s3.

[0036] FIG. 6 is a diagram showing the correspondence between the operation amount and the valve opening adjustment amount. In the example of the definition of the manipulated variable in Fig. 6, the manipulated variable a1 indicates that the valve aperture of the regulating valve 207 is decreased by 1%. The manipulated variable a2 indicates that the valve aperture of the regulating valve 207 is maintained, and the manipulated variable a3 indicates that the valve aperture of the regulating valve 207 is increased by 1%. In other words, the valve aperture of the regulating valve 207 is adjusted depending on the manipulated variable, thereby controlling the state of the thermal power plant 200.

[0037] Returning to Figure 3, we continue the explanation. Here, the system starts from state s1, where the initial state is a temperature of 494°C and the temperature change rate is about 3°C / control step, and transitions to state s4 via states s2 and s3, successfully achieving the target steam temperature of 500°C and the temperature change rate of about 0°C / control step. However, there are cases where state s1 transitions to state s5 stochastically due to the influence of disturbances, etc. In the example, state s5 has insufficient learning data, and the manipulated variable that moves away from the target state may be output, or the manipulated variable may not be able to be estimated. Note that even in state s6, there is no learning data required for calculations to transition to the target state s4 in the shortest control step, but there is enough learning data to calculate the manipulated variable to transition to the target state s4 via state s7.

[0038] FIG. 7 is a flowchart of the data processing. 2 is a diagram showing a process flow performed by the intelligent control device 100. The process flow will be explained using the examples of FIGS.

[0039] First, in step S1201, the operation optimization unit 143 estimates a control law that achieves the purpose of the controlled object of the intelligent control device 100 from the operation data stored in the operation data storage unit 131, and stores the estimated control law in the control law storage unit 133. In the example of Fig. 2, the control law is estimated based on reinforcement learning technology. Reinforcement learning is a type of machine learning, and is a technology in which the system learns an appropriate control method by repeating trial and error.

[0040] Then, in step S1202, the model generation unit 147 generates a control object model that simulates the behavior of the control object using, for example, a state transition probability as shown in FIG. 8 from the driving data stored in the driving data storage unit 131, and stores the generated control object model in the model storage unit 132.

[0041] The form of the control object model that simulates the behavior of the control object may be, for example, a state transition probability matrix, a neural network, a radial basis function network, or a matrix or vector expressing the weights of a neural network or a radial basis function network, but the present invention does not limit the model storage form of the simulation object to these examples. By adopting a neural network for the control object model, it is possible to easily learn the training data. Also, by adopting a radial basis function network for the control object model, it is possible to solve nonlinear problems.

[0042] Next, in step S1203, the range estimation unit 144 estimates a range in which learning data is insufficient to determine whether the control target achieves the objective. In the example of FIG. 3, this corresponds to state s5. The range estimation unit 144 counts the number of pieces of learning data for each combination of range and operation amount, and estimates the range as being insufficient in learning data when the number of pieces of learning data is less than a preset number. The range estimation unit 144 may also estimate a range in which there is no means of transition to the target state no matter how much learning data is increased, or a range in which transition to the target state cannot be made by repeating transitions between two or more ranges, as being included in the "range in which learning data is insufficient." This allows the transition probability calculation unit 145 to calculate the probability of reaching the target state without transitioning to a state in which transition to the target state is not possible due to causes other than insufficient learning data.

[0043] In addition to the method of counting the number of pieces of learning data, there are other methods such as a method of determining whether learning data is insufficient using the density or distribution of the number of pieces of learning data, or a method of determining whether learning data is insufficient using the minimum number of control steps to the target state, but the method of estimating the state in which learning data is insufficient in this embodiment is not limited to these. In addition, a detailed example of a method of estimating the state in which learning data is insufficient will be described later in the second embodiment.

[0044] Returning to Fig. 7, the explanation will be continued. In step S1204, the arithmetic device 140 sets an initial state of the control target. Possible methods for setting the initial state include a method of referring to a list of initial states registered in advance, or a method of randomly selecting a state from states that have transition experience, but the present invention does not limit the method for setting the initial state to these examples.

[0045] Next, in step S1205, the transition probability calculation unit 145 calculates the transition probability from the initial state of the control object to the target state. In order to explain the method of calculating the transition probability to the target state, the transition probability calculation unit 145 uses the example of FIG. 8, for example. In the example of FIG. 8, state s1 is the initial state, and state s4 is the target state, as in FIG. 3. Note that when there are multiple target states, the transition probability calculation unit 145 calculates the probability of transitioning to any one of the multiple target states.

[0046] The transition probability calculation unit 145 determines that the control of the control object has failed when the state has transitioned to one for which the range estimation unit 144 has evaluated that the learning data is insufficient. x corresponds to a state of insufficient training data.

[0047] As shown in FIG. 8, the controlled object transitions from state s1 to state s5 with a probability of 20%. The controlled object transitions from state s2 to state s6 with a probability of 40% and from state s2 to state s3 with a probability of 60%. The controlled object transitions from state s6 to state s7 with a probability of 100%. The controlled object transitions from state s7 to state s3 with a probability of 100%. The controlled object transitions from state s3 to state s x and from state s3 there is a 70% probability of transitioning to state s4.

[0048] In this case, the overall transition probability from the initial state s1 to the target state s4 is 56%, and the transition probability calculation unit 145 calculates the evaluation of control from state s1 as "56% probability of reaching the target state."

[0049] Returning to Fig. 7, the explanation will be continued. In step S1206, the arithmetic device 140 judges whether the initial state has been evaluated. If there is only one initial state, or if the attainment rate to the goal state has already been evaluated for all initial states (Yes), the process proceeds to step S1207. If not (No), the process returns to step S1203, and the arithmetic device 140 sets a new initial state.

[0050] In step S1207, the transition probability calculation unit 145 tallies up the rates of arrival from all the initial states to the goal state, and calculates the average value of the rates of arrival to the goal state. In step S1208, the output control unit 142 outputs the average value as an evaluation result of the controlled object model to the output device 120 together with the range of insufficient learning data estimated by the range estimation unit 144. By outputting the rate of attainment of the target state as the evaluation result of this controlled object model, it is possible to evaluate the sufficiency of the learning data related to the control performance. When the process of step S1208 ends, the process of FIG. 7 ends.

[0051] Note that the transition probability calculation unit 145 may output a statistical value indicating a tendency of the overall attainment rate, such as a maximum or minimum attainment rate, other than the average value of the attainment rate to the goal state, to the output device 120. Furthermore, the arithmetic device 140 may output a transition time from the initial state to the goal state, the number of transitions of the state of the controlled object, and the like to the output device 120, and is not limited thereto.

[0052] According to the present invention, it is possible to evaluate the sufficiency of operation data from the control law calculated by the artificial intelligence technology and the control performance based on this operation data without using a simulator. This shortens the time required to evaluate the adaptability of the artificial intelligence technology to a new plant to be controlled, and is expected to shorten the lead time for applying the artificial intelligence technology to the plant. It is also possible to shorten the time required to adjust the parameters of the artificial intelligence technology. Furthermore, by estimating the range in which learning data is insufficient, it is possible to suitably improve the control performance by adding a relatively small amount of learning data.

[0053] Second Embodiment In the first embodiment, the control law calculated by the operation optimization unit 143 is evaluated by calculating the transition probability from the initial state to the target state using information on a state estimated to have insufficient learning data. In this embodiment, a means for estimating a state with insufficient data from the shortest number of control steps to the target state is described. Note that Figs. 9 and 10 show a case where there is sufficient data, and Figs. 11 and 12 show a case where there is insufficient data, and the details of the means are described by comparing these cases.

[0054] Figure 9 is a diagram showing the relationship between the minimum number of control steps from all states to the target state (steam temperature 500°C and temperature change rate approximately 0°C / control step) and steam temperature and temperature change rate in the steam temperature control problem in the example problem in Figure 2. Each column indicates the steam temperature, and each row indicates the temperature change rate. The contents of each cell indicate the minimum number of control steps. Figure 9 shows the state space of the model to be controlled. Each cell is a discrete division of this state space.

[0055] The state is defined by the steam temperature and the temperature change rate, as in the example in Figure 5, with the steam temperature range being 492 to 503°C, and the temperature change rate range being -4.0 to 4.0°C / control step. Note that the "-" in Figure 9 indicates that there is no means of transition to the target state. For example, in the case of a state of 503°C and 4.0°C / control step, no matter what manipulated variable is selected, the temperature will exceed the defined range, so there is no means of transition to the target state. Therefore, in Figure 9, "-" is written in place of the minimum number of control steps. Note that in Figure 9, it is assumed that all learning data is sufficient and that there is no additional information on state transitions that can be obtained from the learning data.

[0056] Figure 10 shows the standard deviation calculated from the minimum number of control steps in Figure 9. Each column shows the steam temperature, and each row shows the temperature change rate. The content of each cell shows the standard deviation of the minimum number of control steps of the surrounding cells.

[0057] The range estimation unit 144 calculates the standard deviation of the operation amount around each space in a space obtained by discretely dividing the state of the controlled object model, thereby estimating the space where the learning data is insufficient.

[0058] The calculation of the standard deviation covers the range of the vicinity (temperature ±1°C or less, temperature change rate ±1.0°C / control step or less). For example, "steam temperature 494°C, temperature change rate 2°C / control step" and the minimum number of steps of the vicinity state are "4, 3, 5, 4, 4, 3, 5, 4, 4" in order, totaling nine, and the standard deviation of these is 0.67. In FIG. 9, since there was sufficient learning data, the standard deviation is at most 2.76. Note that this standard deviation also depends on the range of the state defined as the vicinity state. The range can be determined by a means of setting it in advance, but the present invention does not limit the range determination means.

[0059] Figure 11 is a diagram showing the minimum number of control steps to reach the target state, similar to Figure 9. Each column shows the steam temperature, and each row shows the temperature change rate. The contents of each cell show the minimum number of control steps of the surrounding cells. However, the case in Figure 11 differs from Figure 9 in that some learning data is missing in the ranges of steam temperature 498 to 502°C and temperature change rate of 1.0°C / control step or more, and steam temperature 500 to 502°C and temperature change rate of -2.0°C / control step or more. Due to the missing learning data, for example, when the steam temperature is 496°C and the temperature change rate is 2.0°C / control step, the number of control steps increases from 3 in Figure 9 to 15.

[0060] Next, FIG. 12 is a diagram showing the standard deviation of the manipulated variable around each space calculated from the minimum number of control steps in FIG. The maximum standard deviation in Figure 10 was 2.76, whereas in Figure 12, the standard deviation in the range of missing training data was all greater than 3, rising to a maximum of 5.36. In this way, states in the range where learning data is insufficient tend to have a high standard deviation. Therefore, by prioritizing data collection in the range of states with high standard deviation, the number of states with insufficient learning data can be reduced, and improvement in control performance can be expected.

[0061] Third Embodiment In this embodiment, a means for adjusting the number of divisions when defining a state will be described by utilizing the means for evaluating the control law in the first embodiment. In the first embodiment, a means for evaluating a controlled object model generated by the model generation unit 147 was described, and in the second embodiment, a learning data estimation unit 149 for estimating a state of insufficient learning data was described. However, the performance of a controlled object model and the definition of insufficient learning data also depend on the number of divisions when defining a state. Therefore, in this embodiment, a state division number adjustment unit 146 is provided that adjusts the number of divisions when defining a state based on the evaluation result of the controlled object model.

[0062] FIG. 13 shows an intelligent control device 101 obtained by adding a state division number adjusting unit 146 to the arithmetic unit 140 of the intelligent control device 100 in FIG. State division number adjustment unit 146 receives the "probability that the controlled object achieves the objective" output by transition probability calculation unit 145, and outputs a new state division number. Here, the state division number refers to the fact that, in the state definition as shown in Fig. 5, each state has a range of each process value, and changing the state division number changes the total number of states and the range of the process value of each state. State division number adjustment unit 146 expands or reduces the width of the space partitioned by operation amount determination unit 150 to increase the probability of transitioning to the target state.

[0063] 14 is a flowchart of the process performed by the intelligent control device 101. This flowchart is obtained by adding steps S1209 and S1210 to the flowchart of FIG. In step S1209, the intelligent control device 101 performs a termination judgment using the average value of the rate of arrival to the goal state calculated by the transition probability calculation unit 145. If the average value of the rate of arrival is equal to or exceeds a preset value, or if it has converged, the process in FIG. 14 is terminated. If the termination condition of the process is not satisfied, the process proceeds to step S1210. Note that if this is the first time the average value of the rate of arrival to the goal state has been input, or if the "rate of arrival to the goal state" has not been input enough to judge the termination condition of the flow, the process proceeds to step S1210.

[0064] In step S1210, state division number adjustment unit 146 uses the average value of the attainment rate to the goal state calculated by transition probability calculation unit 145 as an input, and expands or reduces the width of the space partitioned by operation amount determination unit 150 so as to increase the probability of transitioning to the goal state, thereby updating the number of state divisions. After that, the process returns to step S1201. Note that, although possible updating means include steepest descent method and stochastic gradient descent method, the present invention is not limited to these optimization means.

[0065] (Modification) The present invention is not limited to the above-described embodiment, and includes various modified examples. For example, the above-described embodiment has been described in detail to clearly explain the present invention, and is not necessarily limited to those having all of the configurations described. It is possible to replace a part of the configuration of one embodiment with the configuration of another embodiment, and it is also possible to add the configuration of another embodiment to the configuration of one embodiment. In addition, it is also possible to add, delete, or replace a part of the configuration of each embodiment with another configuration.

[0066] The above configurations, functions, processing units, processing means, etc. may be realized in part or in whole by hardware such as an integrated circuit. The above configurations, functions, etc. may be realized by software by a processor interpreting and executing a program that realizes each function. Information such as the program, table, file, etc. that realizes each function can be stored in a recording device such as a memory, a hard disk, or an SSD (Solid State Drive), or a recording medium such as a flash memory card or a DVD (Digital Versatile Disk).

[0067] In each embodiment, the control lines and information lines are those that are considered necessary for the explanation, and not all control lines and information lines in the product are necessarily shown. In reality, it may be considered that almost all components are connected to each other. [Explanation of symbols]

[0068] 100,101 Intelligent control device 110 Input Device 116 Data reading device 120 Output Device 130 Storage device 131 Operation data storage unit 132 Model Memory Section 133 Control Law Memory Unit 134 Data shortage range memory section 135 Intelligent Control Program 140 Arithmetic equipment 141 Input control section 142 Output control section 143 Operation Optimization Division 147 Model Generation Unit 148 Transition calculation part 144 Range Estimation Unit 145 Transition Probability Calculation Unit 146 State division number adjustment unit 147 Model Generation Unit 148 Transition calculation part 149 Learning Data Estimation Unit 150 Operation amount determination section 200 Thermal Power Plants 201 Boiler drum 202,210,211 Steam piping 203 1st superheater 204 2nd superheater 205 Steam Turbine 206 Spray 207 Regulating valve 208 Spray piping 209 Temperature Sensor

Claims

1. a model generation unit that learns learning data obtained by operating a controlled object and generates a controlled object model that simulates a behavior of the controlled object; a range estimation unit that estimates a range of a state of the control object for which learning data is lacking, using the control object model; a transition probability calculation unit that calculates a transition probability of the control object model transitioning from an initial state to a target state; an output control unit that outputs the range estimated by the range estimation unit and the transition probability calculated by the transition probability calculation unit; An intelligent control device comprising:

2. a transition calculation unit that calculates at least one of a transition time for transition from the initial state to the target state, a number of transitions of the state of the control object, and a probability of transitioning to the target state, using the transition probability calculated by the transition probability calculation unit; a learning data estimation unit that estimates a condition of learning data to be additionally learned by the model generation unit based on a calculation result of the transition calculation unit; The intelligent control device according to claim 1, further comprising:

3. a manipulation amount determination unit that divides a state of the control object model into discrete spaces and determines a manipulation amount of the control object for each of the divided spaces, The range estimation unit estimates a space in which learning data is insufficient based on a standard deviation of the operation amount around the space. The intelligent control device according to claim 1 .

4. The learning data is operation data of a control object. The intelligent control device according to claim 3 .

5. The control object model is a state transition probability matrix. The intelligent control device according to claim 3 .

6. The control object model is a neural network. The intelligent control device according to claim 3 .

7. The control object model is a radial basis function network. The intelligent control device according to claim 3 .

8. The control object model is a matrix or vector in which the weights of the neural network are represented. The intelligent control device according to claim 3 .

9. The control object model is a matrix or vector in which the weights of the radial basis function network are represented. The intelligent control device according to claim 3 .

10. a state division number adjustment unit that expands or reduces a width of the space to increase the probability of transitioning to a target state; The intelligent control device according to claim 3, further comprising:

11. An operation optimization unit that estimates a control law that achieves the objective of the controlled object; The intelligent control device according to claim 1, further comprising:

12. a model generation unit learning learning data obtained by operating the controlled object and generating a controlled object model that simulates a behavior of the controlled object; a range estimation unit estimating a range of a state of the control object for which learning data is insufficient, using the control object model; A transition probability calculation unit calculates a transition probability of the controlled object model transitioning from an initial state to a target state; an output control unit outputting the range estimated by the range estimation unit and the transition probability calculated by the transition probability calculation unit; An intelligent control method comprising:

13. On the computer, A step of learning learning data obtained by operating a controlled object and generating a controlled object model that simulates a behavior of the controlled object; a step of estimating a range of a state of the control object for which learning data is lacking, using the control object model; A step of calculating a transition probability of the control object model transitioning from an initial state to a target state; a step of outputting a range of the state of the control object for which learning data is lacking and the transition probability; An intelligent control program for executing the above.

Images