Apparatus, method, and computer readable medium

Abstract

Provided is an apparatus, method, and computer readable medium, the apparatus having: a supply section that supplies a value of a state parameter to an operation model that outputs a recommended value of a control parameter of a device in correspondence with the value of the state parameter being input in relation to the device; a control parameter acquisition section that acquires the recommended value of the control parameter output from the operation model in correspondence with the value of the state parameter supplied to the operation model by the supply section; an acquisition section that acquires a model evaluation value in correspondence with a result of operating the device in accordance with the recommended value acquired by the control parameter acquisition section; and an evaluation section that evaluates the operation model based on the model evaluation value and a reference evaluation value in correspondence with a result of operating the device by manual operation.

Description

Technical Field

[0001] The present invention relates to an apparatus, a method, and a computer-readable medium. Background Art

[0002] Patent Document 1 describes the following: "Perform learning processing of a first model that outputs recommended control parameters representing first control content recommended to increase a reward value defined by a preset reward function based on input of measurement data."

[0003] Patent Document 1: Japanese Unexamined Patent Application Publication No. 2021-086283

[0004] Patent Document 2: Japanese Unexamined Patent Application Publication No. 2020-027556

[0005] Patent Document 3: Japanese Unexamined Patent Application Publication No. 2019-020885

[0006] Non-Patent Document 1: Takami Go, "Realization of Plant Control AI", Yokogawa Technical Report, Yokogawa Electric Corporation, 2020, Vol. 63, No. 1, pp. 33-36

[0007] Non-Patent Document 2: Imai Takuji, "Yokogawa Electric and NAIST Implement Reinforcement Learning for Chemical Plants to Achieve Advanced Control with Few Trials", Nikkei Robotics, Nikkei BP Co., Ltd., March 2019 issue

[0008] and realize advanced control with few trial times", Nikkei Robotics, Nikkei BP Co., Ltd., March 2019 issue Summary of the Invention

[0009] In a first aspect of the present invention, an apparatus is provided. The apparatus may have a supply unit that supplies a value of a state parameter to an operation model, and the operation model outputs a recommended value of a control parameter of a device in response to input of a value of a state parameter related to the device. The apparatus may have a control parameter acquisition unit that acquires a recommended value of a control parameter output from the operation model corresponding to the supply of the value of the state parameter to the operation model by the supply unit. The apparatus may have an acquisition unit that acquires a model evaluation value corresponding to a result of operating the device according to the recommended value acquired by the control parameter acquisition unit. The apparatus may have an evaluation unit that evaluates the operation model based on the model evaluation value and a reference evaluation value corresponding to a result of operating the device by manual operation.

[0010] The reference evaluation value may be calculated based on a result of inputting manual operation to a simulator of the device.

[0011] The model evaluation value may be calculated based on a result of inputting the recommended value acquired by the control parameter acquisition unit to a simulator of the device.

[0012] Model evaluation values ​​can be calculated based on whether parameters related to equipment operated according to recommended values ​​fall within the target range. Baseline evaluation values ​​can be calculated based on whether parameters related to equipment operated manually fall within the target range.

[0013] The device may also have a target range acquisition unit that acquires a target range set by the operator for a selection parameter selected by the operator among a variety of parameters related to the device.

[0014] The device may also have a display control unit that displays the value of the selected parameter from a variety of parameters, based on the previous operation of the device.

[0015] The display control unit can display the values ​​of each selected parameter from previous operations of the device in a coordinate space with each selected parameter as the coordinate axis.

[0016] Equipment can be equipment used to manufacture articles. Parameters related to the equipment can be at least one of the following: an indicator value representing the quality of the article or the quantity of articles produced.

[0017] The device may also have a learning processing unit that performs learning processing of the operation model using learning data including the values ​​of state parameters and control parameters.

[0018] The learning processing unit can perform learning processing of the operation model using learning data and reward values ​​specified by a pre-defined reward function.

[0019] In a second aspect of the invention, a method is provided. The method may include a supply phase for supplying values ​​of state parameters to an operating model, the operating model outputting recommended values ​​of control parameters for the device corresponding to the input of values ​​representing state parameters related to the device. The method may include a control parameter acquisition phase for acquiring recommended values ​​of control parameters output from the operating model corresponding to the values ​​of the state parameters supplied to the operating model in the supply phase. The method may include an acquisition phase for acquiring a model evaluation value corresponding to the result of operating the device using the recommended values ​​acquired in the control parameter acquisition phase. The method may include an evaluation phase for evaluating the operating model based on the model evaluation value and a benchmark evaluation value corresponding to the result of operating the device using manual operation.

[0020] In a third aspect of the present invention, a computer-readable medium having a program recorded thereon is provided. The program causes a computer to function as a supply unit, that is, to supply values ​​of state parameters to an operation model, which outputs recommended values ​​of control parameters for the device in response to the input values ​​of state parameters related to the device. The program causes the computer to function as a control parameter acquisition unit, that is, to acquire recommended values ​​of control parameters output from the operation model in response to the values ​​of state parameters supplied by the supply unit to the operation model. The program causes the computer to function as an acquisition unit, that is, to acquire a model evaluation value corresponding to the result of operating the device based on the recommended values ​​acquired by the control parameter acquisition unit. The program causes the computer to function as an evaluation unit, that is, to evaluate the operation model based on the model evaluation value and a benchmark evaluation value corresponding to the result of operating the device manually.

[0021] Furthermore, the above summary of the invention does not list all the essential features of the invention. Additionally, sub-components of the above-described feature set can also constitute the invention. Attached Figure Description

[0022] Figure 1 This refers to system 1, which is involved in the implementation method.

[0023] Figure 2 The data structure representing operation model 401.

[0024] Figure 3 This indicates a decision-making form.

[0025] Figure 4 This represents the learning action of operation model 401.

[0026] Figure 5 This indicates other learning actions of operation model 401.

[0027] Figure 6 This represents the learning action of the goal-setting model 414.

[0028] Figure 7 This indicates the evaluation action of operation model 401.

[0029] Figure 8 This indicates the operation action of device 2.

[0030] Figure 9 The target setting model 414A is indicated in the variant example.

[0031] Figure 10 Examples of computers 2200 that can embody all or part of the various embodiments of the present invention are shown. Detailed Implementation

[0032] The present invention will now be described through embodiments thereof, which do not limit the invention as defined in the claims. Furthermore, not all combinations of the features described in the embodiments are essential to the solution of the invention.

[0033] [1. System]

[0034] Figure 1 This figure shows system 1, which is part of this embodiment. System 1 includes device 2 and apparatus 4. Furthermore, the modules shown in the figure are functionally separate modules and may not correspond to the actual device structure. That is, in this figure, they are shown as a single module, but they may not be composed of a single apparatus. Additionally, in this figure, they are shown as different modules, but they may not be composed of different apparatuses.

[0035] [1.1. Equipment 2]

[0036] Equipment 2 is a facility or device that has one or more instruments (not shown). For example, equipment 2 can be a factory or a composite device composed of multiple instruments. As a factory, in addition to industrial factories such as chemical and biological factories, examples include factories that manage and control the wellheads and surrounding areas of gas fields and oil fields, factories that manage and control power generation such as hydropower, thermal power, and nuclear power, factories that manage and control environmental power generation such as solar and wind power, and factories that manage and control water supply and drainage, dams, etc.

[0037] Each instrument is a device, machine, or apparatus, such as a valve, pump, heater, fan, motor, switch, or other actuator that controls at least one physical quantity, such as pressure, temperature, pH, speed, or flow rate, in the process of device 2. The types of instruments may be different, or at least two or more instruments may be of the same type. Each instrument can be controlled by device 2 via wired or wireless means.

[0038] One or more sensors (not shown) can be installed in device 2. Each sensor measures or determines the status related to device 2. Each sensor can measure or determine the operational status of device 2, such as production volume, proportion of impurities, operating status of each controlled object, and alarm generation status. As an example, the operating status of the instrument can be represented by at least one physical quantity controlled by the instrument, such as pressure, temperature, pH, speed, and flow rate. Each sensor can supply the measurement or determination results to device 2.

[0039] [1.2. Apparatus 4]

[0040] Device 4 can operate device 2 using operation model 401, and may include operation model 401, operation unit 402, and parameter acquisition unit 403. Furthermore, device 4 can perform learning processing of operation model 401, and may include input unit 411, storage unit 412, second learning processing unit 413, target setting model 414, first supply unit 415, second acquisition unit 416, and second supply unit 417. Additionally, device 4 can perform learning processing of target setting model 414, and may include first acquisition unit 421 and first learning processing unit 422. Furthermore, device 4 can evaluate operation model 401, and may include simulator 431, display control unit 432, target range acquisition unit 433, evaluation value acquisition unit 434, and evaluation unit 435.

[0041] [1.2.1. Operation Model 401]

[0042] Operation model 401 is a model for operating device 2. Operation model 401 can output recommended values ​​for the control parameters of device 2 in response to input values ​​of state parameters related to device 2. Operation model 401 can supply control parameters to operation unit 402.

[0043] Operation model 401 can output recommended values ​​for control parameters that approximate the state of the target setting data used by operation model 401 to learn the state associated with device 2, in response to the input values ​​of state parameters. The target setting data may include identification information of the parameter of the target setting object among the parameters associated with device 2, and the target range set for that parameter. The target setting data may contain only one combination of parameter identification information and target range, or it may contain multiple combinations.

[0044] The parameters related to device 2 may include state parameters related to device 2 and control parameters related to device 2. State parameters related to device 2 may include state parameters of device 2 and state parameters (also called performance parameters) of the product obtained based on the operation of device 2. State parameters of device 2 may include, for example, pressure, flow rate, temperature, pH, speed, power consumption, concentration, etc. State parameters of device 2 may be parameters related to the energy consumption of device 2, parameters related to the emission of greenhouse gases, or parameters related to yield. State parameters of the product may include, for example, quality index values ​​(also called quality values), production quantity, etc. Quality values ​​may include, for example, values ​​representing the purity, concentration, composition, viscosity, color, etc. of the product. Control parameters may include, for example, the operating amount of a valve. Furthermore, in this embodiment, as an example, the parameters of the target range setting object may be state parameters related to device 2.

[0045] [1.2.2. Operation Unit 402]

[0046] The operation unit 402 operates the device 2 using the operation model 401. The operation unit 402 can be an example of a control parameter acquisition unit, which acquires recommended values ​​of control parameters output from the operation model 401 based on the values ​​of state parameters supplied to the operation model 401 by the parameter acquisition unit 403 (described later). The operation unit 402 can control each instrument of the device 2 using the control parameters output from the operation model 401, thereby operating the device 2. The operation unit 402 can also supply the recommended values ​​of the control parameters output from the operation model 401 to the simulator 431 and simulate the operation of the device 2.

[0047] Furthermore, the operation unit 402 can operate the device 2 based on manual operations input via the input unit 411. The operation unit 402 can supply control parameters corresponding to the manual operations to the simulator 431 and simulate the operation of the device 2.

[0048] [1.2.3. Parameter Acquisition Unit 403]

[0049] The parameter acquisition unit 403 acquires parameters related to device 2. The parameter acquisition unit 403 can acquire control parameters for device 2 from the operation unit 402. The parameter acquisition unit 403 can acquire state parameters related to device 2 from device 2 (in this embodiment, as an example, state parameters of the product and state parameters of device 2). However, the parameter acquisition unit 403 can acquire control parameters for device 2 from device 2, or it can acquire state parameters of the product from the operator. Furthermore, when simulating device 2 using simulator 431, the parameter acquisition unit 403 can acquire state parameters related to device 2 from the parameters related to the simulated device 2 from simulator 431.

[0050] The parameter acquisition unit 403 can store the acquired parameters in the storage unit 412. The parameter acquisition unit 403 can be an example of a supply unit, or it can supply the values ​​of state parameters for the operation model 401.

[0051] [1.2.4. Input Section 411]

[0052] The input unit 411 receives various input operations from the operator. The input unit 411 can receive input operations for the operation plan of device 2 from the operator. Additionally, the input unit 411 can receive input operations for target setting data from the operator. Even if the learning process of the target setting model 414 is not yet complete, the operation plan and target setting data can be input in conjunction.

[0053] Here, the operation plan for equipment 2 can represent at least one of the following: the planned production quantity, target quality, and material type of the items produced by equipment 2. The operation plan for equipment 2 can also represent other information such as the energy efficiency, power consumption, yield rate, and greenhouse gas emissions of equipment 2.

[0054] The input unit 411 can store the input operation plan and target setting data in the storage unit 412. The input unit 411 can supply the input target setting data to the second learning processing unit 413. The input unit 411 can supply the input operation plan to the first supply unit 415.

[0055] [1.2.5. Storage Unit 412]

[0056] Storage unit 412 stores various types of data. Storage unit 412 can store parameters acquired by parameter acquisition unit 403. Additionally, storage unit 412 can store target setting data input by input unit 411 and operation plan of device 2. The target setting data stored in storage unit 412 can be data used for learning operation model 401; for example, it can also be data set by a skilled operator. Furthermore, when target setting data is generated by supplying an operation plan to target setting model 414 (described later), storage unit 412 can further store the target setting data and the operation plan. The data stored in storage unit 412 can be used for learning processing by first learning processing unit 422 and second learning processing unit 413.

[0057] [1.2.6. Second Learning Processing Department 413]

[0058] The second learning processing unit 413 uses learning data, including values ​​of state parameters related to device 2 and values ​​of control parameters of device 2, to perform learning processing of operation model 401 in a manner where recommended values ​​of control parameters are output corresponding to the input values ​​of state parameters. The types of state parameters and control parameters included in the learning data used by the second learning processing unit 413 can be arbitrarily selected by the operator from the parameters acquired by the parameter acquisition unit 403.

[0059] The second learning processing unit 413 can perform learning processing of the operation model 401 through reinforcement learning. For example, the second learning processing unit 413 can perform learning processing of the operation model 401 using learning data and a reward value specified by a pre-set reward function.

[0060] The second learning processing unit 413 can also perform learning processing on the operation model 401 using target setting data. This learning processing involves inputting values ​​corresponding to state parameters and outputting control parameter values ​​that make the state associated with the device 2 approximate the state corresponding to the content of the target setting data. In this case, the second learning processing unit 413 can perform learning processing using a reward value defined by a reward function, which is set based on the content of the target setting data.

[0061] For example, the reward function can be a function that sets the reward value to 1 if the value of the state parameter associated with the device 2, which has been operated using the control parameters output from the operation model 401, meets the content of the target setting data, and sets the reward value to 0 if it does not meet the content. Alternatively, the reward function can be a function that changes the reward value according to the degree to which the value of the state parameter associated with the device 2, which has been operated using the control parameters output from the operation model 401, deviates from the target range of the target setting data. As an example, the reward function can be a function represented by the following equation (1).

[0062] Compensation value = a * energy saving index - b * deviation of quality value (1)

[0063] Furthermore, "a" and "b" in equation (1) can be coefficients. "Energy saving index" refers to the index representing the energy saving degree of equipment 2, which can be a value calculated based on the state parameters of equipment 2. "Deviation degree of quality value" can be the magnitude of the deviation of the quality value of the product from the target range of the quality value set in the target data.

[0064] [1.2.7. Goal Setting Model 414]

[0065] The target setting model 414 is input in response to the operation plan and outputs at least one of the target setting data, parameter identification information, or target range, which should be used for learning by the operation model 401. In this embodiment, as an example, the target setting model 414 may output both parameter identification information and target range. The target setting model 414 may output only one combination of parameter identification information and target range, or it may output multiple combinations.

[0066] [1.2.8. Supply Department 1 415]

[0067] The first supply unit 415 supplies the operation plan to the target setting model 414 in response to the input of the operation plan of the device 2. The first supply unit 415 supplies the new operation plan to the target setting model 414 in response to the input unit 411. As a result, data corresponding to the operation plan is output from the target setting model 414.

[0068] [1.2.9. Second Acquisition Section 416]

[0069] The second acquisition unit 416 acquires output data from the target setting model 414 of the operation plan supplied by the first supply unit 415. In this embodiment, as an example, the second acquisition unit 416 can acquire both parameter identification information and target range from the target setting model 414 as output data. The second acquisition unit 416 can then supply the acquired output data to the second supply unit 417.

[0070] [1.2.10. Second Supply Department 417]

[0071] The second supply unit 417 supplies the second learning processing unit 413 with target setting data corresponding to the output data acquired by the second acquisition unit 416. Thus, the learning processing of the operation model 401 is performed using the target setting data supplied from the second supply unit 417.

[0072] Furthermore, in this embodiment, as an example, the output data from the target setting model 414 includes both parameter identification information and target range. Therefore, the second supply unit 417 can directly supply the output data to the operation model 401 as target setting data.

[0073] [1.2.11. First Acquisition Section 421]

[0074] The first acquisition unit 421 acquires the operation plan of the device 2 and the identification information of at least the parameters in the target setting data for learning the operation model 401. In this embodiment, as an example, the first acquisition unit 421 can acquire both the identification information of the parameters and the target range in the target setting data. The first acquisition unit 421 can supply the acquired data to the first learning processing unit 422.

[0075] [1.2.12. First Learning Processing Department 422]

[0076] The first learning processing unit 422 uses learning data, including the identification information of the parameters acquired by the first acquisition unit 421 and the operation plan, to perform learning processing on the target setting model 414.

[0077] The first learning processing unit 422 performs learning processing of the goal-setting model 414 through teacher-assisted learning methods such as deep learning, but it can also perform learning of the goal-setting model 414 through other machine learning methods. For example, the first learning processing unit 422 can perform learning processing of the goal-setting model 414 using learning data that includes identification information of parameters used for learning the goal-setting data of the operation model 401, and operation plans input in association with the goal-setting data. In this embodiment, as an example, the first learning processing unit 422 can perform learning processing of the goal-setting model 414 using learning data that includes identification information of parameters used for learning the goal-setting data of the operation model 401 set by a skilled operator, and operation plans input in association with the goal-setting data.

[0078] The first learning processing unit 422 can perform learning processing on the target setting model 414 in a manner that makes the content of the output data from the target setting model 414 similar to the content of the target setting data used for learning the operation model 401. Furthermore, the first learning processing unit 422 can perform learning processing on the target setting model 414 in a manner that outputs identification information and target range of the parameters that should be used for learning the operation model 401 to achieve the operation plan, in accordance with the input of the operation plan.

[0079] The first learning processing unit 422 can perform learning processing on the target setting model 414 using learning data that also includes the target range of parameters acquired by the first acquisition unit 421. That is, the first learning processing unit 422 can perform learning processing on the target setting model 414 using learning data that includes identification information of parameters used for learning the operation model 401 and the target range. The first learning processing unit 422 can perform learning processing on the target setting model 414 in a manner that outputs both the identification information of parameters and the target range from the target setting data that should be used to learn the operation model 401 in order to achieve the operation plan, in accordance with the input of the operation plan.

[0080] [1.2.13. Emulator 431]

[0081] Simulator 431 simulates the state of device 2. Simulator 431 can be a dynamic simulator that dynamically simulates from the stable state of device 2 until it stops, or it can be a static simulator that simulates the stable state of device 2.

[0082] The simulator 431 can simulate the state of the device 2 that has been operated based on the values ​​of control parameters supplied from the operation unit 402. The values ​​of the control parameters supplied from the operation unit 402 can be recommended values ​​of control parameters output from the operation model 401, or control parameters corresponding to manual operation. The simulator 431 can supply parameters related to the device 2 operated in the simulator (in this embodiment, as an example, state parameters related to the device 2) to the evaluation value acquisition unit 434 and the parameter acquisition unit 403.

[0083] [1.2.14. Display Control Unit 432]

[0084] The display control unit 432 enables a display device (not shown) to display various information. For example, the display control unit 432 can display identification information and target range of parameters acquired by the second acquisition unit 416 from the target setting model 414. Additionally, the display control unit 432 can read and display parameters acquired by the parameter acquisition unit 403 from the storage unit 412. The display control unit 432 can also display the value of any selected parameter (also called a selected parameter) from a variety of parameters related to the device 2, in response to the operator's selection of such a parameter from these parameters in previous operations of the device 2.

[0085] Furthermore, when the device 2 is manufacturing an article, the parameters related to the device 2 in the display control unit 432, the target range acquisition unit 433 (described later), the evaluation value acquisition unit 434, and the evaluation unit 435 can be state parameters related to the product. In this embodiment, as an example, these can be at least one of an index value representing the quality of the product or the production quantity of the product. Alternatively, the parameters related to the device 2 can be state parameters of the device 2 (for example, the energy efficiency, power consumption, etc. of the device 2) or control parameters of the device 2.

[0086] [1.2.15. Target Range Acquisition Section 433]

[0087] The target range acquisition unit 433 acquires the target range (also called the evaluation target range) set by the operator for evaluating the operation model 401 based on the selection parameters selected by the operator. For example, the target range acquisition unit 433 can acquire the evaluation target range set by the operator based on the previous operation values ​​displayed by the display control unit 432 for each selection parameter. The target range acquisition unit 433 can supply the acquired evaluation target range to the evaluation value acquisition unit 434.

[0088] Furthermore, the target range for evaluation can be the same as or different from the target range in the target setting data used for learning the operational model 401. Additionally, the parameters used to set the target range for evaluation can be the same as or different from the parameters in the target setting data used for learning the operational model 401.

[0089] [1.2.16. Evaluation Value Acquisition Section 434]

[0090] The evaluation value acquisition unit 434 acquires and utilizes the recommended values ​​obtained by the operation unit 402 from the operation model 401 to operate the device 2, resulting in a corresponding model evaluation value. The evaluation value acquisition unit 434 can supply the acquired model evaluation value to the evaluation unit 435.

[0091] The model evaluation value can be an evaluation value used to evaluate the operation model 401. In this embodiment, as an example, the model evaluation value can be calculated based on whether the parameters related to the device 2, which has operated by means of the recommended value output from the operation model 401, fall within the target range for evaluation. Furthermore, the model evaluation value can be the same as the reward value used by the second learning processing unit 413 in the reinforcement learning of the operation model 401, or it can be a different value.

[0092] In addition, the evaluation value acquisition unit 434 can also acquire a benchmark evaluation value corresponding to the result of operating the device 2 manually (for example, by a skilled operator). The evaluation value acquisition unit 434 can supply the acquired benchmark evaluation value to the evaluation unit 435.

[0093] The benchmark evaluation value can be the baseline value for the model evaluation value. The benchmark evaluation value can be calculated in the same way as the model evaluation value, based on whether the parameters related to the device 2, which was operated manually, fall within the target range.

[0094] [1.2.17. Evaluation Department 435]

[0095] The evaluation unit 435 evaluates the operating model 401 based on the model evaluation value and the benchmark evaluation value. The evaluation unit 435 can evaluate the operating model 401 based on the comparison between the model evaluation value and the benchmark evaluation value. For example, if the model evaluation value is better than the benchmark evaluation value, the evaluation unit 435 can evaluate the operating model 401 as good. The evaluation unit 435 can output the evaluation result to the display control unit 432, etc.

[0096] According to the above-described apparatus 4, a target setting model 414 is input corresponding to an operation plan, and outputs at least one of the parameter identification information or target range in the target setting data to be used for learning the operation model 401. Furthermore, the target setting model 414 is trained using training data including the operation plan from apparatus 2 and the parameter identification information in the target setting data used for learning the operation model 401. Therefore, the content of the output data from the target setting model 414 (here, at least one of parameter identification information or target range) can be made similar to the content of the target setting data used in learning the operation model 401 to achieve the operation plan. Therefore, by training the operation model 401 using the output data from the target setting model 414, an operation model 401 that performs appropriate operations corresponding to the operation plan can be generated.

[0097] Furthermore, by utilizing target setting data, the operation model 401 is trained by inputting control parameters corresponding to the state parameters and outputting them in a manner that makes the state associated with device 2 approximate the state corresponding to the content of the target setting data. Therefore, an operation model 401 can be generated that operates device 2 in an appropriate operating state.

[0098] Furthermore, in response to a new operation plan being input, the operation plan is supplied to the target setting model 414, and the operation model 401 is trained using target setting data corresponding to the output data from the target setting model 414. Therefore, whenever the operation plan is changed, by training the operation model 401 using the target setting data corresponding to the operation plan, an operation model 401 that performs appropriate operations corresponding to the operation plan can be generated.

[0099] Furthermore, the operation model 401 is evaluated based on the model evaluation value corresponding to the result of operating the device 2 using the recommended values ​​of the control parameters output from the operation model 401, and the benchmark evaluation value corresponding to the result of operating the device 2 by manual operation. Therefore, it is possible to uniformly judge the quality of the operation results obtained using the operation model 401, and thus the quality of the operation model 401 itself.

[0100] Furthermore, a model evaluation value is calculated based on whether the parameters related to the device 2, which was operated using the recommended values ​​of the control parameters output from the operation model 401, fall within the evaluation target range of the operation model 401. A benchmark evaluation value is also calculated based on whether the parameters related to the device 2, which was operated manually, fall within the evaluation target range. Therefore, it is possible to more consistently determine the quality of the operation results obtained using the operation model 401.

[0101] Furthermore, regarding the selection parameter chosen by the operator from among the various parameters related to device 2, the evaluation target range set by the operator can be obtained, thus allowing any evaluation target range to be set for any parameter. Therefore, the evaluation benchmark for the operation result can be set arbitrarily.

[0102] Furthermore, since the value of the selected parameter is displayed according to the previous operation of the device 2, the target range for evaluation can be set based on the value of the selected parameter in the past.

[0103] Furthermore, in the evaluation of operation model 401, the parameters related to equipment 2 are at least one of an index value representing the quality of the products produced by equipment 2 or the production quantity of the products. Therefore, it is possible to obtain an evaluation result that assesses operation model 401 as a good operation model that improves both production quantity and quality. Thus, by utilizing operation model 401 with a higher evaluation, both production quantity and quality can be improved.

[0104] Furthermore, the second learning processing unit 413 performs learning processing on the operation model 401 using learning data including the values ​​of state parameters and control parameters, thus enabling the learning processing of the operation model 401 with a lower evaluation to obtain an operation model 401 with a higher evaluation.

[0105] Furthermore, by using learning data and a reward value defined by a pre-defined reward function to perform learning processing on the operation model 401, a highly rated operation model 401 can be reliably obtained.

[0106] [2. Operation Model 401]

[0107] Figure 2 The data structure representing operation model 401 is shown. Operation model 401 has a data structure consisting of states s representing a set of sampled state data, combinations of actions a taken in each state (s, a), and weights w calculated based on rewards. Furthermore, these weights can be determined based on rewards defined by a reward function using target setting data. In this figure, as an example, the case of state s = (TI001, TI002, TI003, FI001, FI002, VI001) is shown. Moreover, in this figure, for example, when an action a = 1 is taken in the state s = (-2.47803, -2.48413, -0.07324, 29.71191, 24.2511, 70), it means that the weight calculated based on the reward is w = 144.1484. The next action is determined according to this operation model 401.

[0108] Figure 3This represents an action decision table. An action decision table consists of an input state `s` and the possible actions `a`. In this diagram, as an example, we show the case where the input states are `s = (0.1, 0.2, 0.4, 0.3, 0.8, 0.2)` and the possible actions are `a = (-3, -1, 0, 1, 3)`. For example, this action decision table could be input into... Figure 4 The operational model 401 shown determines the next action. This process will be explained in detail below.

[0109] [3. Operation of Device 4]

[0110] [3.1. Learning Actions of Operation Model 401]

[0111] Figure 4 This represents the learning action of operation model 401. Device 1 can generate operation model 401 through the processing of steps S101 to S119.

[0112] In step S101, the second learning processing unit 413 acquires target setting data. In the operation of this figure, the second learning processing unit 413 can acquire target setting data including identification information of the parameters input via the input unit 411 and the target range of the parameters.

[0113] Furthermore, in step S101, the display control unit 432 can display the content of the target setting data previously used in the learning of the operation model 401. For example, the display control unit 432 can display the previously set target range for a parameter selected by the operator as the target range setting object among various parameters related to the device 2. In addition, the display control unit 432 can further display at least a portion of the area included in the previously set target range as a recommended target range. For example, the display control unit 432 can display the range of the central part of the previously set target range at a predetermined proportion as a recommended target range. As an example, if the previously set target range for parameter Pa is 5 to 15 and the previously set target range for parameter Pb is 10 to 30, when the specified proportion is 90%, the display control unit 432 can set the recommended target range for parameter Pa to 6 to 14 and the recommended target range for parameter Pb to 12 to 28.

[0114] The display control unit 432 can display the previously set target range in a coordinate space with the parameters of the target range as coordinate axes. Additionally, the display control unit 432 can also display a recommended target range in the coordinate space. When the previously set target range is displayed in the coordinate space, the second learning processing unit 413 can obtain the specified range as the target range by specifying the range in the coordinate space using the input unit 411.

[0115] In step S103, the second learning processing unit 413 determines a reward function using target setting data. The second learning processing unit 413 can determine the reward function by increasing the reward value when the state related to the device 2 operated by the operation model 401 is close to the state corresponding to the content of the target setting data. Furthermore, the second learning processing unit 413 can determine the reward function by increasing the reward value when the state parameters related to the device 2 operated by the operation model 401 satisfy the content of the target setting data.

[0116] In step S105, the parameter acquisition unit 403 acquires status parameters related to the device 2. For example, the parameter acquisition unit 403 can acquire status parameters from the device 2 or the simulator 431.

[0117] In step S107, the second learning processing unit 413 determines an action and the control parameters corresponding to the determined action. For example, the second learning processing unit 413 determines the action randomly. Furthermore, the above description shows, as an example, the case where the second learning processing unit 413 randomly determines the action, but it is not limited to this. When the second learning processing unit 413 determines the action, for example, a known AI algorithm such as FKDPP (Factorial Kernel Dynamic Policy Programming) can be used. When using this kernel method, the second learning processing unit 413 generates a vector of states s based on the state data. Next, the second learning processing unit 413, for example, acts as... Figure 3 The action decision table shown generates a combination of state s and all possible actions a. Furthermore, the second learning processing unit 413 outputs the action decision table to, for example... Figure 2The operation model 401 is shown as input. Correspondingly, kernel calculations are performed between each row of the action decision table and each sample data point in the operation model 401 (excluding the weight column), calculating the distance between each sample data point and the action decision table. Furthermore, the expected reward value for each action is calculated by sequentially summing the values ​​obtained by multiplying the calculated distances for each sample data point by the value of the weight column. The operation model 401 selects the action with the highest expected reward value calculated in this way. The second learning processing unit 413 can, for example, determine the action with the highest expected reward value using the updated operation model. During learning, the second learning processing unit 413 can determine the action randomly or, while appropriately selecting whether to use the operation model 401 to determine the action, determine the action. The second learning processing unit 413 supplies control parameters corresponding to the determined action to the operation unit 402.

[0118] In step S109, the operation unit 402 operates the device 2 according to the supplied control parameters. The operation unit 402 can also cause the simulator 431 to perform simulations according to the supplied control parameters.

[0119] In step S111, the parameter acquisition unit 403 acquires state parameters related to the device 2. Thus, it acquires the state parameters corresponding to the changes after the device 2 has been operated using the determined control parameters. Furthermore, if a simulation is performed in step S109, the parameter acquisition unit 403 can acquire the state parameters from the simulator 431.

[0120] In step S113, the second learning processing unit 413 calculates the reward value based on the acquired parameters. The second learning processing unit 413 can calculate the reward value using the reward function determined in step S103.

[0121] In step S115, the second learning processing unit 413 determines whether the processing of acquiring parameters corresponding to the determination of control parameters has exceeded the specified number of steps. Furthermore, this number of steps can be specified in advance by the operator or based on the learning period (e.g., 10 days). If it is determined that the processing has not exceeded the specified number of steps (step S115; No), the second learning processing unit 413 returns the processing to step S107, allowing the process to continue. Thus, the processing of acquiring state parameters corresponding to the determination of control parameters is performed for the specified number of steps.

[0122] In step S115, if it is determined that the above processing has exceeded the specified number of steps (step S115; Yes), the second learning processing unit 413 proceeds to step S117. In step S117, the second learning processing unit 413 updates the operation model 401. For example, in addition to... Figure 2In addition to overwriting the values ​​of the weight column of the operation model shown, the second learning processing unit 413 also adds new sample data that has not been saved up to this point to the operation model 401.

[0123] In step S119, the second learning processing unit 413 determines whether the update process of the operation model 401 has exceeded the specified number of repetitions. Furthermore, this number of repetitions can be specified in advance by the operator or determined based on the suitability of the operation model 401. If it is determined that the above process has not exceeded the specified number of repetitions (step S119; No), the second learning processing unit 413 returns the process to step S105 and continues the process.

[0124] In step S119, if it is determined that the above process has exceeded the specified number of repetitions (step S119; Yes), the second learning processing unit 413 terminates the process. The second learning processing unit 413 can, for example, generate an operation model 401 that outputs control parameters corresponding to the state parameters involved in the device 2.

[0125] [3.2. Other learning actions of operation model 401]

[0126] Figure 5 Other learning actions of operation model 401 are indicated. After the learning process of target setting model 414 is completed, device 1 can generate operation model 401 through the processing of steps S121 to S123 and S103 to S119.

[0127] In step S121, the first supply unit 415 obtains the operation plan for the new input. The first supply unit 415 can obtain the operation plan for the new input to the input unit 411.

[0128] In step S123, the first supply unit 415 supplies the acquired operation plan to the target setting model 414. Thus, in response to the input of a new operation plan, the operation plan is supplied to the target setting model 414.

[0129] In step S125, the second supply unit 417 acquires output data from the target setting model (in this embodiment, as an example, parameter identification information and target range), and acquires target setting data corresponding to the output data. The second supply unit 417 can directly acquire the output data as target setting data.

[0130] The second supply unit 417 can replace this method by displaying the acquired output data as recommended target setting data to the display control unit 432, and acquiring target setting data input by the operator based on the displayed content. For example, if the output data is displayed with a target range of 1 to 10 for a certain parameter, and the operator inputs a target range of 2 to 9 for that parameter, the second supply unit 417 can acquire target setting data with a target range of 2 to 9 for that parameter.

[0131] After that, the learning actions of the operation model 401 can be performed in the same way as steps S103 to S119 above.

[0132] [3.3. Learning Actions of Goal Setting Model 414]

[0133] Figure 6 This represents the learning action of the target setting model 414. The device 1 can generate the target setting model 414 through the processing of steps S131 to S133.

[0134] In step S131, the first acquisition unit 421 acquires the operation plan of the device 2 and identification information of at least the parameters in the target setting data for learning the operation model 401. In this embodiment, as an example, the first acquisition unit 421 can acquire identification information of at least the parameters in the target setting data for learning the operation model 401, and the operation plan input by the input unit 411 associated with the target setting data. Furthermore, the first acquisition unit 421 can acquire both the identification information of the parameters in the target setting data and the target range. The first acquisition unit 421 can acquire the identification information of the parameters in the target setting data, the operation plan, etc., from the storage unit 412.

[0135] In step S133, the first learning processing unit 422 performs learning processing on the target setting model 414 using learning data including the identification information of the parameters acquired by the first acquisition unit 421 and the operation plan. The first learning processing unit 422 can perform learning processing on the target setting model 414 through teacher-assisted learning such as deep learning, or it can perform learning processing on the target setting model 414 in a manner that makes the content of the output data from the target setting model 414 similar to the content of the target setting data used for learning the operation model 401. Furthermore, in this embodiment, as an example, the first learning processing unit 422 can perform learning processing on the target setting model 414 in a manner that outputs the content of the target setting data that should be used for learning the operation model 401 to achieve the operation plan, corresponding to the input of the operation plan.

[0136] [3.4. Evaluation Actions of Operation Model 401]

[0137] Figure 7 This indicates the evaluation action of operation model 401. Device 1 can evaluate the generated operation model 401 through the processing of steps S141 to S175.

[0138] In step S141, the display control unit 432 displays the value of the selected parameter from previous operations of the device 2, corresponding to the operator's selection of any one of various parameters related to the device 2 as a selection parameter. For example, the display control unit 432 can display the values ​​of each selected parameter from previous operations of the device 2 in a coordinate space with each selected parameter as an axis. As an example, the display control unit 432 can display the values ​​of each selected parameter from previous operations of the device 2 separately, or it can display the range of values ​​of each selected parameter from previous operations of the device 2 by displaying the maximum and minimum values ​​of each selected parameter.

[0139] In step S143, the target range acquisition unit 433 acquires the evaluation target range set by the operator for the selected parameters. The target range acquisition unit 433 specifies the range in the coordinate space displayed by the display control unit 432 with each selected parameter as the coordinate axis, and acquires the specified range as the evaluation target range.

[0140] In step S145, the status parameter acquisition unit 403 acquires status parameters related to device 2 from simulator 431. Furthermore, when initially executing step S145, the state of device 2 can be a pre-set initial state.

[0141] In step S147, simulator 431 performs a simulation corresponding to manual operation. Simulator 431 can simulate the state of device 2 operated based on control parameters corresponding to manual operation.

[0142] In step S149, the simulator 431 determines whether the simulation has ended. For example, the simulator 431 may determine whether a simulation end instruction has been input via the input unit 411. If it is determined in step S149 that the simulation end has not been indicated (step S149; No), the process can proceed to step S145. If it is determined in step S149 that the simulation end has been indicated (step S149; Yes), the process can proceed to step S151.

[0143] In step S151, the evaluation value acquisition unit 434 acquires a benchmark evaluation value corresponding to the result of operating the device 2 manually. In this embodiment, as an example, the benchmark evaluation value may be calculated based on the result of manually inputting the simulator 431.

[0144] The benchmark evaluation value can be calculated based on whether the parameters related to the device 2, which has been operated according to control parameters corresponding to manual operation, fall within the evaluation target range. When evaluation target ranges are set for multiple parameters related to the device 2, the benchmark evaluation value can be calculated based on the proportion (i.e., b / a) of the number of parameters (a) for which evaluation target ranges are set, and the number of parameters (b) falling within the corresponding target range. Furthermore, the benchmark evaluation value can be calculated by the simulator 431 or by the evaluation value acquisition unit 434, which acquires the parameters related to the operated device 2 from the simulator 431.

[0145] In step S153, simulator 431 terminates the simulation corresponding to manual operation. The state of the simulated device 2 can be reset to its initial state by terminating the simulation.

[0146] In step S161, the status parameter acquisition unit 403 acquires status parameters related to device 2 from simulator 431. Furthermore, when initially executing step S161, the state of device 2 can be a pre-set initial state.

[0147] In step S163, the parameter acquisition unit 403 supplies the acquired state parameters to the operation model 401. As a result, recommended values ​​for the control parameters are output from the operation model 401.

[0148] In step S165, the operation unit 402 obtains the recommended values ​​of the control parameters output from the operation model 401.

[0149] In step S167, simulator 431 performs a simulation corresponding to the recommended values ​​of control parameters from operation model 401. Simulator 431 can simulate the state of device 2 operating based on the recommended values ​​of control parameters.

[0150] In step S169, simulator 431 determines whether the simulation has ended. If the simulation has not ended in step S169 (step S169; No), the process can proceed to step S161. If the simulation has ended in step S169 (step S169; Yes), the process can proceed to step S171.

[0151] In step S171, the evaluation value acquisition unit 434 acquires a model evaluation value corresponding to the result of operating the device 2 according to the recommended values ​​of the control parameters. In this embodiment, as an example, the model evaluation value can be calculated based on the result of inputting the recommended values ​​acquired by the operation unit 402 into the simulator 431 of the device 2. Similar to the baseline evaluation value, the model evaluation value can be calculated based on whether the parameters related to the device 2 operated according to the recommended values ​​of the control parameters fall within the target range for evaluation.

[0152] In step S173, simulator 431 terminates the simulation corresponding to the recommended values ​​of the control parameters. The state of the simulated device 2 can be reset to its initial state by terminating the simulation.

[0153] In step S175, the evaluation unit 435 evaluates the operational model 401 based on the model evaluation value and the benchmark evaluation value. In this embodiment, as an example, if the model evaluation value is greater than the benchmark evaluation value, the evaluation unit 435 can evaluate the operational model 401 as being in good condition.

[0154] Based on the above actions, the benchmark evaluation value is calculated based on the result of inputting the manual operation into the simulator 431 of device 2, so that the benchmark evaluation value can be obtained quickly without actually operating device 2.

[0155] Furthermore, since the model evaluation value is calculated based on the result of inputting the recommended values ​​of the control parameters obtained from the operation model 401 into the simulator 431 of the device 2, the model evaluation value can be obtained quickly without actually operating the device 2.

[0156] Furthermore, since benchmark evaluation values ​​and model evaluation values ​​are calculated based on the simulation results, it is possible to ensure that device 2 is in the same state before operation, whether it is operated manually or using operation model 401. Therefore, it is possible to judge the quality of the operation results obtained using operation model 401 with good accuracy.

[0157] Furthermore, when setting the target range for evaluation, the values ​​of each selection parameter in the previous operation of the device 2 are displayed in a coordinate space with each selection parameter selected by the operator as the coordinate axis. Therefore, it is easy to grasp the previous values ​​and ranges of the selection parameters and to easily set the target range for evaluation.

[0158] [3.5. Operation of Equipment 2]

[0159] Figure 8 This indicates the operation of device 2. Device 1 can operate device 2 through the processing in steps S181 to S191.

[0160] In step S181, the status parameter acquisition unit 403 acquires status parameters related to device 2. In step S183, the parameter acquisition unit 403 supplies the acquired status parameters to the operation model 401. As a result, recommended values ​​for control parameters are output from the operation model 401. In step S185, the operation unit 402 acquires the recommended values ​​for control parameters output from the operation model 401. In step S187, the operation unit 402 operates device 2 according to the recommended values ​​for control parameters from the operation model 401.

[0161] In step S189, the operation unit 402 determines whether the end of the operation has been indicated. If the end of the operation is not indicated in step S189 (step S189; No), the process can proceed to step S181. If the end of the operation is indicated in step S189 (step S189; Yes), the process proceeds to step S191, and the operation unit 402 terminates the operation of device 2.

[0162] [4. Variation of Target Setting Model 414 (1)]

[0163] Furthermore, in the above embodiment, the following situation was described: the target setting model 414 is subjected to learning processing using learning data including the operation plan, the identification information of parameters in the target setting data used for learning the operation model 401, and the target range. The target setting model 414, after learning processing, outputs the identification information of parameters in the target setting data to be used for learning the operation model 401, corresponding to the input of the operation plan. However, the combination of the content of the learning data, input data, and output data for the target setting model 414 is not limited to this.

[0164] For example, the target setting model 414 performs learning processing using learning data that includes identification information of parameters in the target setting data used for learning the operation model 401, including the operation plan. It can output only the identification information of parameters in the target setting data that should be used for learning the operation model 401, corresponding to the input of the operation plan, without outputting the target range. The learning data may include both the identification information of parameters in the target setting data used for learning the operation model and the target range. The target setting model 414 can output the identification information of a single parameter or the identification information of multiple parameters. When only the identification information of the parameters is output from the target setting model 414, the second supply unit 417 can cause the display control unit 432 to display the identification information of each output parameter, obtain the target range input by the operator for each parameter's identification information, generate target setting data representing the identification information and the target range, and supply it to the second learning processing unit 413.

[0165] Furthermore, the target setting model 414 can perform learning processing using learning data including the operation plan, the identification information of parameters for target setting data used for learning the operation model 401, and the target range. Corresponding to the operation plan and the identification information of the parameters of the setting objects to be set as target ranges, only the target range for that parameter in the target setting data to be used for learning the operation model 401 is output, without outputting the parameter identification information. In this case, the identification information of a single parameter of the setting object to be set as a target range can be input by the operator to the target setting model 414 via the input unit 411, and the target range of that single parameter can be output from the target setting model 414. Alternatively, the identification information of multiple parameters of the setting object to be set as a target range can be input by the operator to the target setting model 414 via the input unit 411, and the target range of each parameter can be output from the target setting model 414. The second supply unit 417 can generate target setting data representing the identification information of each parameter input to the target setting model 414 and the target range of each parameter output from the target setting model 414, and supply it to the second learning processing unit 413.

[0166] [5. Variation of Target Setting Model 414 (2)]

[0167] Furthermore, in the above embodiment, the case where the target setting model 414 is set as a single model has been described, but it is also possible to have multiple models with different functions.

[0168] Figure 9 This refers to the target setting model 414A involved in this variation. The target setting model 414A may have at least one parameter setting model 4141 and at least one target range setting model 4142. In this embodiment, as an example, the target setting model 414A has two parameter setting models 4141a and 4141b, and four target range setting models 4142a to 4142d. Each parameter setting model 4141 outputs identification information of the parameter of the setting object that should be set as the target range, corresponding to the input of the operation plan. Each target range setting model 4142 outputs the target range that should be set for the parameter, corresponding to the input of the operation plan and the identification information of the parameter of the setting object that should be set as the target range.

[0169] The parameter setting model 4141a can output identification information of the parameter (also called parameter Pa) of the object to be set to the target range in response to the input of the operation plan. In this embodiment, as an example, the parameter setting model 4141a supplies the operation plan and the identification information of parameter Pa to the target range setting model 4142a in response to the input of the operation plan from the first supply unit 415.

[0170] The target range setting model 4142a can output the upper limit of the target range to be set for a given parameter in response to the input of the operation plan and parameter identification information. In this embodiment, as an example, the target range setting model 4142a, in response to the input of the operation plan and parameter Pa identification information from the parameter setting model 4141a (which receives the operation plan from the first supply unit 415), sets the upper limit V of the target range for the operation plan, parameter Pa identification information, and parameter Pa. PaMAX The target range setting model 4142b is supplied. Alternatively, the operation plan can be directly input from the first supply unit 415 to the target range setting model 4142a, instead of inputting from the first supply unit 415 to the target range setting model 4142a via the parameter setting model 4141a. The same applies to the target range setting models 4142b-4142d and the parameter setting model 4141b, which will be described later.

[0171] The target range setting model 4142b can output the lower limit of the target range that should be set for the parameter, in response to the input of the operation plan and parameter identification information. In this embodiment, as an example, the target range setting model 4142b outputs the operation plan, the identification information of parameter Pa, and the upper limit V of the target range of parameter Pa, in response to the parameter setting model 4141a from the first supply unit 415, which inputs the operation plan. PaMAX Input the operation plan, the identification information of parameter Pa, and the upper and lower limits V of the target range of parameter Pa. PaMAX V PaMIN Supply to parameter setting model 4141b.

[0172] The parameter setting model 4141b can output the identification information of other parameters (also called parameters Pb) of the set object that should be set to the target range, corresponding to the input of the operation plan and the identification information of the parameter Pa of the set object that has been set to the target range. In this embodiment, as an example, the parameter setting model 4141b is input from the target range setting model 4142b, corresponding to the input of the operation plan, the identification information of parameter Pa, and the upper and lower limits V of the target range. PaMAX V PaMIN The operation plan, the identification information of parameter Pa, and the upper and lower limits V are included. PaMAX V PaMIN Different parameters Pa and Pb identification information are supplied to the target range setting model 4142c.

[0173] The target range setting model 4142c can output the upper limit of the target range that should be set for the parameter, in accordance with the input of the operation plan and parameter identification information. In this embodiment, as an example, the target range setting model 4142c corresponds to the operation plan, parameter Pa identification information, and upper and lower limit values ​​V of the target range supplied from the parameter setting model 4141b. PaMAX V PaMIN The identification information of parameter Pb, the operation plan, the identification information of parameter Pa, and the upper and lower limits V of the target range of parameter Pa. PaMAX V PaMIN The identification information of parameter Pb and the upper limit V of the target range of parameter Pb. PbMAX Supply to target range setting model 4142d.

[0174] The target range setting model 4142d can output the lower limit of the target range to be set for the parameter, corresponding to the input of the operation plan and parameter identification information. In this embodiment, as an example, the target range setting model 4142d corresponds to the operation plan, parameter Pa identification information, and upper and lower limit values ​​V of the target range supplied from the target range setting model 4142c. PaMAX V PaMIN The identification information of parameter Pb, and the upper limit V of the target range of parameter Pb. PbMAX The operation plan, the identification information of parameter Pa, and the upper and lower limits V of the target range of parameter Pa are included. PaMAX V PaMIN The identification information of parameter Pb, and the upper and lower limits V of the target range of parameter Pb. PbMAX V PbMIN Output.

[0175] Based on the target setting model 414A above, corresponding to inputting the operation plan into the parameter setting model 4141, the identification information of the parameters of the setting object that should be set within the target range is output. Therefore, it is possible to make the parameters with the set target range consistent with the parameters of the target setting data used for learning in the operation model 401.

[0176] Furthermore, corresponding to inputting the operation plan and the identification information of parameter Pa, which has been set to the target range, into the parameter setting model 4141b, the identification information of parameter Pb, which should be set to the target range, is output. Therefore, it is possible to make parameter Pb, which has been set to the target range, consistent with the parameter used in the target setting data along with parameter Pa during the learning of the operation model 401.

[0177] Additionally, the operation plan, the identification information of parameter Pa of the set object that has been set as the target range, and the upper and lower limits V of the target range are also included. PaMAX VPaMIN The parameter setting model 4141b is input to the parameter setting model, and the identification information of the parameter Pb, which should be set to the target range, is output. Therefore, the parameter Pb with the target range set can be compared with the identification information of the parameter Pa and the upper and lower limit values ​​V during the learning of the operation model 401. PaMAX V PaMIN The parameters used for setting the target data are consistent.

[0178] Furthermore, corresponding to inputting the identification information of parameters Pa and Pb, which should be set as the target range, into the target range setting model 4142, the target range that should be set for these parameters Pa and Pb is output. Therefore, it is possible to make the target range that should be set for parameters Pa and Pb consistent with the target range of the target setting data used for learning the operation model 401.

[0179] Furthermore, in accordance with the input operation plan for parameter setting model 4141a, the identification information of parameter Pa, which should be set as the target range, is output from parameter setting model 4141a. In accordance with the input operation plan for target range setting model 4142a and the input of the identification information of parameter Pa from parameter setting model 4141, the target range V that should be set for parameter Pa is output. PaMAX V PaMIN Output from target range setting model 4142a. Therefore, the parameters and target range of the target setting data can be obtained automatically and sequentially.

[0180] Furthermore, the parameter setting model 4141a of the target setting model 414A described above can be learned and processed by the first learning processing unit 422 using learning data that includes the identification information of the parameters acquired by the first acquisition unit 421 and the operation plan. Therefore, for the learning data used to learn the parameter setting model 4141a, the target range of the parameters can be omitted, thus making learning processing easier.

[0181] Furthermore, the first learning processing unit 422 can use learning data, including the identification information of the parameters acquired by the first acquisition unit 421, as well as the target range and operation plan, to perform learning processing on the parameter setting model 4141b and the target range setting models 4142a to 4142d. This allows the content of the output data from the models to be similar to the content of the target setting data used to learn the operation model 401 in order to achieve the operation plan.

[0182] [6. Other variations]

[0183] Furthermore, in the above embodiment, the case where the device 4 has an operation model 401 and a target setting model 414 has been described, but it may also not have either of these models. When the device 4 does not have an operation model 401 and a target setting model 414, learning processing can be performed on the operation model 401 and the target setting model 414 stored in an externally connected storage device, the operation model 401 stored in the externally connected storage device can be evaluated, and operation can be performed using the operation model 401 stored in the externally connected storage device.

[0184] Furthermore, while the device 4 has been described with a first acquisition unit 421 and a first learning processing unit 422 for learning the target setting model 414, these components may not be present. In this case, the device 4 can use the trained target setting model 414 to learn the operation model 401. Multiple devices 4 can share the trained target setting model 414, and each device 4 can perform different learning processes for the operation model 401.

[0185] Furthermore, the description has covered the case where the device 4 includes a second learning processing unit 413, which is required to perform learning processing of the operation model 401; however, this unit may not be present. In this case, the device 4 can perform learning processing of the target setting model 414 by acquiring identification information of parameters in the target setting data used for learning the operation model 401 from an external source.

[0186] In addition, the situation where the evaluation value acquisition unit 434 acquires the benchmark evaluation value and model evaluation value corresponding to the simulation result is explained, but it is also possible to acquire the benchmark evaluation value and model evaluation value corresponding to the actual operation result of the device 2.

[0187] In addition, the calculation of the benchmark evaluation value by the evaluation value acquisition unit 434 is explained, but it can also be stored in the device 4 as a fixed value in advance.

[0188] Furthermore, various embodiments of the present invention can be described with reference to flowchart diagrams and block diagrams, where a module can represent (1) a stage of the process of performing an operation, or (2) a part of a device that performs the operation. Specific stages and parts can be installed using dedicated circuits, programmable circuits supplied together with computer-readable instructions stored on a computer-readable medium, and / or processors supplied together with computer-readable instructions stored on a computer-readable medium. Dedicated circuits can include digital and / or analog hardware circuits, and can also include integrated circuits (ICs) and / or discrete circuits. Programmable circuits can include reconfigurable hardware circuits such as logic AND, logic OR, logic XOR, logic NAND, logic NOR, and other logic operations, trigger circuits, registers, field-programmable gate arrays (FPGAs), programmable logic arrays (PLAs), etc., and memory elements.

[0189] A computer-readable medium can include any tangible device capable of storing instructions that can be executed using appropriate means, resulting in a product having instructions stored therein containing executable instructions for making means for performing operations specified in a flowchart or block diagram. Examples of computer-readable media include electronic storage media, magnetic storage media, optical storage media, electromagnetic storage media, semiconductor storage media, etc. More specific examples of computer-readable media include floppy disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), electrically erasable programmable read-only memory (EEPROM), static random access memory (SRAM), optical disc read-only memory (CD-ROM), digital versatile optical disc (DVD), Blu-ray (RTM) discs, memory sticks, integrated circuit cards, etc.

[0190] Computer-readable instructions may include any combination of source code or object code written in one or more programming languages, including object-oriented programming languages ​​such as Smalltalk (registered trademark), JAVA (registered trademark), C++, and existing procedural programming languages ​​such as "C" or similar programming languages.

[0191] Computer-readable instructions can be provided to the processor or programmable circuitry of a general-purpose computer, special-purpose computer, or other programmable data processing device via a local area network (LAN), wide area network (WAN), or the Internet, and executed to create means for performing the operations specified in a flowchart or block diagram. Examples of processors include computer processors, processing units, microprocessors, digital signal processors, controllers, microcontrollers, etc.

[0192] Figure 10 Examples of computer 2200 that can embody all or part of the various embodiments of the present invention are shown. Programs installed in computer 2200 enable computer 2200 to function as an operation associated with a device or one or more parts of such device according to embodiments of the present invention, or to perform such operation or such parts, and / or to perform processes or stages of processes according to embodiments of the present invention. Such programs may be executed by CPU 2212 to enable computer 2200 to perform specific operations associated with several or all of the modules in the flowcharts and block diagrams described in this specification.

[0193] The computer 2200 according to this embodiment includes a CPU 2212, RAM 2214, a graphics controller 2216, and a display device 2218, which are interconnected via a main controller 2210. Additionally, the computer 2200 includes input / output units such as a communication interface 2222, a hard disk drive 2224, a DVD-ROM drive 2226, and an IC card driver, which are connected to the main controller 2210 via an input / output controller 2220. Furthermore, the computer includes conventional input / output units such as a ROM 2230 and a keyboard 2242, which are connected to the input / output controller 2220 via an input / output chip 2240.

[0194] CPU 2212 executes actions according to the programs stored in ROM 2230 and RAM 2214, thereby controlling each unit. Graphics controller 2216 acquires image data generated by CPU 2212 in frame buffer or other storage provided to RAM 2214, and displays the image data on display device 2218.

[0195] Communication interface 2222 communicates with other electronic devices via a network. Hard disk drive 2224 stores programs and data used by CPU 2212 within computer 2200. DVD-ROM drive 2226 reads programs or data from DVD-ROM 2201 and provides the programs or data to hard disk drive 2224 via RAM 2214. IC card drive reads programs and data from IC card and / or writes programs and data to IC card.

[0196] ROM 2230 stores startup programs and / or programs that depend on the hardware of computer 2200, which are executed by computer 2200 upon activation. Additionally, input / output chip 2240 connects various input / output units to input / output controller 2220 via parallel ports, serial ports, keyboard ports, mouse ports, etc.

[0197] The program is provided by a computer-readable medium such as a DVD-ROM 2201 or an IC card. The program is read from the computer-readable medium and installed on a hard disk drive 2224, RAM 2214, or ROM 2230, which are also examples of computer-readable media, and executed by the CPU 2212. The information processing described in the program is read by the computer 2200, enabling cooperation between the program and the aforementioned hardware resources of various types. An apparatus or method can be constituted by using the computer 2200 to perform information manipulation or processing.

[0198] For example, when communication is performed between computer 2200 and external devices, CPU 2212 can execute a communication program loaded in RAM 2214 and issue communication processing instructions to communication interface 2222 based on the processing described in the communication program. Under the control of CPU 2212, communication interface 2222 reads transmission data stored in the transmission buffer processing area provided in recording media such as RAM 2214, hard disk drive 2224, DVD-ROM 2201, or IC card, and sends the read transmission data to the network, or writes received data received from the network to the receive buffer processing area provided on the recording medium, etc.

[0199] In addition, CPU 2212 can read all or a portion of files or databases stored on external recording media such as hard disk drive 2224, DVD-ROM drive 2226 (DVD-ROM 2201), IC card, etc., into RAM 2214, and perform various types of processing on the data in RAM 2214. CPU 2212 then writes the processed data back to the external recording media.

[0200] Various types of information, such as programs, data, tables, and databases, can be stored in recording media and processed. The CPU 2212 can perform various types of processing, including operations, information processing, conditional judgments, conditional branches, unconditional branches, and information retrieval / replacement, as specified by the program's instruction sequence described at any location of this disclosure, on data read from RAM 2214, and write back the results to RAM 2214. Furthermore, the CPU 2212 can retrieve information from files, databases, etc., within the recording medium. For example, when multiple entries, each having an attribute value of a first attribute associated with a second attribute value, are stored in the recording medium, the CPU 2212 can retrieve from these multiple records an entry whose first attribute value matches a condition, read the attribute value of the second attribute stored within that entry, and obtain the attribute value of the second attribute associated with the first attribute that satisfies the pre-defined condition.

[0201] The programs or software modules described above can be stored on or near computer-readable media on computer 2200. Alternatively, recording media such as hard disks or RAM provided to a server system connected to a dedicated communication network or the Internet can be used as computer-readable media, thereby providing the programs to computer 2200 via the network.

[0202] The present invention has been described above using embodiments, but the technical scope of the present invention is not limited to the scope described in the above embodiments. Those skilled in the art will understand that various modifications or improvements can be made to the above embodiments. As clearly understood from the claims, such modifications or improvements can also be included within the technical scope of the present invention.

[0203] Regarding the execution order of actions, sequence, steps, and stages in the apparatus, system, program, and method shown in the claims, specification, and drawings, it should be noted that, unless explicitly stated as "earlier" or "earlier," and the output of a previous process is not used in a subsequent process, they can be implemented in any order. Even if terms such as "firstly," "nextly," etc., are used for convenience in describing the flow of actions in the claims, specification, and drawings, it does not mean that they must be performed in that order.

[0204] Explanation of the label

[0205] 1 System

[0206] 2 Equipment

[0207] 4 devices

[0208] 401 Operation Model

[0209] 402 Operations Department

[0210] 403 Parameter Acquisition Department

[0211] 411 Input Section

[0212] 412 Storage Section

[0213] 413 Second Learning Processing Department

[0214] 414 Goal Setting Model

[0215] 415 First Supply Department

[0216] 416 Second Acquisition Department

[0217] 417 Second Supply Department

[0218] 421 First Acquisition Department

[0219] 422 First Learning Processing Department

[0220] 431 Simulator

[0221] 432 Display Control Unit

[0222] 433 Target Range Acquisition Department

[0223] 434 Evaluation Value Acquisition Department

[0224] 435 Evaluation Department

[0225] 2200 Computers

[0226] 2201 DVD-ROM

[0227] 2210 Main Controller

[0228] 2212 CPU

[0229] 2214 RAM

[0230] 2216 Graphics Controller

[0231] 2218 Display Device

[0232] 2220 Input / Output Controller

[0233] 2222 Communication Interface

[0234] 2224 Hard Drive

[0235] 2226 DVD-ROM drive

[0236] 2230 ROM

[0237] 2240 Input / Output Chip

[0238] 2242 Keyboard

[0239] 4141 Parameter Setting Model

[0240] 4142 Target Range Setting Model

Claims

1. An apparatus, wherein, The device has: The supply department supplies the values ​​of the state parameters of the equipment to the operation model, which is a model for operating the equipment. The operation model outputs recommended values ​​of the control parameters of the equipment in response to the input of the values ​​of the state parameters. An operation unit acquires a recommended value of a control parameter output from the operation model corresponding to the value of the state parameter supplied by the supply unit to the operation model, and operates the device accordingly based on the recommended value of the control parameter acquired from the operation model. A simulator that obtains the recommended value of the control parameters from the operation unit and simulates the operation of the device based on the recommended value of the control parameters, wherein the supply unit obtains the status parameters of the device from the simulator; The acquisition unit acquires a model evaluation value corresponding to the result of the simulator simulating the operation of the device based on the recommended values ​​of the acquired control parameters, and acquires a benchmark evaluation value corresponding to the result of the simulator simulating the operation of the device based on control parameters corresponding to manual operation of the simulator; and The evaluation department evaluates the operational model based on the model evaluation value and the benchmark evaluation value.

2. The apparatus according to claim 1, wherein, The benchmark evaluation value is calculated based on the results of inputting manual operations into the simulator of the device.

3. The apparatus according to claim 1 or 2, wherein, The model evaluation value is calculated based on whether the parameters associated with the device that operated according to the recommended value fall within the target range. The benchmark evaluation value is calculated based on whether the parameters related to the equipment operated manually fall within the target range.

4. The apparatus according to claim 3, wherein, The device also includes a target range acquisition unit that acquires the target range set by the operator for a selection parameter selected by the operator among a variety of parameters related to the device.

5. The apparatus according to claim 4, wherein, The device also includes a display control unit that displays the value of the selection parameter for previous operations of the device, corresponding to selecting the selection parameter from the plurality of parameters.

6. The apparatus according to claim 5, wherein, The display control unit displays the values ​​of each selection parameter from previous operations of the device in a coordinate space with each selection parameter as the coordinate axis.

7. The apparatus according to any one of claims 4 to 6, wherein, The equipment is used for manufacturing articles. The parameters associated with the equipment are at least one of an indicator value representing the quality of the article or the production quantity of the article.

8. The apparatus according to claim 1 or 2, wherein, The device also includes a learning processing unit that performs learning processing of the operation model using learning data including the values ​​of state parameters and control parameters.

9. The apparatus according to claim 8, wherein, The learning processing unit performs learning processing of the operation model using the learning data and the reward value specified by a pre-set reward function.

10. A method, wherein, The method has the following stages: During the supply phase, the supply unit supplies the values ​​of the state parameters of the equipment to the operation model, which is a model for operating the equipment. The operation model outputs recommended values ​​of the control parameters of the equipment in accordance with the input of the state parameters. In the control parameter acquisition phase, recommended values ​​of control parameters output from the operation model are acquired in accordance with the values ​​of the state parameters supplied to the operation model. A phase in which the device is operated in accordance with the recommended values ​​of the control parameters obtained from the operation model; During the simulation phase, a simulator is used to simulate the operation of the device based on the recommended values ​​of the control parameters, wherein the supply unit obtains the status parameters of the device from the simulator; In the acquisition phase, a model evaluation value corresponding to the result of the simulator simulating the operation of the device based on the recommended values ​​of the acquired control parameters is acquired; and a benchmark evaluation value corresponding to the result of the simulator simulating the operation of the device based on control parameters corresponding to manual operation of the simulator is acquired; and During the evaluation phase, the operational model is evaluated based on the model evaluation value and the benchmark evaluation value.

11. A computer-readable medium having a program recorded thereon that enables a computer to function as a functional unit: The supply department supplies the values ​​of the state parameters of the equipment to the operation model, which is a model for operating the equipment. The operation model outputs recommended values ​​of the control parameters of the equipment in response to the input of the values ​​of the state parameters. An operation unit acquires a recommended value of a control parameter output from the operation model corresponding to the value of the state parameter supplied by the supply unit to the operation model, and operates the device accordingly based on the recommended value of the control parameter acquired from the operation model. A simulator that obtains the recommended value of the control parameters from the operation unit and simulates the operation of the device based on the recommended value of the control parameters, wherein the supply unit obtains the status parameters of the device from the simulator; The acquisition unit acquires a model evaluation value corresponding to the result of the simulator simulating the operation of the device based on the recommended values ​​of the acquired control parameters, and acquires a benchmark evaluation value corresponding to the result of the simulator simulating the operation of the device based on control parameters corresponding to manual operation of the simulator; and The evaluation department evaluates the operational model based on the model evaluation value and the benchmark evaluation value.

Images