Machine learning device, production plan determination device, and inference device

Abstract

To provide a machine learning device capable of relieving a workload required for the formulation of a production plan.SOLUTION: A machine learning device 4 produces a learning model 11 used in a production plan formulation device 5 for a product produced by a reaction apparatus which performs a prescribed reaction process under a driving operation by a worker. The machine learning device 4 includes: a learning data storage section 41 which stores a plurality of sets of learning data including at least items of the product as input data; a machine learning section 42 which causes the learning model 11 to learn a relative relation of the input data and allocated data indicating the allocation of a production resource required for production of items by inputting the plurality of sets of learning data into the learning model 11; and a learned model storage section 43 which stores a learning model 42 learned by the machine learning section 42.SELECTED DRAWING: Figure 6

Description

Technical Field

[0001] The present invention relates to a machine learning device, a production plan determination device, and an inference device.

Background Art

[0002] As a production facility for producing a product from raw materials by a predetermined reaction process, a reaction device including a reaction tank is widely used. As an example of the reaction device, Patent Document 1 discloses a reaction device including a reaction tank, a temperature adjustment jacket provided on the outer periphery of the reaction tank, a stirring blade disposed inside the reaction tank, and a stirring power serving as a power source for the stirring blade. In the reaction device as disclosed in Patent Document 1, for example, manual operation by an operator is performed according to a standard production process table in which the input amount and input order of raw materials, temperature adjustment of the reaction tank, stirring speed, reaction time, etc. are determined.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In a production factory where a plurality of reaction devices for performing a reaction process under the operation of an operator are installed to produce a plurality of product items, a work shift is organized for a plurality of operators to work in, for example, a three-shift system (day shift, quasi-night shift, night shift). In the production factory, in order to timely respond to a production order specifying an item, it is necessary to formulate a production plan in which production resources such as reaction devices and operators are allocated to each production order.

[0005] However, when formulating a production plan, there are various factors that depend on the site of the production factory. Therefore, in order for the manager of the production factory to determine the allocation of production resources, these factors must be carefully considered, and the production planning work has required a great deal of effort. In particular, as production resources, when determining the allocation of a reactor, factors related to the combination of the product item and the reactor need to be considered. When determining the allocation of workers, factors related to the combination of the product item and the worker, the work skills of the worker, and the balanced distribution of work shifts need to be considered. As production resources, when simultaneously determining the allocation of a reactor and workers, in addition to the above, factors related to the combination of the product item, the reactor, and the worker need to be considered. Therefore, it has been very difficult to formulate an appropriate production plan that improves production quality and production efficiency.

[0006] The present invention has been made in view of the above problems, and an object thereof is to provide a machine learning device, a production plan formulation device, and an inference device that can reduce the workload required for formulating a production plan.

Means for Solving the Problems

[0007] In order to achieve the above object, a machine learning device according to an aspect of the present invention is a machine learning device that generates a learning model used in a production plan formulation device for a product produced by a reaction device that performs a predetermined reaction process under the operation of an operator, a learning data storage unit that stores a plurality of sets of learning data including at least the product item of the product as input data, a machine learning unit that causes the learning model to learn the correlation between the input data and allocation data indicating the allocation of production resources required for producing the item by inputting a plurality of sets of the learning data into the learning model, and a learned model storage unit that stores the learning model learned by the machine learning unit.

Effects of the Invention

[0008] According to the machine learning device of the present invention, it is possible to provide a learning model capable of inferring the allocation of production resources based on the items of products. Therefore, by using this learning model, it is possible to reduce the workload required for formulating a production plan.

[0009] Problems, configurations, and effects other than those described above will be clarified in the form for implementing the invention described later.

Brief Description of the Drawings

[0010]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Embodiment for Carrying Out the Invention

[0011] Hereinafter, embodiments for carrying out the present invention will be described with reference to the drawings. In the following, the scope necessary for the description for achieving the object of the present invention will be schematically shown, and mainly the scope necessary for the description of the corresponding part of the present invention will be described, and the parts where the description is omitted shall be based on known techniques.

[0012] FIG. 1 is a schematic overall view showing an example of the production management system 1. The production management system 1 assigns the production resources of the production factory 10 to a production order, and under the operation of workers 31, 32, 33, …, 3 N (hereinafter abbreviated as “worker 3”), a reaction plan for the product produced by the reaction apparatuses 21, 22, 23, …, 2 M (hereinafter abbreviated as “reaction apparatus 2”) is formulated, and production management is performed in accordance with the production plan.

[0013] For the production order, the item of the product is specified, and the production quantity and delivery date of the product are also specified.

[0014] In the production factory 10, a plurality of reaction apparatuses 2 are installed, and the operation of the reaction apparatuses 2 is respectively performed by a plurality of workers 3. The production resources are physical resources and human resources necessary for producing a product from raw materials. A representative example of physical resources is the reaction apparatus 2. A representative example of human resources is the worker 3.

[0015] The reaction apparatus 2 produces a product by performing a batch or continuous reaction process on raw materials. The reaction apparatus 2 is used, for example, in the chemical field for producing synthetic resins and the like in a polymerization process, the food field for producing sake, soy sauce and the like in a brewing process, and the pharmaceutical field for producing drugs, vaccines and the like in a culture reaction. Note that the reaction apparatus 2 is not limited to the above examples, and may produce any product and be used in various fields.

[0016] In this embodiment, the reaction apparatus 2 will be mainly described in the case of producing synthetic resins (for example, polypropylene, polyethylene, polyvinyl chloride, melamine resin, epoxy resin, urethane resin, acrylic resin, and silicone resin, etc.) by performing a batch polymerization process (a form of reaction process) using one or more types of monomers, solvents, polymerization initiators, additives, etc. as raw materials.

[0017] Operator 3 performs the operation of the reaction apparatus 2. Specifically, operator 3 performs a production process of manually operating each part of the reaction apparatus 2 while charging raw materials into the reaction apparatus 2 and checking the progress of the reaction process, and a changeover process of cleaning the reaction apparatus 2 or preparing for the next production process after the production process ends. Operator 3 operates the reaction apparatus 2, for example, in a three-shift system (day shift, swing shift, night shift). Therefore, when the production process including the reaction process spans multiple work shifts, multiple operators 3 take turns to perform the operation of one reaction apparatus 2.

[0018] The production management system 1 mainly includes a machine learning device 4, a production plan formulation device 5, a production management database device 6, and a production simulation device 7. The machine learning device 4, the production plan formulation device 5, the production management database device 6, and the production simulation device 7 are configured by, for example, general-purpose or dedicated computers (see FIG. 5 described later), and are connected to a wired or wireless network 8 so as to be able to transmit and receive various data to and from each other.

[0019] The machine learning device 4 operates as the main body in the learning phase of machine learning and generates a learning model 11 used in the production plan formulation device 5 by machine learning. The learned learning model 11 is provided to the production plan formulation device 5 via the network 8, a recording medium, etc. The machine learning device 4 adopts, for example, reinforcement learning or supervised learning as the machine learning method. In this embodiment, the case of adopting reinforcement learning will be mainly described.

[0020] The production plan determination device 5 operates as the main body in the inference phase of machine learning, and uses the learning model 11 generated by the machine learning device 4 to determine the production plan for the product produced by the reaction device 2. The production plan determination device 5 receives a production order and determines the production plan for the product by allocating production resources (in this embodiment, the reaction device 2 and the operator 3) to the production order.

[0021] The production management database device 6 includes a production management database 60 (see FIG. 3 described later) that stores various data necessary for creating a production plan.

[0022] The production simulation device 7 is configured to model the reaction process and execute a simulation of the reaction process. For example, when the reaction device 2 and the operator 3 are allocated as production resources for producing a specified item product, the production simulation device 7 executes a simulation of the reaction process when the reaction device 2 produces the item under the operation of the operator 3, and estimates the production evaluation index when the product is produced.

[0023] FIG. 2 is a process management diagram showing an example of a production plan. The process management diagram represents the production plan formulated for a plurality of production orders in a Gantt chart format. In the example of FIG. 2, the time is arranged on the horizontal axis and the reaction device 2 is arranged on the vertical axis, and the allocation of the reaction device 2 and the operator 3 for each item P1 to P4 is shown. The allocation of the operator 3 is represented by the characters "31 to 36" described in the rectangle.

[0024] FIG. 3 is a data configuration diagram showing an example of the production management database 60. The production management database 60 is composed of, for example, a production order table 600, a product table 601, a reaction device table 602, an operator table 603, and a production management table 604.

[0025] The production order table 600 has records for each production order specified by a production order ID. In each record, an item specified by an item ID, a production quantity, and a delivery date are registered.

[0026] The product table 601 has records for each item specified by the item ID of the product. In each record, an item name, raw materials, a production process table (input quantity and order of raw materials, temperature control of the reaction tank, stirring speed, reaction time, etc.), and a target value of the production evaluation index are registered. Multiple types of raw materials may be registered as raw materials, and a polymerization initiator, an additive, etc. may be further registered.

[0027] The reaction apparatus table 602 has records for each reaction apparatus 2 specified by a reaction apparatus ID. In each record, the installation location of the reaction apparatus 2, the specifications of the reaction apparatus 2 (size, shape, material, etc.), and the item of the product that has a production record in the past are registered. Note that multiple reaction apparatuses 2 may have the same specifications, or some or all of the specifications may be different.

[0028] The worker table 603 has records for each worker 3 specified by a worker ID. In each record, a work shift up to a predetermined period in the future and the item of the product that has a production record in the past are registered.

[0029] The production management table 604 has records for each production management information specified by a production management ID. In each record, a production order ID, a reaction apparatus ID, a worker ID, a scheduled start time, a scheduled end time, an actual start time, an actual end time, and an actual value of the production evaluation index are registered. The reaction apparatus ID, the worker ID, the scheduled start time, and the scheduled end time are information representing a production plan, and the actual start time, the actual end time, and the actual value of the production evaluation index are information representing a production result. Based on the production management information registered in each record of the production management table 604, the process management diagram shown in FIG. 2 can be created.

[0030] The production evaluation index is any index that represents the production quality and production efficiency of the product. The actual value of the production evaluation index is, for example, a measured value obtained by measuring the product produced by the reaction device 2 with measuring instruments such as a weighing scale, a viscometer, and a densitometer, or an evaluation value calculated based on a plurality of measured values. The target value of the production evaluation index is predetermined as the target value for the measured value or the evaluation value. Further, the actual value of the production evaluation index is the production actual time obtained from the difference between the actual start time and the actual end time, and the target value of the production evaluation index is predetermined as the production target time for the production actual time.

[0031] Figure 4 is a schematic configuration diagram showing an example of the reaction device 2. The reaction device 2 mainly includes a reaction tank 20, a stirrer 21, a sensor group 22, a control device group 23, and an operation display panel 24. Power from a commercial power supply (not shown) is supplied to each part of the reaction device 2.

[0032] The reaction tank 20 includes a reaction tank main body 200 having a substantially vertically long cylindrical shape, a raw material inlet 201 formed on the upper surface portion of the reaction tank main body 200, a product outlet 202 formed on the bottom surface portion of the reaction tank main body 200, and a jacket 203 provided on the outer periphery of the reaction tank main body 200 through which cold water and warm water as heat mediums flow. The reaction tank 20 further includes a cold water circulation flow path 204 for circulating cold water and a warm water circulation flow path 205 for circulating warm water. Note that a viscometer for measuring the viscosity of the product or a densitometer for measuring the density of the product may be provided at the product outlet 202.

[0033] The stirrer 21 includes a propeller-shaped stirring blade 210 disposed inside the reaction tank main body 200, a motor 211 disposed above the reaction tank main body 200 as a drive source for rotating the stirring blade 210, and a shaft-shaped rotating shaft 212 connecting between the stirring blade 210 and the motor 211.

[0034] The sensor group 22 includes a reaction tank sensor group 220 for measuring physical quantities and state quantities of each part of the reaction tank 20, a stirrer sensor group 221 for measuring physical quantities and state quantities of each part of the stirrer 21, and an environment sensor group 222 for measuring physical quantities and state quantities of the environment where the reaction device 2 is installed.

[0035] The reaction tank sensor group 220 includes a jacket heat medium temperature sensor 220A for measuring the heat medium temperature T4, a cold water temperature sensor 220B for measuring the cold water temperature T5, a warm water temperature sensor 220C for measuring the warm water temperature T6, a jacket heat medium flow rate sensor 220D for measuring the heat medium flow rate F1, a cold water flow rate sensor 220E for measuring the cold water flow rate F2, a warm water flow rate sensor 220F for measuring the warm water flow rate F3, three reaction tank temperature sensors 220G for measuring the reaction tank temperatures T1, T2, and T3 respectively, a reaction tank pressure sensor 220H for measuring the reaction tank pressure P1, and a reaction tank weight sensor 220I for measuring the reaction tank weight W1 indicating the weight of the raw materials inside the reaction tank body 200.

[0036] The agitator sensor group 221 includes an agitator torque sensor 221A for measuring the agitator torque ST1 indicating the torque applied to the motor 211, a motor rotation speed sensor 221B for measuring the motor rotation speed R1, a motor current sensor 221C for measuring the motor current value I1, a vibration sensor 221D for measuring the vibration value O1 when the agitator 21 operates, and an acoustic sensor 221E for measuring the acoustic value N1 when the agitator 21 operates.

[0037] The environment sensor group 222 includes an environment temperature sensor 222A for measuring the environment temperature T7 and an environment humidity sensor 222B for measuring the environment humidity H1.

[0038] The control device group 23 includes a cooling device 230 for controlling the cold water temperature T5, a heating device 231 for controlling the warm water temperature T6, a cold water flow rate adjustment valve 232 and a cold water pump 233 for controlling the cold water flow rate F2, a warm water flow rate adjustment valve 234 and a warm water pump 235 for controlling the warm water flow rate F3, and an inverter 236 for supplying driving power to the motor 211 and controlling the rotation state (on or off, rotation speed, etc.) of the motor 211.

[0039] The operation display panel 24 is electrically connected to each part of the reaction device 2. The operation display panel 24 displays each measured value measured by the sensor group 22, and accepts the operation operations of the operator 3, and outputs each control command value corresponding to the operation operations to the control device group 23.

[0040] In addition, each measured value by the sensor group 22 may be directly displayed on the meters and instruments of the sensor group 22 instead of the operation display panel 24. Further, the operation operation on the control device group 23 may be directly performed on the switches, levers, etc. of the control device group 23 instead of the operation display panel 24. Furthermore, the operation display panel 24 may record each measured value by the sensor group 22 and each control command value corresponding to the operation operation on the control device group 23 as operation history data, and when the reaction process is completed, the results of measuring the viscosity and density of the product with a viscometer and a densitometer may be recorded as the actual values of the production evaluation indicators.

[0041] Figure 5 is a hardware configuration diagram showing an example of the computer 900. The machine learning device 4 , the production plan determination device 5, the production management database device 6, and the production simulation device 7 are each composed of a general-purpose or dedicated computer 900.

[0042] As shown in Figure 5, the computer 900 mainly includes a bus 910, a processor 912, a memory 914, an input device 916, an output device 917, a display device 918, a storage device 920, a communication I / F (interface) unit 922, an external device I / F unit 924, an I / O (input / output) device I / F unit 926, and a media input / output unit 928. Note that the above components may be appropriately omitted according to the application for which the computer 900 is used.

[0043] The processor 912 is composed of one or more arithmetic processing units (such as a CPU (Central Processing Unit), MPU (Micro-processing unit), DSP (digital signal processor), GPU (Graphics Processing Unit), etc.), and operates as a control unit that oversees the entire computer 900. The memory 914 stores various data and programs 930, and is composed of, for example, a volatile memory (such as DRAM, SRAM, etc.) that functions as a main memory, a non-volatile memory (ROM), a flash memory, etc.

[0044] The input device 916 is composed of, for example, a keyboard, a mouse, a numeric keypad, an electronic pen, etc., and functions as an input unit. The output device 917 is composed of, for example, a sound (voice) output device, a vibration device, etc., and functions as an output unit. The display device 918 is composed of, for example, a liquid crystal display, an organic EL display, an electronic paper, a projector, etc., and functions as an output unit. The input device 916 and the display device 918 may be integrally configured, such as a touch panel display. The storage device 920 is composed of, for example, an HDD (Hard Disk Drive), an SSD (Solid State Drive), etc., and functions as a storage unit. The storage device 920 stores various data necessary for the execution of the operating system and the program 930.

[0045] The communication I / F unit 922 is connected to a network 940 such as the Internet or an intranet (which may be the same as the network 8 in FIG. 1) either wired or wirelessly, and functions as a communication unit that transmits and receives data to and from other computers according to a predetermined communication standard. The external device I / F unit 924 is connected to an external device 950 such as a camera, printer, scanner, reader / writer, etc. either wired or wirelessly, and functions as a communication unit that transmits and receives data to and from the external device 950 according to a predetermined communication standard. The I / O device I / F unit 926 is connected to an I / O device 960 such as various sensors and actuators, and functions as a communication unit that transmits and receives various signals and data such as detection signals from sensors and control signals to actuators to and from the I / O device 960. The media input / output unit 928 is composed of, for example, drive devices such as a DVD (Digital Versatile Disc) drive and a CD (Compact Disc) drive, and reads and writes data to a media 970 such as a DVD and a CD (non-volatile storage medium).

[0046] In the computer 900 having the above configuration, the processor 912 calls and executes the program 930 stored in the storage device 920 in the memory 914, and controls each part of the computer 900 via the bus 910. Note that the program 930 may be stored in the memory 914 instead of the storage device 920. The program 930 may be recorded in the media 970 in an installable file format or an executable file format, and provided to the computer 900 via the media input / output unit 928. The program 930 may be provided to the computer 900 by downloading it via the network 940 through the communication I / F unit 922. Also, the computer 900 may implement various functions realized when the processor 912 executes the program 930 by hardware such as, for example, an FPGA (f ield-programmable gate array) and an ASIC (application specific integrated circuit).

[0047] The computer 900 is composed of, for example, a stationary computer or a portable computer, and is an electronic device in any form. The computer 900 may be a client computer, or may be a server computer or a cloud computer. The computer 900 may also be applied to other devices other than the machine learning device 4, the production plan determination device 5, the production management database device 6, and the production simulation device 7.

[0048] (Machine learning device 4) FIG. 6 is a block diagram showing an example of the machine learning device 4. The machine learning device 4 includes a learning data acquisition unit 40, a learning data storage unit 41, a machine learning unit 42, and a learned model storage unit 43. The machine learning device 4 is composed of, for example, the computer 900 shown in FIG. 5. In that case, the learning data acquisition unit 40 is composed of an input device 916, a communication I / F unit 922, or an I / O device I / F unit 926, the machine learning unit 42 is composed of a processor 912, and the learning data storage unit 41 and the learned model storage unit 43 are composed of a storage device 920.

[0049] The learning data acquisition unit 40 is an interface unit that is connected to various external devices via the network 8 and acquires learning data that at least includes the item of the product as input data. The external devices are the production management database device 6, the production simulation device 7, etc. Note that the external devices may be a part of these, or other devices may be further connected.

[0050] The learning data storage unit 41 is a database that stores a plurality of sets of learning data acquired by the learning data acquisition unit 40. The specific configuration of the database constituting the learning data storage unit 41 may be designed as appropriate.

[0051] The machine learning unit 42 performs machine learning using the learning data stored in the learning data storage unit 41. That is, the machine learning unit 42 inputs a plurality of sets of learning data into the learning model 11, and causes the learning model 11 to learn the correlation between the input data included in the learning data and the allocation data indicating the allocation of production resources necessary for producing the product items of the products included in the input data, thereby generating a learned learning model 11. In the present embodiment, the case where a neural network is adopted as the learning model 11 for realizing the machine learning by the machine learning unit 42 will be described.

[0052] The learned model storage unit 43 is a database that stores the learned learning model 11 generated by the machine learning unit 42. The learned learning model 11 stored in the learned model storage unit 43 is provided to the actual system (for example, the production plan determination device 5) via the network 8, a recording medium, or the like. In FIG. 6, the learning data storage unit 41 and the learned model storage unit 43 are shown as separate storage units, but these may be configured as a single storage unit.

[0053] FIG. 7 is a schematic diagram showing an example of data used in the machine learning device 4 and the relationship with reinforcement learning. The machine learning unit 42 functions as an agent for reinforcement learning. In the basic mechanism of reinforcement learning, the agent observes the state of the environment under predetermined conditions and selects an action according to a predetermined policy for the observed state. Then, when the state of the environment changes due to the selected action, the agent receives a reward corresponding to the change in the state and evaluates the value of the selected action. As such a series of processes, the observation of the state, the selection of the action, and the evaluation of the value By repeating the evaluation, the learning model 11 is made to learn a policy for selecting an action so as to obtain the most rewards.

[0054] When the reinforcement learning by the machine learning unit 42 is made to correspond to the basic mechanism of the above-described reinforcement learning, the environment corresponds to a production factory 10 that produces a product so as to satisfy a production order by operating M (M is an integer of 2 or more) reaction apparatuses 2 by N (N is an integer of 2 or more) workers 3.

[0055] The state s is represented by a production order received at the production factory 10 and an operating state of production resources. The production order is at least one in which the item of the product is specified, and the item of the product specified in the production order is referred to as the "production order item". Further, the operating state of the production resources is represented by an operating state indicating whether or not a plurality of reaction apparatuses 2 are in operation and an operation state indicating whether or not a plurality of workers 3 are in operation.

[0056] The action a is a candidate for an option when allocating production resources necessary for producing a production order item. For example, it includes at least one of an allocation of the reaction apparatus 2 to be operated when producing the production order item among a plurality of reaction apparatuses 2 and an allocation of the worker 3 who performs an operation when producing the production order item among a plurality of workers 3. When the reaction process for producing the production order item spans a plurality of work shifts, as the allocation of the worker 3, each worker 3 who performs an operation in each work shift is allocated.

[0057] As shown in FIG. 7, the action a according to the present embodiment is represented by each combination of the reaction apparatus 2 and the worker 3. When the reaction process for producing the production order item spans a plurality of work shifts, the action a is represented by a combination of the reaction apparatus 2 and each worker 3 who performs an operation in each work shift. In the present embodiment, for simplicity of explanation, the case where the reaction process ends within a single work shift will be described.

[0058] The reward r is calculated based on the difference between the target value of the production evaluation index for the production order item and the actual value of the production evaluation index when the production order item is produced using the production resources assigned by the action a. The actual value of the production evaluation index may be obtained based on, for example, past production management information registered in the production management database 60, or may be estimated by executing a simulation of the reaction process by the production simulation device 7.

[0059] The reward r is calculated such that the smaller the difference between the target value of the production evaluation index and the actual value of the production evaluation index, the larger it becomes. When the reaction device 2 assigned by the action a is in operation, the reward r is corrected to be smaller, and when the operator 3 assigned by the action a is in the driving operation, the reward r may be corrected to be smaller. Further, the reward r may further consider viewpoints other than the production evaluation index. For example, when the reaction device 2 or the operator 3 having past production results is assigned, the reward r may be corrected to be larger.

[0060] When adopting reinforcement learning as machine learning, the learning data includes only the input data corresponding to the state s. That is, the learning data is configured not to include output data. As shown in FIG. 7, the input data constituting the learning data according to the present embodiment includes the production order item, the operating state indicating whether or not a plurality of reaction devices 2 are in operation, and the operation state indicating whether or not a plurality of operators 3 are in the driving operation.

[0061] FIG. 8 is a schematic diagram showing an example of a neural network model used in the machine learning device 4. In FIG. 8, the evaluation when a predetermined action a is taken for the state s is performed using the action value function Q(s, a) of the Q-learning method.

[0062] The action value function Q(s, a) uses, for example, a method called DQN (Deep Q-Network), with the state s as the input variable and each action a in the state s mnThe action value function Q(s, a mn ) when taking (m = 1, 2, …, M and n = 1, 2, …, N) respectively can be approximately calculated by a neural network model with Q(s, a mn ) as the output variable. In this case, the machine learning unit 42 adjusts the weights wk of the neural network model, for example, so that an error function (e.g., TD error) including the reward r, learning rate α, and discount rate γ as variables is minimized, thereby updating the action value function Q(s, a mn ), and causing the learning model 11 to learn the correlation between the input data (state s) and the allocation data (action a mn ) indicating the allocation of production resources. Note that as a reinforcement learning method, any method may be adopted, and in addition to the Q-learning method, for example, the SARSA method, the Monte Carlo method, etc. may also be adopted.

[0063] The learning model 11 is configured as a neural network model shown in FIG. 8 to approximately calculate the action value function Q(s, a mn ). The neural network model shown in FIG. 8 is composed of i neurons (x1 to xi) in the input layer, p neurons (y11 to y1p) in the first intermediate layer, q neurons (y21 to y2q) in the second intermediate layer, and j (= M × N) neurons (z1 to zMN) in the output layer.

[0064] To each neuron in the input layer, the production order items as the input data (state s) included in the learning data are associated. Also, to each neuron in the input layer, each of the operating states of the M reaction apparatuses 2 and the operating states of the N workers 3 are associated.

[0065] To each neuron in the output layer, the action value function Q(s, a mn ) when taking each action a mn (m = 1, 2, …, M and n = 1, 2, …, N) is associated, and each neuron in the output layer outputs the value of the action value function Q(s, a mn ) of each action a mn respectively.

[0066] The first intermediate layer and the second intermediate layer are also called hidden layers. As a neural network, in addition to the first intermediate layer and the second intermediate layer, it may further have a plurality of hidden layers, or may have only the first intermediate layer as the hidden layer. Also, between the input layer and the first intermediate layer, between the first intermediate layer and the second intermediate layer, and between the second intermediate layer and the output layer, synapses connecting the neurons of each layer are formed, and weights wk (k is a natural number) are associated with each synapse.

[0067] (Machine learning method) FIG. 9 is a flowchart showing an example of a machine learning method by the machine learning device 4.

[0068] First, in step S100, the learning data acquisition unit 40 prepares a desired number of learning data as preliminary preparation for starting machine learning, and stores the prepared learning data in the learning data storage unit 41. Several methods can be adopted for preparing the learning data. For example, the learning data acquisition unit 40 may acquire the input data (state s) of the learning data based on the past production management information registered in the production management table 604 of the production management database 60, or may predict future production orders and acquire the input data (state s) of the learning data based on virtual production management information.

[0069] Next, in step S110, the machine learning unit 42 prepares a pre-learning model 11 to start machine learning. The pre-learning model 11 prepared here is composed of the neural network model illustrated in FIG. 8, and the weights wk of each synapse are set to initial values.

[0070] Next, in step S120, the machine learning unit 42 randomly acquires, for example, one set of learning data from the plurality of sets of learning data stored in the learning data storage unit 41.

[0071] Next, in step S121, the machine learning unit 42 acquires a target value of a production evaluation index for a production order item in the input data included in one of the learning data acquired in step S120. The target value of the production evaluation index is acquired, for example, by referring to the product table 601 of the production management database 60.

[0072] Next, in step S130, the machine learning unit 42 inputs the input data (state s1) included in the one learning data acquired in step S120 to the input layer of the prepared learning model 11 before learning (or during learning). As a result, each action a mn The value of (action value function Q(s,a mn ) is output.

[0073] Next, in step S140, the machine learning unit 42 performs the inference process on each of the actions a mn The action value function Q(s,a mn ), for example, a specific behavior a corresponding to the maximum value is selected. As a method for selecting a specific behavior a, for example, the greedy method, the ε-greedy method, or the like may be adopted.

[0074] Next, in step S150, the machine learning unit 42 acquires the performance value of the production evaluation index when the action a selected in step S140 is taken for the state s1. That is, the machine learning unit 42 acquires the performance value of the production evaluation index when the production order item as the input data (state s1) is produced by the reaction apparatus 2 and the worker 3 as the production resources corresponding to the action a selected in step S140.

[0075] Next, in step S160, the machine learning unit 42 calculates the reward r based on the difference between the target value of the production evaluation index acquired in step S121 and the actual value of the production evaluation index acquired in step S150.

[0076] Next, in step S170, the machine learning unit 42 adjusts the weights wk of the neural network model so that the error function is minimized based on the reward r calculated in step S160, thereby updating the action value function Q(s, a mn ). Thus, the machine learning unit 42 causes the learning model 11 to learn the correlation between the input data (state s) and the assigned data (action a mn ). Note that the update of the action value function Q(s, a mn ) does not have to be performed every time, and for example, it may be performed only when a predetermined condition is satisfied.

[0077] Next, in step S180, the machine learning unit 42 determines whether machine learning needs to be continued. As a result, if it is determined to continue (No in step S180), the process returns to step S120, and the steps S120 to S170 are performed on the learning model 11 during learning. If it is determined that machine learning is completed (Yes in step S180), the process proceeds to step S190.

[0078] Then, in step S190, the machine learning unit 42 stores the learned learning model 11 generated by adjusting the weights wk associated with each synapse in the learned model storage unit 43, and ends the series of machine learning methods shown in FIG. 9. As the learned learning model 11, for example, parameters representing the structure of the neural network and the values of the adjusted weights wk are stored. In the machine learning method, step S100 corresponds to the learning data storage step, steps S110 to S180 correspond to the machine learning step, and step S190 corresponds to the learned model storage step.

[0079] As described above, according to the machine learning apparatus 4 and the machine learning method according to the present embodiment, it is possible to provide a learning model 11 that enables the formulation of a production plan for a product produced by a reaction apparatus 2 that performs a predetermined reaction process under the operation of an operator 3 by allocating production resources for production order items. Therefore, by using this learning model 11, it is possible to reduce the work load required for formulating a production plan.

[0080] (Production Plan Determination Device 5) FIG. 10 is a block diagram showing an example of the production plan determination device 5. The production plan determination device 5 includes an input data acquisition unit 50, an inference unit 51, a learned model storage unit 52, and an output processing unit 53. The production plan determination device 5 is configured by, for example, the computer 900 shown in FIG. 5. In that case, the input data acquisition unit 50 is composed of an input device 916, a communication I / F unit 922, or an I / O device I / F unit 926, the inference unit 51 and the output processing unit 53 are composed of a processor 912, and the learned model storage unit 52 is composed of a storage device 920.

[0081] The input data acquisition unit 50 is an interface unit that is connected, for example, to a manager terminal device (not shown) used by the manager of the production factory 10 via the network 8 and acquires input data including at least the item of the product. The input data acquisition unit 50 receives a production order that is a target for formulating a production plan from the manager terminal device, and acquires input data based on the item of the product specified in the production order and the operating state of the production resources at that time. The input data according to the present embodiment is composed of the item of the product specified in the production order, the operating states of the plurality of reaction devices 2, and the operating states of the plurality of workers 3.

[0082] The inference unit 51 performs an inference process of inputting the input data acquired by the input data acquisition unit 50 into the learning model 11 and inferring the allocation of production resources required when producing the item of the product included in the input data. For the inference process, a learned learning model 11 obtained by performing machine learning using the machine learning device 4 and the machine learning method is used.

[0083] The inference unit 51 not only has the function of performing inference processing using the learning model 11, but also, as preprocessing for the inference processing, has a preprocessing function of adjusting the input data acquired by the input data acquisition unit 50 into a desired format or the like and inputting it to the learning model 11, and as postprocessing for the inference processing, has a postprocessing function of adjusting the allocated data output from the learning model 11 into a desired format or the like by applying a predetermined logical formula or calculation formula. Note that the inference result of the inference unit 51 is preferably stored in the learned model storage unit 52 or another storage device (not shown). The past inference results can be used, for example, as learning data for online learning or relearning for further improving the inference accuracy of the learning model 11.

[0084] The learned model storage unit 52 is a database that stores the learned learning model 11 used in the inference processing of the inference unit 51. Note that a plurality of learning models 11 may be stored in the learned model storage unit 52 and selectively used by the inference unit 51. For example, a plurality of learning models 11 may be prepared for each difference in the number and type of input data and output data.

[0085] The output processing unit 53 performs output processing for outputting the inference result of the inference unit 51, that is, the allocated data. Various output means can be adopted as specific output means. For example, the output processing unit 53 may be registered in the production management table 604 of the production management database 60 by transmitting the allocated data to the production management database device 6, or may be displayed on the administrator terminal device by transmitting it to the administrator terminal device. At that time, the allocation of the production resources indicated by the allocated data may be adopted by the administrator of the production factory 10 as the final production plan, or may be partially corrected by the administrator of the production factory 10 as a temporary production plan.

[0086] (Production plan determination method) FIG. 11 is a flowchart showing an example of a production plan determination method by the production plan determination device 5.

[0087] First, in step S200, the input data acquisition unit 50 acquires input data (state s) by receiving a production order in which the item of the product is specified.

[0088] Next, in step S210, the inference unit 51 performs inference by subjecting the input data (state s) to preprocessing (which may be omitted) and inputting it to the input layer of the learning model 11, and the output data (each action a mn of the action value function Q(s, a mn ) value) of the output layer of the learning model 11 is obtained.

[0089] Next, in step S211, as an example of the post-processing of reinforcement learning, the inference unit 51, based on the value of the action value function Q(s, a mn of each action a mn ) output from each neuron in the output layer as output data, selects the action a that gives the maximum value among them.

[0090] Next, in step S220, the output processing unit 53 outputs the allocation data corresponding to the action a selected in step S211, and ends the series of production plan determination methods shown in FIG. 11. In the production plan determination method, step S200 corresponds to the input data acquisition step, steps S210 and S211 correspond to the inference step, and step S220 corresponds to the output processing step.

[0091] As described above, according to the production plan determination device 5 and the production plan determination method according to the present embodiment, by using the learning model 11, by allocating production resources to the production order item, a production plan for the product produced by the reaction device 2 that performs a predetermined reaction process under the operation of the operator 3 can be determined. Therefore, the work load required for determining the production plan can be reduced.

[0092] (Other Embodiments) The present invention is not limited to the above-described embodiments, and various modifications can be made and implemented without departing from the gist of the present invention. And all of them are included in the technical idea of the present invention.

[0093] In the above embodiment, the machine learning device 4 and the production plan determination device 5 have been described as being configured by separate devices, but they may be configured by a single device. In that case, the single device may appropriately perform machine learning by online learning and determine a production plan. Further, the machine learning device 4 or the production plan determination device 5 may function as at least one of the production management database device 6 and the production simulation device 7.

[0094] In the above embodiment, the case where reinforcement learning is adopted as the machine learning method in the machine learning device 4 has been described, but supervised learning may be adopted. In that case, the machine learning unit 42 may cause the learning model 11 to learn the correlation between the input data and the allocation data by inputting a plurality of sets of learning data including the input data and the allocation data into the learning model 11.

[0095] In the above embodiment, the case where a neural network is adopted as the learning model 11 for realizing machine learning by the machine learning unit 42 has been described, but other machine learning models may be adopted. Examples of other machine learning models include tree types such as decision trees and regression trees, ensemble learning such as bagging and boosting, recurrent neural networks, convolutional neural networks, neural network types including LSTM (including deep learning) cluster types such as hierarchical clustering, non-hierarchical clustering, k-nearest neighbor method, k-means method, multivariate analysis such as principal component analysis, factor analysis, logistic regression, support vector machines, etc. are included.

[0096] In the above embodiment, the input data has been described in the case where it includes production order items, the operating states of a plurality of reactors 2, and the operating states of a plurality of workers 3. However, it may be any data that includes at least production order items. Therefore, the input data may be configured to include, for example, neither the operating states of a plurality of reactors 2 nor the operating states of a plurality of workers 3, or it may include data other than these. When the input data includes only production order items, for example, the machine learning unit 42 may exclude the reactors 2 in operation or the workers 3 during operation from the actions a of reinforcement learning, or may correct the reward r of reinforcement learning so that it becomes small when the reactors 2 in operation or the workers 3 during operation are selected. Also, by preparing input data corresponding to various production situations where the reactors 2 in operation or the workers 3 during operation are different, the machine learning unit 42 may generate a plurality of learning models 11 for each production situation.

[0097] (Machine learning program and production plan determination program) The present invention can also be provided in the form of a program (machine learning program) 930 for causing the computer 900 shown in FIG. 5 to execute each step included in the machine learning method according to the above embodiment. Further, the present invention can also be provided in the form of a program (reaction tank operation support program) 930 for causing the computer 900 shown in FIG. 5 to execute each step included in the production plan determination method according to the above embodiment.

[0098] (Inference device, inference method, and inference program) The present invention can be provided not only in the form of the production plan determination device 5 (production plan determination method or production plan determination program) according to the above embodiment, but also in the form of an inference device (inference method or inference program) used for determining the production plan of a product. In that case, the inference device (inference method or inference program) may include a memory and a processor, and the processor among them may execute a series of processes. The series of processes includes an input data acquisition process (input data acquisition step) of acquiring input data including at least the item of the product, and an inference process (inference step) of inferring the allocation of production resources required when producing the item.

[0099] By providing it in the form of an inference device (inference method or inference program), it can be more easily applied to various devices compared to the case of implementing the production plan determination device 5. When the inference device (inference method or inference program) infers the allocation of production resources, it is naturally understandable to those skilled in the art that the inference method implemented by the inference unit 51 of the production plan determination device 5 may be applied using the learned learning model 11 generated by the machine learning device 4 and the machine learning method according to the above embodiment.

Explanation of Reference Numerals

[0100] 1... Production management system, 2... Reaction device, 3... Operator, 4... Machine learning device, 5... Production plan determination device, 6... Production management database device, 7... Production simulation device, 8... Network, 10... Production plant, 11... Learning model, 20... Reaction tank, 21... Agitator, 22... Sensor group, 23... Control equipment group, 24... Operation display panel, 40... Learning data acquisition unit, 41... Learning data storage unit, 42... Machine learning unit, 43... Learned model storage unit, 50... Input data acquisition unit, 51... Inference unit, 52... Learned model storage unit, 53... Output processing unit, 60... Production management database, 200... Reaction tank body, 201... Raw material inlet, 202... Product outlet, 203…Jacket, 204…Cold water circulation path, 205…Hot water circulation path 210…Agitating blade, 211…Motor, 212…Rotating shaft, 220…Reaction tank sensor group, 220A…Jacket heat medium temperature sensor, 220B…Cold water temperature sensor, 220C…Hot water temperature sensor, 220D…Jacket heat medium flow rate sensor, 220E…Cold water flow rate sensor, 220F…Hot water flow rate sensor, 220G…Reaction tank temperature sensor, 220H…Reaction tank pressure sensor, 220I…Reaction tank weight sensor 221…Agitator sensor group, 221A…Agitator torque sensor, 221B…Motor rotation speed sensor, 221C…Motor current sensor, 221D…Vibration sensor, 221E…Acoustic sensor, 222…Environment sensor group, 222A…Environment temperature sensor, 222B…Environment humidity sensor 230…Cooling equipment, 231…Heating equipment, 232…Cold water flow rate adjustment valve, 233…Cold water pump, 234…Hot water flow rate adjustment valve, 235…Hot water pump, 236…Inverter, 600…Production order table, 601…Product table, 602…Reaction device table, 603…Operator table, 604…Production management table, 900…Computer

Claims

1. A machine learning device that generates a learning model used in a production plan determination device for a product produced by a reaction device that performs a predetermined reaction process under the operation of an operator, A learning data storage unit that stores a plurality of sets of learning data including at least the item of the product as input data, A machine learning unit that, by inputting a plurality of sets of the learning data into the learning model, causes the learning model to learn, by reinforcement learning, the correlation between the input data and the allocation data indicating the allocation of production resources required for producing the item, A learned model storage unit that stores the learning model learned by the machine learning unit, and includes: The allocation data Includes the allocation of the operator who performs the operation when producing the item among a plurality of the operators, Machine learning device.

2. The machine learning unit Calculates a reward so that the smaller the difference between the target value of the production evaluation index for the item based on the input data and the actual value of the production evaluation index when the item based on the input data is produced using the production resources indicated by the allocation data, the larger the reward, Based on the reward, causes the learning model to learn the correlation, The machine learning device according to claim 1.

3. The machine learning unit When the operator with production performance in the past is assigned as the assignment of the operator indicated by the assignment data, corrects the reward so as to increase, The machine learning device according to claim 2.

4. The input data Further includes an operation state indicating whether or not a plurality of the operators are in the operation, The machine learning unit If the operator assigned as the assignment indicated by the assignment data is in the middle of an operation, correct it so that the reward decreases. The machine learning device according to claim 2 or claim 3.

5. The assignment data further includes an assignment of the reactor to be operated when producing the item among the plurality of reactors. The machine learning device according to claim 1.

6. The machine learning unit Calculate the reward so that the smaller the difference between the target value of the production evaluation index for the item based on the input data and the actual value of the production evaluation index when the item based on the input data is produced using the production resources indicated by the assignment data, the greater the reward, Based on the reward, cause the learning model to learn the correlation. The machine learning device according to claim 5.

7. The machine learning unit If the reactor or operator having a past production record is assigned as the assignment of the reactor or operator indicated by the assignment data, correct the reward so that it increases. The machine learning device according to claim 6.

8. The input data further includes an operation state indicating whether or not each of the plurality of reactors is in operation, and an operation state indicating whether or not each of the plurality of operators is in an operation operation, The machine learning unit If the reactor assigned as the assignment of the reactor indicated by the assignment data is in operation, correct it so that the reward decreases, and if the operator assigned as the assignment of the operator indicated by the assignment data is in an operation operation, correct it so that the reward decreases. The machine learning device according to claim 6 or claim 7.

9. The actual value of the production evaluation index is the measured value obtained by measuring the product produced by the reaction device with a measuring instrument, or an evaluation value calculated based on a plurality of the measured values measured by a plurality of the measuring instruments, the target value of the production evaluation index is predetermined with respect to the measured value or the evaluation value, The machine learning device according to any one of claims 2 to 4 and claims 6 to 8.

10. A production plan determination device for determining a production plan for a product produced by a reaction device that performs a predetermined reaction process under the operation of an operator, an input data acquisition unit that acquires input data including at least the item of the product, an inference unit that inputs the input data acquired by the input data acquisition unit into a learning model and infers the allocation of production resources required when producing the item, The learning model is one in which the correlation between the input data and allocation data indicating the allocation of the production resources is learned by reinforcement learning, The allocation data is includes the allocation of the operator who performs the operation when producing the item among a plurality of the operators, The inference unit is inputs the input data into the learning model and infers the allocation of the operator as the allocation of the production resources, A production plan determination device.

11. An inference device used for determining a production plan for a product produced by a reaction device that performs a predetermined reaction process under the operation of an operator, The inference device includes a memory and a processor, The processor is an input data acquisition process for acquiring input data including at least the item of the product, When the input data is acquired in the input data acquisition process, as an allocation of production resources required for producing the item, an inference process for inferring an allocation of the worker who performs the operation when producing the item among a plurality of the workers is executed. Inference device.

Images