Model generation method, computer program, and information processing device

Abstract

The invention provides a model generation method, a computer program, and an information processing apparatus. A computer executes the following processing: acquiring a plurality of variable groups for constructing a process recipe, state data representing a state of a processed object before a specific step of the process recipe is executed, and target data representing a target state of the processed object; a state evaluation unit that selects one variable group from the plurality of variable groups, and evaluates the state of the object to be processed obtained by executing one step represented by the selected one variable group; and generating a model for process recipe construction by reinforcement learning using the acquired state data and target data and a reward determined on the basis of the evaluated state of the object to be processed.

Description

Technical Field

[0001] The present invention relates to a model generation method, a computer program and an information processing device. Background Art

[0002] In a conventional semiconductor processing apparatus, a process recipe is created by describing control values ​​for various control components constituting the semiconductor processing apparatus in each step, and various processes are performed in accordance with each step of the created process recipe.

[0003] Patent Document 1 discloses a method of generating a prediction model to explore input parameter values ​​set to a semiconductor processing device in order to process a target processed shape. The prediction model represents the relationship between the input parameter values ​​and the output values ​​that are actually measured values ​​in the processing results.

[0004] Patent Document 1: Japanese Patent Application Publication No. 2019-165123 Summary of the invention

[0005] The present disclosure aims to provide a method for generating a model, a computer program, and an information processing device that can present a variable group recommended for a process recipe according to the current shape of a processed object.

[0006] In the model generation method disclosed in the present invention, a computer performs the following processing: acquiring multiple variable groups for constructing a process recipe, state data representing the state of a processed object before executing a specific step of the process recipe, and target data representing the target state of the processed object; selecting one variable group from the multiple variable groups, and evaluating the state of the processed object obtained by executing a step represented by the selected variable group; and generating a model for process recipe construction by reinforcement learning using the acquired state data and target data, and a reward determined based on the evaluated state of the processed object.

[0007] According to the present disclosure, a variable group recommended for a process recipe can be presented according to the current shape of the object to be processed. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Figure 1 This is an explanatory diagram for explaining the outline of the information processing system according to the embodiment.

[0009] Figure 2 This is an explanatory diagram for explaining the structure of data obtained in the manufacturing process.

[0010] Figure 3 This is an explanatory diagram for explaining the structure of a proxel.

[0011] Figure 4 It is a block diagram showing the internal structure of the information processing device.

[0012] Figure 5 This is a conceptual diagram showing an example of a proxel database.

[0013] Figure 6 This is a schematic diagram of the reinforcement learning algorithm.

[0014] Figure 7 is a schematic diagram showing an example of the structure of a learning model.

[0015] Figure 8 This is a flowchart that explains the sequence of generating a learning model.

[0016] Fig. 9 It is a flowchart illustrating the recipe construction sequence using the learning model.

[0017] Fig.10 This is a schematic diagram showing a structural example of a learning model in Implementation Example 2.

[0018] Fig.11 This is a flowchart explaining the processing procedure of fine-tuning.

[0019] Fig.12 This is a flowchart that explains the evaluation sequence of the process recipe. DETAILED DESCRIPTION

[0020] Hereinafter, one embodiment will be described with reference to the drawings.

[0021] (Implementation Method 1)

[0022] Figure 1 1 is an explanatory diagram for explaining the overview of the information processing system involved in the embodiment. The information processing system involved in the embodiment includes an information processing device 100 and a semiconductor manufacturing device 200. The information processing device 100 and the semiconductor manufacturing device 200 are connected, for example, to be able to communicate. The semiconductor manufacturing device 200 may include any device that implements a semiconductor manufacturing process, such as an exposure device, an etching device, a film forming device, an ion implantation device, an ashing device, and a sputtering device. With respect to the manufacturing process performed in the semiconductor manufacturing device 200, the information processing device 100 constructs a recommended process recipe and prompts the constructed process recipe to the manufacturer, etc. (user).

[0023] The process recipe used in the manufacturing process is composed of a plurality of steps. Here, a step represents the smallest processing unit that changes the state (properties of the processing object, the state of the semiconductor manufacturing device) in the semiconductor manufacturing process. Therefore, in the case where the state changes with the passage of time, in this embodiment, before the passage of time and after the passage of time are respectively understood as different steps.

[0024] The information processing apparatus 100 selects one variable group used in each step from a plurality of variable groups prepared in advance, and determines the processing content to be performed in each step, thereby constructing a process recipe recommended to the user.

[0025] In this embodiment, a series of variables (set values, control values, etc.) that achieve the same effect in each step are recorded as a variable group. In the following description, the variable group is also recorded as proxel. Proxel is the smallest unit of data used to determine the processing content in each step. It is the same as calling the smallest unit of an image (Picture Element) a pixel and the smallest unit of a three-dimensional image (Volume Element) a voxel. Figure 1 In the example, a proxel is represented by a regular hexagon. For more information about proxels, use Figure 2-Figure 3 Explain specifically.

[0026] In this embodiment, a learning model MD1 based on reinforcement learning is used to explore the proxel (refer to Figure 7 ). For example, when state data representing the current state of the processing object (processed object) and target data representing the target state of the processing object set by the user are input, the learning model MD1 learns to output information about the proxel recommended in the next step. The information processing device 100 performs operations based on the learning model MD1 in each step, and determines the proxel to be selected in each step based on the operation results, thereby constructing a process recipe recommended to the user. The structure of the learning model MD1 and the method for generating the learning model MD1 will be described in detail later.

[0027] Figure 2 This is an explanatory diagram for explaining the structure of data obtained in the manufacturing process. According to the multiple steps constituting the process recipe, the semiconductor manufacturing process is implemented step by step. In each of these multiple steps, for example, data of six items including initial data (I), setting data (R), output data (E), measurement data (Pl), experimental data (Pr), and target data (Pf) are obtained.

[0028] The initial data (I) is data related to the object to be processed. The initial data (I) is set by the user, for example. The initial data (I) includes data such as initial CD, material, thickness, aspect ratio, and mask coverage.

[0029] The setting data (R) is data set for the semiconductor manufacturing device. The setting data (R) is set by the user according to the properties of the processing object, the final target object, the characteristics of the semiconductor manufacturing device used, etc. The setting data (R) includes, for example, the pressure (Pressure) in the chamber, the power (Power) of the high-frequency power supply, the gas flow rate (Gas), the temperature in the chamber, the surface temperature (Temperature) of the processing object, and other data.

[0030] Output data (E) is data output from a semiconductor manufacturing device, and includes data such as peak-to-peak voltage (Vpp) of an RF signal, DC self-bias voltage (Vdc), luminous intensity (OES) based on luminous spectroscopy, and reflected wave power (Reflect).

[0031] The measurement data (Pl) is data related to the implementation environment of the manufacturing process. The measurement data (Pl) is measured using various sensors and measuring devices. The measurement data (Pl) includes, for example, data such as plasma density, ion energy, and ion flux.

[0032] Experimental data (Pr) is data related to the result obtained in each step. Experimental data (Pr) is measured using various sensors and measuring devices. Experimental data (Pr) includes, for example, etching rate, deposition rate, XY coordinates, film type, vertical / lateral division, and other data.

[0033] Target data (Pf) is data related to the final target object. The user sets the target data (Pf) according to the properties that the final target object should achieve. The target data (Pf) includes data such as limit dimension (CD), depth (Depth), taper angle (Taper), tilting angle (Tilting), bowing, etc.

[0034] also, Figure 2 The items shown are examples, and the types of items included in each step are not limited to the types shown in the figure. For example, there may be items that are not included depending on the manufacturing process or step, or different items may be included depending on the manufacturing process or step. Figure 2 The data of each item shown is an example, and the type of data included in each item is not limited to the type shown in the figure. The data of each item can be appropriately set according to the manufacturing process and steps.

[0035] In this embodiment, a series of variables (variable group) that obtain the same effect in the same manufacturing process and the same step is defined as a proxel. Here, the effect in a predetermined step of the manufacturing process is a quantity derived as the difference between the state of the processed object before executing the step and the state of the processed object after executing the step.

[0036] Figure 3 This is an explanatory diagram for explaining the structure of proxel. For simplicity, the structure of proxel is explained by taking a two-dimensional feature space with the horizontal axis set as the first variable and the vertical axis set as the second variable as an example. By making various changes to the first and second variables in the process recipe and executing the same manufacturing process multiple times, and analyzing the execution results, it is possible to find the range of the first and second variables that achieve the same effect in each step.

[0037] exist Figure 3 Each region indicated by the symbols R1 to R10 in the figure is a region defined by determining the range of the first variable and the second variable. When the same step in the same manufacturing process is performed using the variables in each region R1, R2, and R3 among these regions R1 to R10, the same effect such as "Effect 1" is obtained. In this case, the groups of multiple variables included in the regions R1, R2, and R3 are grouped into one variable group, and these variable groups are defined as one proxel (set as proxel1).

[0038] Similarly, when the same step in the same manufacturing process is performed using variables in each of regions R4, R5, R6, and R7, the same effect, namely "effect 2" different from "effect 1", is obtained. In this case, the groups of multiple variables included in regions R4, R5, R6, and R7 are grouped into one variable group, and these variable groups are defined as another proxel (set to proxel2) different from the previous proxel1.

[0039] Similarly, when the same step in the same manufacturing process is executed using the variables in each region of regions R8 and R9, the same effect, namely "effect 3" different from "effect 1" and "effect 2", is obtained. In this case, the groups of multiple variables included in regions R8 and R9 are grouped into one variable group, and these variable groups are defined as another proxel (set to proxel3) different from the previous proxel1 and proxel2.

[0040] Furthermore, when the same step in the same manufacturing process is executed using the variables in region R10, a separate effect, "Effect 4", which is different from "Effect 1" to "Effect 3", is obtained. In this case, the plurality of variable groups included in region R10 are grouped into one variable group, and these variable groups are defined as another proxel (set to proxel 4) different from the previous proxel 1 to proxel 3.

[0041] exist Figure 3 In the text, for simplicity, the structure of proxel is explained using a two-dimensional feature space. However, when each step contains K (K is an integer greater than 1) variables, it is sufficient to use the K-dimensional feature space to find the range (spatial region) of each variable that produces the same effect, and to define proxel by grouping the variables according to each effect.

[0042] Next, the structure of the information processing device 100 for constructing a recipe using proxel will be described. Figure 4 1 is a block diagram showing the internal structure of the information processing device 100. The information processing device 100 is a dedicated or general-purpose computer including a control unit 101, a storage unit 102, a communication unit 103, an operation unit 104, and a display unit 105, for example.

[0043] The control unit 101 includes a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), etc. The ROM included in the control unit 101 stores a control program for controlling the operation of each hardware unit included in the information processing device 100. The CPU in the control unit 101 reads and executes the control program stored in the ROM and various computer programs stored in the storage unit 102, and controls the operation of each hardware unit, so that the entire device functions as the information processing device of the present disclosure. The RAM included in the control unit 101 temporarily stores data used in the execution of calculations.

[0044] In the embodiment, the control unit 101 is configured to include a CPU, a ROM, and a RAM, but the configuration of the control unit 101 is not limited to the above configuration. The control unit 101 may also include, for example, one or more control circuits or arithmetic circuits including a GPU (Graphics Processing Unit), an FPGA (Field Programmable Gate Array), a DSP (Digital Signal Processor), a quantum processor, a volatile or non-volatile memory, etc. In addition, the control unit 101 may also include functions such as a clock that outputs date and time information, a timer that measures the elapsed time from when a measurement start instruction is given to when a measurement end instruction is given, and a counter that counts numbers.

[0045] The storage unit 102 includes storage devices such as a HDD (Hard Disk Drive), an SSD (Solid State Drive), and an EEPROM (Electronically Erasable Programmable Read Only Memory). The storage unit 102 stores various computer programs executed by the control unit 101 and various data used by the control unit 101.

[0046] The computer programs (program products) stored in the storage unit 102 include a learning program PG1 for generating a learning model MD1, a recipe building program PG2 for recipe building, a simulator SIM for virtually performing a manufacturing process according to a process recipe, etc. These computer programs may be a single computer program or may be composed of a plurality of computer programs. In addition, these computer programs may partially use existing libraries.

[0047] The computer programs such as the learning program PG1 and the recipe building program PG2 stored in the storage unit 102 are provided by a non-temporary recording medium RM on which the computer programs are recorded in a readable manner. The recording medium RM is a portable memory such as a CD-ROM, a USB memory, an SD (Secure Digital) card, a micro SD card, a Compact Flash (registered trademark), etc. The control unit 101 uses a reading device not shown in the figure to read various computer programs from the recording medium RM, and stores the various read computer programs in the storage unit 102. In addition, the computer program stored in the storage unit 102 can also be provided through communication. In this case, the control unit 101 only needs to obtain the computer program through communication via the communication unit 103, and store the obtained computer program in the storage unit 102.

[0048] The storage unit 102 stores a learning model MD1. The learning model MD1 is a learning model generated by reinforcement learning described later. The storage unit 102 stores structural information of layers constituting the learning model MD1, information of nodes constituting each layer, and model parameters such as weights and biases between nodes.

[0049] In addition, the storage unit 102 includes a proxel database DB. Figure 5 : is a conceptual diagram showing an example of a proxel database DB. The proxel database DB stores each proxel determined as described above in association with a process ID, a data range of each data, an effect, a process ID, and a step number. That is, in the proxel database DB, a plurality of process conditions summarized according to each manufacturing process, each step, and each effect are stored. In the proxel database DB, when a proxel ID is specified, a semiconductor process condition (variable group) that obtains the same effect is determined.

[0050] In the present embodiment, the information processing device 100 is provided with a proxel database DB inside. However, the proxel database DB may be provided outside the information processing device 100 and accessed via the communication unit 103 to obtain required data.

[0051] The communication unit 103 has a communication interface for sending and receiving various data with an external device. As the communication interface of the communication unit 103, a communication interface based on a communication standard such as a LAN (Local Area Network) can be used. The external device may include a semiconductor manufacturing device 200 and a server device not shown. When the communication unit 103 inputs data to be sent from the control unit 101, the communication unit 103 sends the data to the external device at the destination, and when receiving data sent from the external device, the communication unit 103 outputs the received data to the control unit 101.

[0052] The operation unit 104 includes operation devices such as a touch panel, keyboard, and switches, and receives various operations and settings from a user, etc. The control unit 101 performs appropriate control based on various operation information given from the operation unit 104 , and stores setting information in the storage unit 102 as needed.

[0053] The display unit 105 includes a display device such as a liquid crystal monitor or an organic EL (Electro-Luminescence), and displays information to be reported to a user or the like in accordance with an instruction from the control unit 101 .

[0054] In addition, the information processing device 100 in this embodiment may be a single computer or a computer system composed of a plurality of computers, peripheral devices, etc. In addition, the information processing device 100 may be a virtual machine that virtualizes an entity or a cloud. Furthermore, in this embodiment, the information processing device 100 and the semiconductor manufacturing device 200 are described as separate entities, but the information processing device 100 may also be disposed inside the semiconductor manufacturing device 200.

[0055] In this embodiment, a reinforcement learning algorithm is used to construct a process recipe to be presented to the user. Figure 6 This is a schematic diagram of the reinforcement learning algorithm. The reinforcement learning algorithm is an algorithm that deals with the following problem, which refers to an intelligent agent in a certain environment observing the current state of the observed object and deciding what action to take. The following is an explanation of one method of reinforcement learning, namely DQN (Deep Q-Network).

[0056] The learning model in reinforcement learning takes the current state s of the input observation object as input. t In the case of , learning is performed to output the value of the action value function (Q value) for each of the desirable actions a1, a2, ..., an (n is an integer greater than 2). The method of approximating the action value function through a neural network and performing reinforcement learning is DQN.

[0057] In this embodiment, a learning model MD1 is expressed using a neural network that approximates the action value function, and reinforcement learning is performed based on the current state of the processed object to output information related to the value of each proxel when it is selected. t For example, it is differential data between state data representing the current state of the object to be processed and target data representing the target state of the object to be processed. More specifically, differential data between image data representing the current shape of the object to be processed and image data representing the target shape of the object to be processed can be used.

[0058] The learning model MD1 is for the current state s t The value of the action value function Q(s) is output for each possible action a1, a2, ..., an (n is an integer greater than 2). t , a1), Q(s t , a2), …, Q(s t , an). The value of the action value function represents the value of the action value function in state s tThe expected value of the future benefit obtained when action a is selected is also called Q value. That is, the value of the action value function (Q value) does not represent short-term rewards, but represents the value in the long-term sense. In this embodiment, action a is equivalent to executing a step of the constituent process according to the conditions specified by the selected proxel.

[0059] The agent refers to the Q value output from the learning model MD1 for each action and chooses to t The action a with the highest Q value among the possible actions a1, a2, ..., an t . Depending on the action selected a t To update the environment and determine the next state s t+1 In this embodiment, the agent is the control unit 101 of the information processing device 100, and the environment is a simulator that virtually executes a manufacturing process according to a process recipe.

[0060] The agent takes action a t The next state s generated by the choice t+1 , get rewards from the environment t+1 When the manufacturing process to be implemented is etching, the remuneration r is determined based on, for example, the amount of the object to be processed, the loss of the residual cutting, the loss of over-cutting, the legitimacy of the selected proxel, etc. t+1 .

[0061] The agent learns to take actions that maximize future rewards (benefits) by repeated trial and error. Specifically, the agent uses the state s t , status t+1 , and for the previous action a t The reward r t+1 , the learning model MD1 is updated sequentially based on the following mathematical formula (1).

[0062] Q(s t , a t )←Q(s t ,a t )+α{r t+1 +γ·maxQ(s t+1 ,a +1) -Q(s t ,a t )}

[0063] ......(1) Here, α is the learning coefficient, γ is the discount rate, r t+1 for action tThe reward obtained as a result of learning. The learning coefficient α is a parameter that determines the speed of learning and satisfies the relationship of 0<α<1. The discount rate γ is a parameter that indicates how much the evaluation of the future state is discounted and satisfies the relationship of 0<γ<1.

[0064] In Q-learning, the model parameters of the learning model MD1 are learned using the error back propagation method, etc., so that the second term on the right side of the mathematical formula (1) is zero. This means that after taking action a t The state s t Transfer to state s t+1 In the case of t The Q value is close to the next state s t+1 The value of the case where the Q value is the highest.

[0065] The agent repeatedly updates the learning model MD1 until the specified termination condition is met. By repeatedly updating, the learning model MD1 is learned so that the reward r t+1 The end condition is appropriately set, for example, when a predetermined number of updates have been performed, when the shape of the object to be processed approaches the target shape, when the object to be processed cannot be cut any further, and the like.

[0066] The structure of the learning model MD1 generated by the above reinforcement learning will be described. Figure 7 is a schematic diagram showing a configuration example of a learning model MD1. The learning model MD1 includes, for example Figure 7 The first layer L1 to the sixth layer L6 are shown as six layers. The first layer L1 has a slice layer. The first layer L1 is input with differential data between the image data representing the current shape of the object to be processed and the image data representing the target shape. The first layer L1 intercepts a part of the input differential data and outputs it to the second layer of the subsequent stage.

[0067] The second layer L2 has a CNN (Convolutional Neural Network) block, a MaxPooling layer, a BatchNormalization layer, and a ReLU (Rectified Linear Unit) layer. The second layer L2 extracts features from the differential data input from the first layer L1 and outputs the extracted data to the third layer in the latter stage.

[0068] The third layer L3 has a CNN block, a maximum pooling layer, a batch normalization layer, and a ReLU layer similarly to the second layer L2. The second layer L2 extracts features from the data input by the second layer L2 and outputs the extracted data to the third layer in the subsequent stage.

[0069] The fourth layer L4 includes a smoothing layer, the fifth layer L5 includes a linear layer and a ReLU layer, and the sixth layer includes a linear layer. The final linear layer of the sixth layer L6 includes the same number of nodes as the number of possible actions a1, a2, ..., an, and the value of the action value function is output from each node for the corresponding action a1, a2, ..., an.

[0070] In this embodiment, since the difference between the image data representing the current shape of the object to be processed and the image data representing the target shape is input to the learning model MD1, there is an advantage that it is easy to select an appropriate proxel when approaching the target shape.

[0071] Next, the operation of the information processing device 100 will be described.

[0072] The information processing device 100 performs a process of generating the learning model MD1 in a learning phase before starting actual operation, and executes a process of building a recipe using the learning model MD1 in an operation phase after generating the learning model MD1.

[0073] Figure 8 2 is a flowchart for explaining the generation sequence of the learning model MD1. In the learning phase, the control unit 101 of the information processing device 100 executes the learning program PG1 stored in the storage unit 102 and executes the following sequence to generate the learning model MD1. In addition, in the phase before starting the learning, the model parameters describing the learning model MD1 are given initial values.

[0074] The control unit 101 acquires image data representing a target shape of the object to be processed (step S101). The target shape is a target cross-sectional shape set by a user for the object to be processed. The control unit 101 can acquire the target shape from a user terminal (not shown) via the communication unit 103, for example.

[0075] The control unit 101 acquires image data representing the current shape of the object to be treated (step S102). Here, the image data representing the current shape of the object to be treated is, for example, image data of a cross-sectional shape of the object to be treated obtained by calculation using a simulator SIM.

[0076] The control unit 101 inputs the difference data between the image data representing the current shape of the processed object and the image data representing the target shape into the learning model MD1, and performs the operation based on the learning model MD1 (step S103). Through the operation based on the learning model MD1, the value of the action value function (Q value) is obtained for each possible action.

[0077] The control unit 101 selects a proxel based on the calculation result based on the learning model MD1 (step S104).t The control unit 101 selects the action a with the highest Q value among the calculated Q values ​​of the actions, thereby selecting the proxel. The control unit 101 stores the information of the selected proxel in the storage unit 102.

[0078] The control unit 101 refers to the proxel database DB and reads out the process conditions stored in association with the selected proxel (step S105). The control unit 101 only needs to read out the data range of the setting data, output data, measurement data, and experimental data as the process conditions.

[0079] The control unit 101 executes simulation using the simulator SIM based on the process conditions read from the proxel database DB, and calculates the shape of the object to be processed (step S106 ).

[0080] The control unit 101 calculates a reward based on the calculated shape of the processed object (step S107). For example, the control unit 101 calculates the amount of cutting by comparing the current shape of the processed object obtained in step S102 with the shape of the processed object calculated by the simulator SIM in step S106, and assigns a reward of, for example, -0.1 to 0.1 according to the amount of cutting. In addition, the control unit 101 may also calculate the loss of cutting residue or the loss of over-cutting by comparing the target shape obtained in step S101 with the shape of the processed object calculated in step S106, and assign a reward of, for example, -0.1 to 0.1 to the loss of cutting residue, and assign a reward of, for example, -0.1 to 0 to the loss of over-cutting. Moreover, the control unit 101 may also assign a reward of -1 if the proxel input in step S103 is improper.

[0081] The control unit 101 determines whether to end the learning (step S108). For example, when the number of steps in the manufacturing process is greater than a threshold, when the loss due to overcutting is greater than a threshold, when the difference between the current shape and the target shape is less than a threshold, when an improper proxel is selected more than a set number of times, etc., the control unit 101 determines to end the learning.

[0082] If it is determined that the learning is not to be terminated (S108: No), the control unit 101 updates the model parameters including the weights and biases between the nodes constituting the learning model MD1 (step S109), and returns the process to step S103. After returning the process to step S103, the control unit 101 sets the shape calculated by the simulator SIM as the current shape and performs calculations using the learning model MD1. In Q learning, the model parameters of the learning model MD1 are learned by repeatedly performing the calculations of the above steps S103 to S109 so that the second term of the above mathematical formula (1) approaches zero.

[0083] When it is determined that the learning is to be completed (S108: Yes), the control unit 101 ends the processing according to this flowchart. At this time, the model parameters of the learning model MD1 that has been learned are stored in the storage unit 102.

[0084] The control unit 101 may also execute the same process multiple times. Figure 8 The learning model MD1 is generated in the order shown in the flowchart of . In this case, the control unit 101 can perform reinforcement learning by assigning rewards after the completion of each process. For example, the control unit 101 can assign a reward of -1 to 1 for the final cutting residue loss and a reward of 0 to 1 according to the number of steps. In addition, the control unit 101 can also assign a reward of -1 to 0 for the loss of excessive cutting.

[0085] In this embodiment, a learning algorithm based on Q-learning is described as an example, but the method of generating the learning model MD1 is not limited to Q-learning. For example, any reinforcement learning algorithm such as TD learning (Temporal Difference Learning), policy gradient method (Policy gradients), SARSA (State-Action-Reward-State-Action), Actor-critic, etc. can be used.

[0086] Fig. 9 1 is a flowchart for explaining the recipe construction procedure using the learning model MD1. In the application phase after generating the learning model MD1, the control unit 101 of the information processing device 100 executes the recipe construction program PG2 stored in the storage unit 102 to execute the following sequence for constructing a process recipe. In addition, in the phase before starting the application, the storage unit 102 stores the learned model parameters.

[0087] The control unit 101 initializes the index of the proxel used for the process recipe to i=1 (step S121 ).

[0088] The control unit 101 obtains image data representing the initial shape and the target shape of the object to be processed (step S122). The initial shape is the cross-sectional shape of the object to be processed set by the user before the process starts, and the target shape is the target cross-sectional shape set by the user for the object to be processed. The control unit 101 can obtain the image data from a user terminal (not shown) through the communication unit 103, for example.

[0089] The control unit 101 inputs the difference data between the image data representing the current shape of the processed object and the image data representing the target shape into the learning model MD1, and performs the operation based on the learning model MD1 (step S123). The operation result based on the learning model MD1 includes the information of the recommended proxel. In this embodiment, the information of the action with the highest Q value among the Q values ​​calculated for the possible actions is equivalent to the information of the recommended proxel.

[0090] The control unit 101 selects a proxel based on the calculation result based on the learning model MD1 (step S124 ). The control unit 101 stores the selected proxel in the storage unit 102 as a proxel of index i.

[0091] The control unit 101 refers to the proxel database DB, reads out the process conditions stored in correspondence with the selected proxel (step S125), and executes simulation using the simulator SIM to calculate the shape of the object to be processed (step S126).

[0092] The control unit 101 determines whether to end the process (step S127). For example, if the difference between the shape of the object (current shape) calculated in step S126 and the target shape is smaller than a threshold, the control unit 101 determines to end the process.

[0093] If it is determined that the process is not completed (S127: No), the control unit 101 increases the index i of the proxel by +1 (step S128) and returns the process to step S123. The control unit 101 regards the shape calculated in step S126 as the current shape and repeats the calculation based on the learning model MD1.

[0094] On the other hand, when it is determined that the process is to be terminated (S127: Yes), the control unit 101 constructs a process recipe by combining proxels with indexes i=1 to n (step S129), and outputs information of the constructed process recipe (step S130). Here, n is the index of the proxel at the end of the process. The control unit 101 may also send the information of the constructed process recipe from the communication unit 103 to the user terminal, or may display it on the display unit 105.

[0095] As described above, in Embodiment 1, reinforcement learning can be used to generate a learning model MD1 for recipe construction. Since the learned learning model MD1 can be stored in the storage unit 102 in advance, when it is used, the model parameters of the learning model MD1 can be read from the storage unit 102 and calculated, so that a process recipe recommended to the user can be constructed.

[0096] In this embodiment, the difference between image data representing the current shape and image data representing the target shape is used as input to the learning model MD1. Therefore, when the current shape is close to the target shape, it is easy to select an appropriate proxel and a more appropriate process recipe can be constructed.

[0097] In this embodiment, the difference data of the initial shape and the target shape are input to the learning model MD1 after learning, and the process recipe of the entire manufacturing process is constructed. Alternatively, the control unit 101 may input the difference data between the shape data at a first time point of the manufacturing process and the shape data at a second time point later than the first time point into the learning model MD1, and generate the process recipe of the part from the first time point to the second time point.

[0098] In addition, the control unit 101 may also display the value of the action value function calculated for the proxel together with the information of the proxel selected in each step. In this embodiment, since the proxel is selected based on the value of the action value function, the value of the action value function is also prompted to the user, thereby being able to prompt the user with the reliability of the proxel selection result.

[0099] Furthermore, the control unit 101 may also use Grad-CAM (Gradient-weighted Class Activation Mapping) technology to display the area of ​​interest in proxel selection through a heat map. For example, the control unit 101 only needs to extract the results of the second layer L2, the third layer L3, and the fourth layer L4 that constitute the learning model MD1, perform error back propagation using the classification results based on the fourth layer L4, calculate the gradients of the convolutional layers in the third layer L3 and the second layer L2, and thereby generate a heat map representing the area of ​​interest.

[0100] (Implementation Method 2)

[0101] In the second embodiment, a modification example of the learning model will be described.

[0102] Fig.10 1 is a schematic diagram showing a configuration example of a learning model MD2 in Embodiment 2. The configuration of the learning model MD2 in Embodiment 2 is the same as that of the learning model MD1 in Embodiment 1, and includes a first layer L1 to a sixth layer L6.

[0103] In the second embodiment, in addition to the differential data between the image data representing the current shape of the processed object and the image data representing the target shape, the information of the proxel selected last time is also input into the learning model MD2. That is, the learning model MD2 in the second embodiment is input with the differential data between the image data representing the shape of the processed object before the i-th (i is an integer greater than 2) step in the process recipe and the image data representing the target shape, and the information of the proxel selected when the i-1-th step is executed. Specifically, it is sufficient to make the index of the proxel selected last time a one-hot representation and combine it with the tensor smoothed in the fourth layer L4. That is, it is sufficient to combine a vector in which the element corresponding to the selected proxel is set to 1 and the remaining elements are set to 0.

[0104] In the second embodiment, the information of the proxel selected last time is taken into consideration to perform reinforcement learning. For example, if the same proxel as the last time is selected, the reward is set to zero, and if a proxel different from the last time is selected, the reward is set to a negative value (e.g., -0.5) to perform reinforcement learning. The processing order of reinforcement learning and the order of formula construction after learning are the same as those in the first embodiment, so the detailed description thereof is omitted.

[0105] In the second embodiment, since reinforcement learning is performed by taking into account the information of the proxel selected previously, it is expected that a more natural process recipe with less switching of proxels can be constructed.

[0106] (Implementation 3)

[0107] In the third embodiment, fine-tuning of the learning model is described.

[0108] The information processing device 100 of the third embodiment fine-tunes the learning model MD1 when the initial shape of the object to be processed is changed, when the data of the proxel stored in the proxel database DB is changed, etc. The following describes the case where the initial shape of the object to be processed is changed and the learning model MD1 is fine-tuned.

[0109] Fig.11 This is a flowchart for explaining the processing procedure of fine-tuning. When fine-tuning the learned learning model MD1 to generate a new learning model, the control unit 101 reads the model parameters of the learning model MD1 from the storage unit 102 (step S301).

[0110] The control unit 101 acquires image data representing the target shape of the object to be processed (step S302 ), and acquires image data representing the initial shape of the object to be processed after the change (step S303 ).

[0111] The control unit 101 inputs the difference data between the image data representing the current shape of the object to be processed and the image data representing the target shape into the learning model MD1, and performs the operation based on the learning model MD1 (step S304). That is, the operation can be performed based on the model parameters read in step S301. Through the operation based on the learning model MD1, the value of the action value function (Q value) is obtained for each possible action.

[0112] The control unit 101 selects a proxel based on the calculation result of the learning model MD1 (step S305 ), and reads out the process conditions stored in association with the selected proxel (step S306 ).

[0113] The control unit 101 executes simulation using the simulator SIM based on the process conditions read from the proxel database DB to calculate the shape of the object to be processed (step S307 ), and calculates a reward based on the calculated shape of the object to be processed (step S308 ).

[0114] The control unit 101 determines whether the learning is completed (step S309), and if it is determined that the learning is not completed (S309: No), updates the model parameters including the weights and biases between the nodes constituting the learning model MD1 (step S310), and returns the process to step S304.

[0115] When it is determined that the learning is finished ( S309 : Yes), the control unit 101 stores the finally obtained model parameters in the storage unit 102 as model parameters of a new learning model (step S311 ).

[0116] As described above, in the third embodiment, the learned learning model MD1 can be utilized to generate a new learning model through fine-tuning, so even when a learning model is generated for a new object to be processed, the time required for learning can be shortened.

[0117] exist Fig.11 In the flowchart of , fine-tuning is described when the initial shape is different, but even when the data of the proxel stored in the proxel database DB is changed, a new learning model can be generated without spending time by fine-tuning.

[0118] (Implementation 4)

[0119] In the fourth embodiment, a configuration for evaluating a recipe set by a user using the learning model MD1 will be described.

[0120] Fig.124 is a flowchart for explaining the evaluation procedure of a process recipe. The control unit 101 acquires the process recipe set by the user (step S401). The control unit 101 can acquire the process recipe from a user terminal (not shown) via the communication unit 103, for example.

[0121] The control unit 101 reads the model parameters of the learning model MD1 from the storage unit 102 (step S402). Here, the model parameters of the learning model MD1 generated for the same or similar object to be processed as assumed in the process recipe set by the user are read.

[0122] The control unit 101 acquires the initial shape and the target shape (step S403 ), and inputs these difference data into the learning model MD1 , thereby executing calculation based on the learning model MD1 (step S404 ).

[0123] The control unit 101 calculates the evaluation value of the proxel selected in the process recipe set by the user (step S405). For example, based on the calculation result of step S404, the control unit 101 calculates the probability that the proxel selected by the user is selected by the learning model MD1 as the evaluation value.

[0124] The control unit 101 reads out the process conditions from the proxel database DB based on the information of the proxel selected by the user (step S406 ), and calculates the shape using the simulator (step S407 ).

[0125] The control unit 101 determines whether the process is completed (step S409 ). If it is determined that the process is not completed ( S409 : No), the process returns to step S404 , and in the next step, the evaluation of the proxel selected by the user is performed.

[0126] When it is determined that the process is finished (S409: Yes), the control unit 101 outputs the evaluation result of the process recipe including the evaluation value of each proxel (step S410). The control unit 101 may send the calculated evaluation result of the process recipe from the communication unit 103 to the user terminal or display it on the display unit 105.

[0127] As described above, in the fourth embodiment, by using the learned learning model MD1, the process recipe set by the user can be quantitatively evaluated.

[0128] The matters described in each embodiment can be combined with each other. In addition, the independent claims and dependent claims described in the claims can be arbitrarily combined in all combinations without considering the reference form. In addition, although the form of recording the reference to two or more other claims in the claims (multiple dependent claim form) is used, it is not limited to this. It is also possible to record in the form of recording multiple dependent claims (multiple reference multiple claims) that at least reference one multiple dependent claim.

[0129] The embodiments disclosed herein are illustrative in all aspects and should not be considered restrictive. The scope of the present disclosure is indicated by the claims rather than the above-mentioned meaning, and is intended to include all modifications within the meaning and scope equivalent to the claims.

[0130] Description of Reference Numerals

[0131] 100…information processing device; 101…control unit; 102…storage unit; 103…communication unit; 104…operation unit; 115…display unit; PG1…learning program; PG2…recipe building program; SIM…simulator; MD1, MD2…learning model; DB…proxel database.

Claims

1. A model generation method, The following processing is performed by the computer: Acquire a plurality of variable groups for constructing a process recipe, state data representing a state of a processed object before executing a specific step of the process recipe, and target data representing a target state of the processed object; Selecting a variable group from the plurality of variable groups, and evaluating a state of the object to be processed obtained by executing a step represented by the selected variable group; as well as A model for constructing a process recipe is generated by reinforcement learning using the acquired state data and target data and a reward determined according to the evaluated state of the object to be processed.

2. The model generation method according to claim 1, wherein: The model is configured to output information on a variable group recommended in each step of the process recipe when data based on state data indicating a state of a processed object and target data indicating a target state of the processed object is input.

3. The model generation method according to claim 2, wherein: The state data is image data showing the shape of the object to be processed before the specific step is performed. The target data is image data representing a target shape of the object to be processed, The computer executes a process of inputting difference data between image data representing the shape of the object to be processed and image data representing the target shape into the model.

4. The model generation method according to claim 1, wherein: The computer executes the following processing: executing the one step through a simulator.

5. The model generation method according to claim 1, wherein: The computer performs the following processing: The state of the object to be processed obtained by executing the one step is evaluated by calculating the change in the state of the object to be processed before and after executing the one step and the difference between the state of the object to be processed after executing the one step and a target state.

6. The model generation method according to claim 5, wherein: The computer performs the following processing: The reward is determined based on the calculated change in state and the calculated difference from the target state.

7. The model generation method according to claim 1, wherein: The model is constructed to output information on a recommended variable group in the i-th step upon inputting state data representing the state of the object to be processed before executing the i-th step in the process recipe to be constructed and target data representing the target state of the object to be processed, as well as information on a variable group selected when executing the i-1-th step, wherein i is an integer greater than 2.

8. The model generation method according to claim 7, wherein: The computer performs the following processing: The reward is determined according to whether the variable group selected when executing the (i-1)th step is the same as the variable group selected when executing the (i)th step.

9. The model generation method according to claim 1, wherein: The computer performs the following processing: The process is performed multiple times, and the remuneration is determined according to the state of the object to be processed finally obtained in each process and the number of steps in each process.

10. A computer program for causing a computer to execute the following processing: Acquire a plurality of variable groups for constructing a process recipe, state data representing a state of a processed object before executing a specific step of the process recipe, and target data representing a target state of the processed object; Selecting one variable group from the plurality of variable groups, and evaluating a state of the object to be processed obtained by executing a step represented by the selected one variable group; as well as A model for constructing a process recipe is generated by reinforcement learning using the acquired state data and target data and a reward determined according to the evaluated state of the object to be processed.

11. The computer program according to claim 10, wherein: Used to cause the computer to perform the following processing: A new model is generated for another object to be processed that is different from the object to be processed, using the model parameters of the generated model as initial values.

12. A computer program for causing a computer to execute the following processing: Acquire state data indicating a state of a processed object before executing a specific step of a process recipe, and target data indicating a target state of the processed object; Inputting the acquired state data and target data into a model generated by the model generation method according to any one of claims 1 to 9, and performing calculations based on the model; Determining a variable group to be used in the step based on the operation results of the model; as well as The process recipe is constructed based on groups of variables determined respectively in a plurality of steps.

13. The computer program according to claim 12, wherein: The model is configured to output the value of the action-value function for each of the plurality of variable groups. The method is used to cause the computer to execute the following processing: outputting the value of the action-value function of the variable group selected in each step.

14. The computer program according to claim 12, wherein: Used to cause the computer to perform the following processing: A partial visualization of the state data will facilitate selection of the variable group.

15. The computer program according to claim 12, wherein: Used to cause the computer to perform the following processing: The process recipe set by the user is compared with the process recipe constructed by the model.

16. An information processing device comprising: A storage unit storing a plurality of variable groups for constructing a process recipe; an acquisition unit that acquires state data indicating a state of the object to be processed before executing a specific step of the process recipe and target data indicating a target state of the object to be processed; an evaluation unit that selects one or more variable groups from the plurality of variable groups and evaluates a state of the object to be processed obtained by executing one or more steps represented by the selected one or more variable groups; and The generation unit generates a model for constructing a process recipe by reinforcement learning using the acquired state data and target data and a reward calculated based on the evaluated state of the object to be processed.

17. An information processing device comprising: an acquisition unit that acquires state data indicating a state of a processed object before a specific step of a process recipe is executed, and target data indicating a target state of the processed object; A calculation unit that inputs the acquired state data and target data into a model generated by the model generation method according to any one of claims 1 to 9, and performs calculations based on the model; a determination unit that determines a variable group to be used in the step based on a calculation result of the model; as well as The construction unit constructs the process recipe based on the variable groups respectively determined in the plurality of steps.

Images