Proxy interpreter for upgrading automated retention systems

Abstract

The present disclosure relates to an agent interpreter for upgrading an automated reservation system. The present disclosure generally relates to upgrading an existing automated reservation system. More specifically, the present disclosure relates to systems and methods for an agent interpreter system for collecting and merging setup, configuration, operation, and quality inspection data from multiple interface devices and controllers of a reservation system, and then using the merged data to build a reinforcement learning module to automatically perform all functions without the intervention of a human operator. The merged data in the agent interpreter module can be further analyzed using deep learning methods for data analysis and artificial intelligence to reliably and consistently classify defect criteria for products in order to further improve the quality of inspections. Defect criteria classification enables the agent interpreter system to highlight potential problems and help preventive maintenance of the reservation automation system.

Description

Technical Field

[0001] The present disclosure relates to an agent interpreter for upgrading an automated retention system. Background Art

[0002] In the field of automated manufacturing, it has become very important to be able to adapt computing and information processing capabilities to a more competitive, technologically advanced and error-free environment. However, since legacy systems are critical components in any production automation line, a lot of effort and expense must be spent in trying to completely rewrite the legacy system software or move or migrate the system functions to a more efficient, practical and cost-effective production environment. Rewriting the legacy system from scratch is usually not a practical option due to the inherent burden of the system, the risk of failure, data loss and the lack of understanding of how the system architecture of the legacy system was designed and how it actually performs internally. The reason is that all support from the original equipment manufacturer (OEM) is cut off.

[0003] Automated systems have been used in various microelectronic manufacturing and packaging processes. For example, in a typical semiconductor manufacturing facility (Fab), sliced ​​wafers are usually loaded onto equipment after setting and configuring device parameters. These processes are usually completed by operators who are prone to errors and are also affected by the fact that each operator can set and configure device parameters in different ways for a specific batch. After processing the wafers, the operator also needs to re-check the defective silicon chips and decide whether they are really defective or whether they should be reclassified as non-defective. Here, the human factor is affected by many errors. The manual operation of equipment in manufacturing facilities has gradually been replaced by automated processing to alleviate the expensive semiconductor manufacturing problems associated with non-automated manual operations.

[0004] Even after legacy manufacturing systems reach a point where the OEM decides to stop upgrade support or forces the customer to purchase a new model of equipment to accommodate new inspection capabilities or simply automate a specific task or process, some of the manually-operated processing continues. Manufacturers are caught in a dilemma because the increased capital expenditure for purchasing new models of equipment will increase their overall production costs as their old but reliable legacy systems are abandoned. Some critical manual operations involving human operators for setup, configuration and verification of defects or classification of certain types of new defects remain essential to ensure defect-free products are delivered to customers. Such manual operations involving human inspectors are known to be prone to errors during operation, inspection, classification, documentation and training, as human errors and fatigue are constant barriers to maintaining high efficiency and optimal quality.

[0005] Additionally, setting up a legacy manufacturing system for inspecting new silicon chips or integrated circuits is highly dependent on the ability, experience, and training of the operators. Selecting the correct recipe file for a specific machine setup is particularly important when encountering multiple types of silicon chips from the same product line. Recipe or configuration setting files accumulate over the years, and new human inspectors can find it difficult to select the correct file for the optimal settings for the machine.

[0006] Another problem area in manual operations at any process involves the collection and classification of data. The data can be in the form of parameter settings, defect classification, data collection, etc. related to the manufacturing process. Manufacturing operators or inspectors often manually enter data at each process step and interact with the system computer program several times for each single wafer batch processed. In addition, there are inconsistencies between different operators / inspectors, which further leads to error-prone quality inspections. Therefore, the consistency issue is an issue that needs to be properly addressed.

[0007] What is clearly needed by manufacturers is an adequate solution or framework for ensuring that the multiple interfaces that communicate with legacy systems are fully and securely integrated through tools that remain transparent to the manufacturer / end user and also introduce new technology that provides a fully automated reinforcement learning system that enables them to continue to use the existing legacy machine base and eliminate or minimize all human intervention (whether it is related to machine setup or post-inspection quality checks) to ensure high consistency in accuracy and repeatability for high-quality output. While this requirement can apply to legacy machines, it also applies to newer equipment that may still require a human to make certain key decisions at different processing steps. Summary of the invention

[0008] The present invention, which will be referred to as "agent interpreter" hereinafter, provides systems and methods for automating manufacturing processes by configuring a hardware agent interpreter unit that will build domain knowledge through reinforcement learning to operate a piece of reserve equipment by monitoring each individual activity of a human inspector on a mouse / keyboard and a set of input / output ports. The domain knowledge resident in the agent interpreter will be used to control the reserve equipment and ultimately eliminate the need for a human inspector. The agent interpreter system enables the reserve system to adapt and expand to manufacture newer products without human intervention, whether the agent interpreter system is related to the operation of the reserve equipment or in the process of quality control.

[0009] In one embodiment of the present invention, a system and method for implementing an agent interpreter to manage and control at least one reservation system is provided. The system and method include the following steps: (a) capturing an image of a display monitor that a setup and quality control operator is viewing and checking; (b) collecting keyboard position coordinates and mouse position coordinates relative to the captured image during the setup and configuration process; (c) recording and storing mouse, certain input / output ports, and keyboard commands triggered by the operator, and analyzing the activities initiated by the relevant commands; (d) analyzing and monitoring the subsequent results displayed on the monitor and all input / output ports activated by the command; (e) mapping the responses made by the reservation system to build a response library based on the activated commands; and (f) using the response library to analyze multiple command activities and then control the reservation device without any human intervention. Ultimately, the agent interpreter overwrites the input mouse and keyboard commands of the reservation system with its own command sequence, effectively acting as a human controlling the reservation system. Thereby achieving the ultimate goal of automating the reservation system without installing any software on the reservation system itself.

[0010] In another embodiment of the present invention, a system and method for creating configuration and recipe files for multiple devices is provided within an agent interpreter to automate equipment setup. The system and method include the following steps: (a) capturing an image of a display monitor being viewed by a quality control operator; (b) collecting keyboard position coordinates, mouse position coordinates, and certain input ports relative to the captured image during the setup and configuration process; (c) recording and storing the mouse, certain input / output ports, and keyboard commands triggered by the operator, and analyzing the activities initiated by the relevant commands; (d) creating a recipe or setup file containing configuration parameters for a specific device; and (e) using the recipe file to automatically set up and configure the retention system without human intervention during subsequent production processes.

[0011] In another embodiment of the present invention, a system and method for implementing a deep learning module is provided within an agent interpreter to improve the quality of defect inspection. The system and method include the following steps: (a) classifying defect standards as instructed by human inspectors; (b) applying deep learning techniques to classified defects and improving the defect identification process; (c) creating new domain knowledge based on deep learning techniques; and (d) where applicable, using new domain knowledge to inspect and reclassify defects to further improve the accuracy and repeatability of inspections. The new reclassification results are used by the agent interpreter to change the inspection results in the retention system by overwriting mouse and keyboard inputs and replicating how humans would manually change the results. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The present invention will be described with reference to specific embodiments thereof and with reference to the accompanying drawings, in which like reference numerals refer to like parts, and in which:

[0013] Figure 1 is a block diagram of a typical currently existing automated system having a computer system that causes a retention system to perform a method according to a computer program;

[0014] Figure 2 is a block diagram of an embodiment of an automation system according to the present invention having a computer system connected to an agent interpreter that collects information from devices such as a mouse, certain input / output ports and keyboard commands associated with the devices, images displayed on a monitor during setup and configuration of the reservation system;

[0015] Figure 3 It is based on the description Figure 1 Flowchart of the processing steps followed by a human operator of the system during a typical inspection and triage inspection process.

[0016] Figure 4 It is described according to Figure 2 Flowchart of processing steps during training or teaching of an embodiment of the present invention is shown.

[0017] Figure 5 It is based on the description Figure 2 Flowchart of the processing steps followed during the creation of a reinforcement learning module of the present invention is shown.

[0018] Figure 5a is a flow chart depicting steps for reinforcement learning according to an embodiment of the present invention.

[0019] Figure 6 is a flow chart illustrating the automatic operation of the agent interpreter system during normal operation of the machine without intervention by a human operator. DETAILED DESCRIPTION

[0020] The present invention relates to a method for automating the setup, configuration and operation of a microelectronic manufacturing process. Although the embodiments provided below relate to a method for automating a microelectronic manufacturing process for manufacturing semiconductor devices, it should be understood that the method of the present invention can be used to automate any microelectronic manufacturing process to manufacture, for example, flat panel devices, disk drive devices, etc. It is intended to automate a set of processes to enable the retention device to be used to minimize human intervention, improve the quality of the process by using deep learning techniques to improve the quality of the manufacturing process, and extend the service life of the retention device in the process. The present invention relates to a method for automating a manufacturing process rather than automating a specific type of device or manufacturing process.

[0021] Figure 1 is a block diagram view of a general automation system that includes various components of a mechanical manufacturing system and a control system called a retention system. Figure 1 Inferred from the PC control system 28, generally including certain devices, such as processors, memory (not shown), input devices including mouse 32 and keyboard 26, output devices including display 24, emergency buttons (not shown), signal tower indicators, etc., these devices are controlled through input / output ports 30, some of which are connected through their associated interfaces through USB, Ethernet ports, etc. It should be understood that more peripheral devices can be linked to the control system 28 to interface with external networks or devices for implementing specific types of processing. The control system 28 interfaces with the manufacturing reservation equipment 20 to perform the steps of manufacturing, inspection, and classification, and outputs necessary data to an external interface (not shown) for data merging and management.

[0022] Figure 2 is a block diagram view of an embodiment of the automation system of the present invention implemented using a proxy server 42 that communicates with various peripheral devices of the legacy system to control the automation equipment 20 through the PC control system 28. In the new system architecture of the present invention, all devices that were originally connected to the PC control system 28 are now connected to the proxy interpreter 42, which in turn communicates with the PC control system 28 through their respective ports. Figure 2 , input devices including a mouse 32 and a keyboard 26 are connected to the proxy server 42 via interfaces 50 and 48, respectively. The display port of the PC control system 28 is connected to the input display port of the proxy server via interface 34. The proxy interpreter is connected to the mouse port via interface 54, to the keyboard port via interface 52, and communicates with the PC control system 28, monitors and records all activities, and builds domain knowledge for a specific part of the automation device (in this case, the automation device 20). The display monitor 24 is connected to the proxy server 42 via interface 40. It should be understood that more peripheral devices can be linked to the proxy interpreter 42 to enable it to perform additional tasks when needed. The proxy interpreter uses the domain knowledge built over time to control the automation device 20 through the PC control system 28 interface to perform the steps of controlling and operating the retention machine 20.

[0023] Figure 3is a flowchart of a general processing flow in an automated machine. The flowchart starts at step 60. In step 62, the operator scans the batch code from the batch document and downloads information related to the batch. Then, the operator selects the relevant setting file from the configuration file list based on the device to be processed. In step 64, the operation of the equipment begins and the necessary processing steps (in this case, the inspection of the silicon chip) begin. In step 70, the computer program controlling the machine checks whether the silicon chip undergoing inspection is the last chip. If it is the last silicon chip, the program proceeds to step 90. If it is not the last silicon chip, the program moves to step 74. In step 74, the operator compares the results of the inspected silicon chip with the results of the silicon chip in the wafer map data file. If the results match, the program moves to the next step 76. If the results do not match, the program proceeds to step 82, in which the operator takes a closer look at the defect identified by the inspection program and determines whether it is indeed a defect and does not match the results in the downloaded wafer map file. The operator classifies the defective silicon chip under the appropriate category and updates the information in step 86. When looking at the defect more closely in step 82, if the operator determines that the silicon die identified as defective is not defective and the result matches the downloaded wafer map inspection result, the operator moves the determination to the next step 76 without updating the wafer map data file. The program then moves from step 76 to step 70, where the entire process is repeated until the last silicon die in the wafer.

[0024] Figure 4 is a flow chart of an embodiment of the automated system of the present invention implemented using the agent interpreter 84 for use during training or teaching, and all other processing steps of the flow chart are the same as Figure 3 The same flowchart as in. Agent Interpreter Reference Monitor 24 ( Figure 2 ) The image displayed on the monitor comes from the mouse 32 ( Figure 2 )、Keyboard 26( Figure 2 ) and uses a reinforcement learning module to learn the operation of the device 20. All data related to the controls and commands encountered at the output of steps 74 and 82 are merged and stored in the agent interpreter. The merged data is analyzed to help build a reinforcement learning module, which is then used to automatically control the device 20 without the involvement of a human operator.

[0025] Figure 5 is a flow chart showing the steps followed in creating a reinforcement learning module that primarily learns and stores a sequence of operations for a retention device. Figure 4 The proxy interpreter 84 shown is Figure 5100 is the entry point for the reinforcement learning resident in the agent interpreter. Preferably, the first step may be to capture an image on a display monitor as in step 102. All information and data collected from external interface devices (e.g., a mouse, certain input / output ports, and keyboard input or commands) are related to the current image captured and stored in the agent interpreter. When the machine is set up and configured in step 104, the reinforcement learning module stores and merges the inputs and outputs collected as part of the process triggered by the operator. As shown in step 106, the recording and documentation of operational activities and the intervention of the operator to trigger any specific process (including but not specific to the verification of the inspected device) preferably continues for each individual silicon chip on the wafer. In step 108, the reinforcement learning module creates an operational flow for various commands related to the processes in the operational sequence of the reserved equipment. The agent interpreter uses these commands and their associated processes to operate the reserved equipment without human intervention. Step 110 represents the end of the agent interpreter reinforcement learning module process flow chart. The agent interpreter can also analyze operator input regarding quality control and defect classification to create an automatic defect classification (ADC) method using deep learning technology, enabling the retained equipment to perform quality inspections with greater accuracy and reliability. The deep learning module will reside in the agent interpreter along with the reinforcement learning module, and together they will help improve the performance of the retained equipment in terms of both features and productivity. The agent interpreter helps increase the service life of the retained equipment, which is the main feature of the present invention.

[0026] exist Figure 5a In more detail, the Figure 5 Steps related to the reinforcement learning module 108 in step 150. The reinforcement learning module in step 150 is implemented using the "Dueling Double Deep Q Network" (D3QN) architecture, which consists of two networks: the MAIN network for learning through interaction with the environment, which uses rewards for positive behaviors and penalties for negative behaviors to determine the correct actions in the interactive environment, and the TARGET network (which is a frozen version of MAIN in k training steps) used to stabilize the dynamic target. The D3QN system also uses two streams: the VALUE stream 152, which is used to learn the common Q value (quality) of each machine state (the offset value of all actions in this state); and the ADVANTAGE stream 154, which is used to learn which action should be taken in a specific state. The experience buffer 170 continuously updates the advantage streams in steps 172 and 174 to improve the Q value of each machine state, sums the Q values ​​at step 176, and then returns to the D3QN module in step 150.

[0027] The main network obtains inputs from feature maps 164 generated by an object detection model for display screen 178 (e.g., improved YOLO (Look Only Once)) within frozen model 160, and confidence vectors of images and text in the screen from deep learning networks (e.g., improved YOLO and improved CTPN (Connectionist Text Proposal Network), respectively). The confidence vectors are used as filters to ensure that the action classifier 162 does not take actions that are irrelevant to the current state. In addition, a custom-built LSTM (Long Short-Term Memory) model is used to distinguish similar screens in different states.

[0028] Figure 6 is a flow chart of the steps during normal operation of the machine, in which the agent interpreter system is based on Figure 5 All commands are initiated by the reinforcement and deep learning modules built during the process flow. The typical proxy interpreter process starts at step 120 and continues to scan the batch information or documents from the batch traverser in step 122. The batch information is further analyzed by the proxy interpreter and the relevant keyboard and mouse commands are sent to the central server in step 124 to download the settings and configuration information to the reserved machine control system. In step 126, the control system in the reserved machine will start the operation of the machine by checking whether the current silicon chip of the camera being inspected is the last chip. If so, the process jumps to step 136 and ends Figure 6 If not, the operation proceeds to the next step 130, in which defects and other information related to the current silicon chip under the camera are extracted from the wafer map file. In step 133, the silicon chip under the camera is further inspected using a deep learning module to perform highly complex analysis for enhanced inspection, thereby obtaining more reliable inspection results.

[0029] The deep learning module in step 133 is constructed using an architecture including an improved EfficientNet and an improved Faster-RCNN (region-based convolutional neural network). These deep learning models are trained to identify defects on the surface of an object by analyzing input images using an improved ResNET-101 (residual network) layer.

[0030] In step 134, the results obtained at step 133 are compared with the results in step 130. If the comparison results are the same, the operation proceeds to step 128, in which the machine indexes the wafer to the next silicon die to be inspected. If the comparison results in step 134 are not the same, then in step 132, the proxy interpreter sends the relevant keyboard and mouse commands to the retention control system to update the current silicon die results in the wafer map file. In fact, in step 132, the results existing in the wafer map file in step 124 are overwritten by the new results for the silicon die being inspected. The operation proceeds to step 128, in which the next silicon die to be inspected is indexed under the camera. Subsequently, the operation proceeds to step 126. The process continues and repeats until the last silicon die to be inspected. This key basic feature of applying the new and enhanced inspection method to the retention machine through the proxy interpreter system is the main feature of the present invention.

[0031] The methods described herein do not necessarily need to be performed in the order described, and the order of steps of such methods should be understood to be exemplary only. Likewise, in methods consistent with various embodiments of the present invention, additional steps may be included in such methods, and certain steps may be omitted or combined.

[0032] Although embodiments of the present invention are described herein, it should be understood that the foregoing embodiments and advantages are merely examples and should not be construed as limiting the scope of the present invention or the claims. Many other modifications and embodiments may be designed by those skilled in the art by applying any neural-based computational model that will fall within the spirit and scope of the principles of the present disclosure. This teaching may also be easily applied to other types of retention systems. More particularly, within the scope of the present disclosure, the drawings, and the appended claims, a variety of variations and modifications in the arrangement of the subject combination arrangement are possible. In addition to variations and modifications in the arrangement, alternative uses will also be apparent to those skilled in the art.

Claims

1. An agent interpreter system, connected to a PC control system, which uses artificial intelligence to control a retention machine, the agent interpreter system comprising: a server communicatively coupled to the PC control system via a plurality of channels including inputs and outputs, which operates the retention machine with reference to the image displayed on the display terminal, receives and sends operation commands, teaching sequences and corresponding responses via hardware interfaces including Ethernet, USB, the operation commands being related to the image; and The reinforcement learning module is arranged as follows: learning and storing said operating commands and sequences as part of a process triggered by an operator when setting up and configuring said reserved machine; constructing domain knowledge of a plurality of operating states of the reserved machine using a competitive dual deep Q-network implementation to create a set of recipe files with various parameters for a specific device type; The competitive dual deep Q network is implemented by two streams, the two streams including a value stream for learning a common quality value for each operating state of the machine and a dominance stream for learning a corresponding action for a given state of the machine, and An operation flow is created for the operation commands related to a process, so that the agent interpreter can operate the reserved machine without human intervention.

2. The proxy interpreter system according to claim 1, wherein: The hardware interface includes input / output ports, USB ports, Ethernet ports, VGA ports, mouse, keyboard and display interfaces to operate the retention system through the PC control system, and the hardware interface is used to learn and create the domain knowledge required to operate the retention system.

3. The agent interpreter system according to claim 2, wherein: The agent interpreter can reside as a software module within the PC control system that controls the reservation system, and the agent interpreter can utilize its interface to learn and create the domain knowledge required to operate the reservation system.

4. The agent interpreter system according to claim 2, wherein: The proxy interpreter accumulates the domain knowledge from interactions with the retention system through commands and responses monitored through various interfaces for all operational states of the retention system.

5. The agent interpreter system according to claim 4, wherein: Commands and responses stored in a recipe file for a specific device type are obtained based on keyboard commands and mouse movements made by a human operator with reference to the image on the display, and the commands and responses are combined with related responses received from the retention system over the Ethernet and I / O interfaces to the proxy interpreter to create domain knowledge for the retention system.

6. The agent interpreter system according to claim 4, wherein: The domain knowledge created by applying deep learning techniques and continuous reinforcement learning is then used to operate the retention system without the intervention of a human operator.

7. A method for constructing an artificial intelligence module by training an agent interpreter system through a deep learning module, the agent interpreter system being used to select actions to be performed by interacting with a reserved system and by receiving observations of the state and operation sequence of the system, wherein: The method comprises: A set of activities triggered from the reserved system interaction environment is obtained by a reinforcement learning module, wherein each activity includes a process representing a set of events and a related command or a set of related commands in response to the activity; Domain knowledge is built by implementing reinforcement learning of multiple operating states of the reserved system using a competitive dual deep Q network to create a set of recipe files with various parameters for a specific device type; the competitive dual deep Q network is implemented by two streams, the two streams including: a value stream for learning a common quality value for each operating state of the machine and an advantage stream for learning a corresponding action for a given state of the machine; processing observations and actions associated with said observations during setup and operation of said retention system and implementing methods for rewarding and punishing positive and negative behaviors to arrive at an optimal behavioral model for efficient operation of said retention system; creating, by the reinforcement learning module, a confidence vector for each individual action to act as a filter to prevent the action classifier from responding to irrelevant operating states of the selection of the retention system; By implementing a "competitive dual deep Q network", the advantage flow of the reinforcement learning module is continuously reviewed and updated.

8. The method according to claim 7, wherein: The dominant stream is regularly updated via an experience buffer to aid quality improvement and fine-tune the quality value for a given machine state.

9. The method according to claim 7, wherein: The deep learning module is arranged to improve the quality of defect inspection of a surface of an object, and the deep learning module comprises: a frozen model, wherein the frozen model is generated and continuously updated using an object detection model by assigning confidence vectors to ensure that an action classifier does not take action on any irrelevant state of the retention system; a customized long short-term memory model for distinguishing similar images in different states of the retention system; A set of feature maps using the modified look-once and modified linker text proposal networks for images and text, respectively.

10. The method according to claim 9, wherein: The reinforcement learning module implemented on the image and text information is derived from the display screen, and any actions transmitted or taken to the retention system are streamed through the agent interpreter.

11. The method according to claim 10, wherein: The domain knowledge created within the agent interpreter for all operating states of the reserved system is then utilized by the agent interpreter to automatically operate the reserved system without intervention by a human operator.

Images