Wafer access abnormality detection method, device, equipment and readable storage medium

Abstract

The present application provides a detection method, device, equipment and readable storage medium for wafer access anomaly, the method comprising: determining an image to be detected and a target detection model including a feature extraction module, a feature combination module, a correction module and anomaly detection module based on the execution action of a robotic arm; inputting the image to be detected into the feature extraction module to obtain a first feature data set; inputting the first feature data set into the feature combination module to obtain a feature data group; performing data correction processing on the feature data group through a generative adversarial network of the correction module to obtain a corrected feature data group; if the abnormal detection module determines that there are abnormal pixels in the feature data group based on the corrected feature data group and the feature data group, then a detection result of access anomaly is generated. The present application determines the image to be detected and the target detection model by executing an action, and detects whether the wafer access is abnormal based on the target detection model, thereby improving the detection accuracy.

Description

Technical Field

[0001] The present application relates to the field of semiconductor technology, and in particular to a method, device, equipment and readable storage medium for detecting wafer access anomalies. Background Art

[0002] Wafers are generally stored in bins to protect them and prevent them from being polluted by the environment. When accessing and using wafers, the wafers need to be taken out of the bin by a robotic arm. If there is an abnormality in the placement of the wafer in the bin, or if there is an abnormality in the movement of the robotic arm to grab or store the wafer, the wafer will be damaged and wasted. In the prior art, the placement of the wafer is generally detected by infrared detection, but it can only be detected when the wafer is stored in a bin, and it is impossible to determine the cause of the wafer abnormality, resulting in inaccurate detection. Summary of the invention

[0003] The present application provides a wafer access anomaly detection method, device, equipment and readable storage medium, which determines the image to be detected and the target detection model by executing actions by a robotic arm, thereby determining whether the wafer access is abnormal and improving the accuracy of detection.

[0004] In a first aspect, an embodiment of the present application provides a method for detecting wafer access anomalies, which is applied to a server of a wafer testing system, wherein the wafer testing system includes the server, a wafer, a material box for storing the wafer, and a robotic arm communicatively connected to the server, and the method includes:

[0005] Determine an image to be detected and a target detection model based on an execution action of the robot arm, wherein the execution action represents an action of the robot arm grabbing the wafer from the material box or placing the wafer into the material box, and the target detection model includes a feature extraction module, a feature combination module, a correction module and an abnormality detection module;

[0006] Input the image to be detected into the feature extraction module, and process the image to be detected by the feature extraction module to obtain a first feature data set, wherein the first feature data set includes different feature data, each of the feature data includes image width, image length and number of image channels, and the image width of each of the feature data is a preset image width, and the image length of each of the feature data is a preset image length;

[0007] Inputting the first feature data set into the feature combination module, combining the feature data in the first feature data set through the feature combination module to obtain a feature data group, wherein the number of image channels of the feature data group is the sum of the number of image channels of each feature data in the first feature data set;

[0008] Inputting the feature data group into the correction module, performing data correction processing on the feature data group through the generative adversarial network of the correction module, and obtaining a corrected feature data group;

[0009] The corrected feature data group and the feature data group are input into the abnormality detection module. If the abnormality detection module determines that there are abnormal pixels in the feature data group, a detection result of the wafer access abnormality is generated.

[0010] In a second aspect, an embodiment of the present application provides a wafer access abnormality detection device, which is applied to a server of a wafer testing system, wherein the wafer testing system includes the server, a wafer, a material box for storing the wafer, and a robotic arm communicatively connected to the server, and the device includes:

[0011] a determination unit, configured to determine an image to be detected and a target detection model based on an execution action of the robot arm, wherein the execution action represents an action of the robot arm grabbing the wafer from the material box or placing the wafer into the material box, and the target detection model includes a feature extraction module, a feature combination module, a correction module, and an anomaly detection module;

[0012] A feature extraction unit, used for inputting the image to be detected into the feature extraction module, and processing the image to be detected by the feature extraction module to obtain a first feature data set, wherein the first feature data set includes different feature data, each of the feature data includes image width, image length and number of image channels, and the image width of each of the feature data is a preset image width, and the image length of each of the feature data is a preset image length;

[0013] a combining unit, configured to input the first feature data set into the feature combining module, and combine the feature data in the first feature data set through the feature combining module to obtain a feature data group, wherein the number of image channels of the feature data group is the sum of the number of image channels of each feature data in the first feature data set;

[0014] A correction processing unit, used for inputting the feature data group into the correction module, performing data correction processing on the feature data group through the generative adversarial network of the correction module, and obtaining a corrected feature data group;

[0015] A generating unit is used to input the corrected feature data group and the feature data group into the abnormality detection module, and if the abnormality detection module determines that there are abnormal pixels in the feature data group, a detection result of the wafer access abnormality is generated.

[0016] In a third aspect, an embodiment of the present application provides a terminal device, comprising at least one processor, a communication interface and a memory, wherein the communication interface is used to send and / or receive data, the memory is used to store a computer program, and the at least one processor is used to call the computer program stored in the memory to implement any method as described in the first aspect of the present application.

[0017] In a fourth aspect, an embodiment of the present application provides an electronic device, comprising a processor, a communication module, a memory, a communication interface, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the processor, and the program includes instructions for executing the steps of any method of the first aspect of the present application.

[0018] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program for electronic data exchange, wherein the computer program enables a computer to execute part or all of the steps described in any method of the first aspect of the embodiment of the present application.

[0019] In a sixth aspect, the present application provides a computer program, wherein the computer program is operable to cause a computer to execute some or all of the steps described in any method of the first aspect of the present application. The computer program may be a software installation package.

[0020] It can be seen that in the embodiment of the present application, the image to be detected and the target detection model are determined based on the execution action of the robotic arm. The execution action characterizes the action of the robotic arm grabbing wafers from a material box or placing wafers into a material box. The target detection model includes a feature extraction module, a feature combination module, a correction module and an abnormality detection module. The target detection model is determined by executing the action to improve the detection efficiency. The image to be detected is input into the feature extraction module, and the image to be detected is processed by the feature extraction module to obtain a first feature data set, the first feature data set includes different feature data, each feature data includes image width, image length and image channel number, and the image width of each feature data is a preset image width, and the image length of each feature data is a preset image length; the first feature data set is input into the feature combination module, and the feature data in the first feature data set are combined by the feature combination module to obtain a feature data group, and the image channel number of the feature data group is the sum of the image channel numbers of each feature data in the first feature data set; the feature data group is input into the correction module, and the feature data group is subjected to data correction processing by the generative adversarial network of the correction module to obtain a corrected feature data group; the corrected feature data group and the feature data group are input into the abnormal detection module, and if the abnormal detection module determines that there are abnormal pixels in the feature data group, a detection result of wafer access abnormality is generated. In this application, the target detection model corresponding to the execution action is used to process the image to be detected to improve the detection efficiency. And the target detection model can extract multiple feature data of the image to be detected, and increase the richness of the extracted feature data. A feature data group is generated based on the first feature data set, and a corrected feature data group corresponding to the feature data group is obtained to determine whether there are abnormal pixels in the image to be detected, and then determine whether the wafer access is abnormal, thereby improving the detection accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0022] Figure 1 A schematic diagram of the structure of a wafer testing system provided in an embodiment of the present application;

[0023] Figure 2 A schematic diagram of the structure of a server provided in an embodiment of the present application;

[0024] Figure 3 A schematic diagram of a process flow of a wafer access abnormality detection method provided in an embodiment of the present application;

[0025] Figure 4 A schematic diagram of an image to be detected provided in an embodiment of the present application;

[0026] Figure 5 A block diagram of the functional units of a wafer access anomaly detection device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0027] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0028] The terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or devices.

[0029] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0030] For details, please refer to Figure 1 , Figure 1 A schematic diagram of the structure of a wafer testing system provided in an embodiment of the present application. The wafer testing system includes a server, at least one robotic arm and a material box. The server can be connected to at least one robotic arm. Each of the at least one robotic arm is connected to the server so that each robotic arm can exchange data with the server through a network. Figure 1 As shown, the at least one robot arm may specifically include a first robot arm, a second robot arm, ..., an Nth robot arm. The material box is loaded with wafers, and one material box may correspond to one robot arm, or one material box may correspond to multiple robot arms, or multiple material boxes may correspond to one robot arm. The number of material boxes and the number of robot arms may be set according to actual needs, and are not limited here.

[0031] See also Figure 2 , Figure 2 A schematic diagram of the structure of a server provided in an embodiment of the present application, such as Figure 2 As shown, the server includes a processor 201, a communication module 202, a memory 203, a program 204 and an image acquisition module 205. The number of the processors 201 can be set according to actual needs. The processor 201 is communicated with the memory 203, the communication module 202 and the image acquisition module 205 through an internal communication bus.

[0032] Among them, the program 204 is stored in the above-mentioned memory 203 and is configured to be executed by the above-mentioned processor 201. The program 204 includes instructions for executing any step in the following method embodiment. It can be understood that the number of programs 204 can be set according to actual needs, and there is no specific limitation here.

[0033] The processor 201 may be, for example, a central processing unit (CPU), a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. It may implement or execute various exemplary logic blocks, units and circuits described in conjunction with the disclosure of the present application. The processor 201 may also be a combination that implements a computing function, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, and the like. The communication unit may be a communication module 202, a transceiver, a transceiver circuit, and the like, and the storage unit may be a memory 203.

[0034] The memory 203 may be a volatile memory or a nonvolatile memory, or may include both volatile and nonvolatile memories. Among them, the nonvolatile memory may be a read-only memory (ROM), a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), and direct RAM bus RAM (DR RAM).

[0035] The image acquisition module 205 can acquire video information or image information, such as video information when the robot takes the wafer or image information after the wafer is stored. It is understandable that the above-mentioned image acquisition module 205 may not be integrated into the server, and there is no specific limitation here. The server in this application may also be replaced by other control devices, or the server and the robot are set as one, and there is no specific limitation here.

[0036] Please combine the following Figure 3 The method in the embodiment of the present application is described in detail:

[0037] See also Figure 3 , Figure 3 A schematic diagram of a process flow of a wafer access abnormality detection method provided in an embodiment of the present application is shown in FIG. Figure 3 As shown, a method for detecting abnormal wafer access is used for a server of a wafer testing system, wherein the wafer testing system includes the server, a wafer, a material box for storing the wafer, and a robot arm in communication with the server, and the method includes:

[0038] Step S301, determining an image to be detected and a target detection model based on the execution action of the robotic arm.

[0039] Among them, the execution action represents the action of the robot arm grabbing the wafer from the material box or placing the wafer into the material box, and the target detection model includes a feature extraction module, a feature combination module, a correction module and an abnormality detection module. Specifically, the robot arm execution action is the action of the robot arm grabbing the wafer from the material box, or the action of the robot arm placing the wafer into the material box. If the execution actions performed by the robot arm are different, different detection models can be used for detection to improve detection efficiency. By detecting the images to be detected and the target detection models corresponding to different execution actions, it is quickly determined which link in the wafer picking up causes the abnormal picking up.

[0040] Step S302: input the image to be detected into the feature extraction module, and process the image to be detected by the feature extraction module to obtain a first feature data set.

[0041] Wherein, the first feature data set includes different feature data, each of which includes image width, image length and number of image channels, and the image width of each of the feature data is a preset image width, and the image length of each of the feature data is a preset image length. Specifically, after determining the image to be detected and the target detection model, the image to be detected is input into the feature extraction module of the target detection model, the feature extraction module processes the image to be detected, extracts different feature data of the image to be detected, and synthesizes the extracted feature data into the first feature data set to provide data support for subsequent detection.

[0042] Step S303: input the first feature data set into the feature combination module, and combine the feature data in the first feature data set through the feature combination module to obtain a feature data group.

[0043] Among them, the number of image channels of the feature data group is the sum of the number of image channels of each feature data in the first feature data set. The first feature data set is input into the feature combination module of the target detection model, and the feature data in the first feature data set are combined by the feature combination module. Specifically, the number of image channels of each feature data in the first feature data set is added to obtain the added number of channels, which is the number of channels of the feature data group, ensuring the diversity of the number of channels in the feature data. And the image width and image length of the feature data group are respectively the preset image width and the preset image length, which is convenient for subsequent data processing and improves the efficiency of data processing.

[0044] Step S304: input the feature data group into the correction module, and perform data correction processing on the feature data group through the generative adversarial network of the correction module to obtain a corrected feature data group.

[0045] The generative adversarial network includes a generator, which is trained in an adversarial way so that the generator can generate realistic sample data. After obtaining the feature data set, the feature data set is input into the generative adversarial network of the correction module to obtain a corrected feature data set, which represents the feature data set that should be obtained under the normal access wafer condition corresponding to the execution action.

[0046] In a possible example, the correction module may also include a convolutional autoencoder, the encoder of which is a dimensionality reduction technique that helps improve the performance of the model by compressing the input data and reducing the number of input values. After the feature data set is input into the correction module, the convolutional autoencoder is first used to reduce the dimension to obtain a reduced feature data set, and then the reduced feature data set is input into the generative adversarial network for correction to improve data processing efficiency.

[0047] Step S305 , inputting the corrected feature data group and the feature data group into the abnormality detection module, and if the abnormality detection module determines that there are abnormal pixels in the feature data group, generating a detection result of the wafer access abnormality.

[0048] Among them, after obtaining the corrected feature data group, the corrected feature data group and the feature data group are input into the abnormality detection module, and whether there are abnormal pixels in the feature data group is determined based on the corrected feature data group and the feature data group. If so, a detection result of wafer access abnormality is generated.

[0049] It can be seen that in this example, the image to be detected and the target detection model are determined by executing actions to improve the detection efficiency and accuracy. By detecting the image to be detected through the target detection model, it is possible to quickly determine whether there are abnormal pixels in the image to be detected, and then determine the wafer access abnormality, thereby improving the accuracy of detection.

[0050] In a possible example, the feature extraction module includes multiple feature extraction submodules and a multi-scale feature adjustment submodule, and the image to be detected is input into the feature extraction module, and the image to be detected is processed by the feature extraction module to obtain a first feature data set, including: inputting the image to be detected into the multiple feature extraction submodules of the feature extraction module for feature extraction to obtain reference feature data of multiple different scales; obtaining a reference feature data set composed of the reference feature data of multiple different scales; inputting the reference feature data set into the multi-scale feature adjustment submodule, and adjusting the image width and image length of each reference feature data in the reference feature data set to the preset image width and the preset image length respectively by the multi-scale feature adjustment submodule, so as to obtain the first feature data set composed of the adjusted reference feature data.

[0051] In a specific example, the feature extraction module includes a plurality of feature extraction submodules and a multi-scale feature adjustment submodule. The image to be detected is input into a plurality of feature extraction submodules for feature extraction, and a plurality of reference feature data of different scales are obtained, and a reference feature data set composed of a plurality of reference feature data of different scales is formed, and the image width, image length and image channel number between each two reference feature data in the reference feature data set are different. Specifically, each feature extraction submodule includes a plurality of parallel convolutional layers and a pooling layer connected to the convolutional layer, and the convolutional layer may include a convolutional kernel, and each convolutional kernel may correspond to an information channel, for example, there may be information channels for extracting contours, extracting textures, extracting colors, etc., and the input image to be detected can be processed simultaneously through the parallel convolutional layers, and the convolutional kernels of different convolutional layers are convoluted with the image to be detected, so that the feature data of different dimensions contained in the image to be detected can be obtained, so that it can be convenient for subsequent detection based on the features contained in the image to be detected. And a plurality of feature extraction submodules are stacked, so that the input of the latter feature extraction submodule is the feature map generated by the image to be detected output by the previous feature extraction submodule, and then the feature data corresponding to the feature maps of different scales and complexities are extracted. For example, the feature extraction module includes 9 feature extraction submodules, the 9 feature extraction submodules are stacked, and each feature extraction submodule includes three parallel convolution layers, the convolution kernels of the three convolution layers are 1x1 convolution kernel, 3x3 convolution kernel and 5x5 convolution kernel respectively, the three convolution layers receive the input image to be detected in parallel or receive the feature map output by the previous feature extraction submodule, due to different convolution kernels, the three convolution layers output feature maps of different sizes, and then the feature map output by the convolution layer is input to the pooling layer for pooling processing, thereby further reducing the dimension of the feature map, and extracting the main features to retain the most significant reference feature data. After obtaining the reference feature data set, the reference feature data set is input into the multi-scale feature adjustment submodule, and the image width and image length of each reference feature data in the reference feature data set are adjusted to the preset image width and preset image length respectively by the multi-scale feature adjustment submodule, and the first feature data set composed of the adjusted reference feature data is obtained.

[0052] It can be seen that in this example, feature data of the image to be detected is extracted through multiple feature extraction submodules containing parallel convolution layers, so that feature data corresponding to feature maps of different scales can be obtained, which improves processing efficiency and the richness of input samples. The image width and image length of the reference feature data obtained are adjusted through the multi-scale feature adjustment submodule to reduce the amplitude of subsequent processing and improve subsequent processing efficiency.

[0053] In a possible example, determining the image to be detected and the target detection model based on the execution action of the robot arm includes: if the execution action of the robot arm represents the action of the robot arm grabbing the wafer from the material box, then obtaining video information of the robot arm performing the execution action; setting the first image in the video information when the robot arm grabs the wafer in the material box as the image to be detected; searching for a first reference detection model corresponding to the execution action from a preset database, the preset database including the correspondence between different execution actions and different reference detection models; and setting the first reference detection model to the target detection model.

[0054] In a specific example, when the robot arm performs the action of grabbing a wafer from a material box, video information of the robot arm grabbing the wafer is obtained, and the video information records the process of the robot arm grabbing the wafer from the material box. After obtaining the video information, the video duration of the entire video information is determined, and the difference between the video duration and the preset duration is calculated. The preset duration represents the time required for the pre-detected robot arm to grab the wafer from the material box. If the difference is less than the preset difference, the video information is divided according to the duration division ratio to obtain multiple divided videos, and each divided video is sorted according to the corresponding time sequence, and the corresponding number of retained frames is determined according to the sequence number. Since different divided videos correspond to different time periods when the robot arm grabs the wafer, different numbers of retained frames can be set for different divided videos, so that the required images to be detected can be quickly extracted, saving resources and improving processing efficiency. For example, after the video information is segmented, the first segmented video, the second segmented video, and the third segmented video are obtained. The first segmented video is a video of the robot arm starting to grab the wafer, which includes the process of the robot arm running to the material box. Therefore, the number of reserved frames can be set to 20 for the first segmented video, and 20 frames of images are extracted from the first segmented video according to the number of reserved frames. The number of reserved frames for the second segmented video is set to 70, and 70 frames of images are extracted from the second segmented video. The number of reserved frames corresponding to each segmented video can be set according to actual needs, and there is no restriction here. Please refer to Figure 4 , Figure 4 A schematic diagram of an image to be detected provided in an embodiment of the present application. After acquiring the image of the video information, detect whether each image shows the content of the robot arm grabbing the wafer from the front of the material box. The front of the material box is the surface where the opening for accessing the wafer is located. If it is included, the image is set as the image to be detected. Figure 4 The image to be detected shown includes the content of the robot arm grabbing the wafer from the front of the material box. By detecting whether the image includes the content of the robot arm grabbing the wafer from the front of the material box, the input image to be detected is made a valid image, thereby improving the accuracy of the detection result.

[0055] Alternatively, the difference information between two video frames in the video information is compared, and the retained video frame is determined based on the difference information. It is detected whether the retained video frame includes the content of the robot arm grabbing the wafer from the front of the material box. If so, the video frame is used as the image to be detected.

[0056] The preset database includes the correspondence between different execution actions and different reference detection models. The first reference detection model corresponding to the execution action is searched from the preset database, and the first reference detection model is set as the target detection model.

[0057] It can be seen that in this example, if the execution action of the robot arm represents the action of the robot arm grabbing a wafer from a material box, the image to be detected is obtained from the video information, so that the extracted image to be detected is a valid image, thereby improving the detection efficiency.

[0058] In a possible example, determining the image to be detected and the target detection model based on the execution action of the robot arm includes: if the execution action of the robot arm represents the action of the robot arm placing the wafer into the material box, then setting a second image of the wafer placed in the material box after the robot arm stops moving as the image to be detected; searching a second reference detection model corresponding to the execution action from a preset database, the preset database including the correspondence between different execution actions and different reference detection models; and setting the second reference detection model to the target detection model.

[0059] In a specific example, if the execution action of the robot arm is to place the wafer into a material box, then directly detecting the state of the wafer placed in the material box can confirm whether there is any abnormality in the process of the wafer being placed by the robot arm. Therefore, when the execution action of the robot arm is to place the wafer into the material box, the second image of the wafer placed in the material box after the robot arm stops moving is set as the image to be detected. The second image includes the front of the material box, and the front of the material box represents the surface where the opening for accessing the wafer is located. The preset database includes the correspondence between different execution actions and different reference detection models. The second reference detection model corresponding to the execution action is searched from the preset database, and the second reference detection model is set as the target detection model.

[0060] It can be seen that in this example, if the execution action is to place the wafer into the material box, the second image when the wafer is placed in the material box after the robot arm stops moving is directly obtained, and the second image is used as the image to be detected to improve the detection efficiency, and the second reference detection model corresponding to the execution action is obtained, and the second reference detection model is set as the target detection model to improve the adaptability of the target detection model, thereby improving the accuracy of the detection result.

[0061] In a possible example, the corrected feature data group and the feature data group are input into the abnormality detection module, and if the abnormality detection module determines that there are abnormal pixels in the feature data group, a detection result of the wafer access abnormality is generated, including: inputting the corrected feature data group and the feature data group into the abnormality detection module, and calculating the difference between the pixel coordinates of the corrected feature data group and the pixel coordinates of the feature data group through the abnormality detection module; comparing the difference with a preset difference to obtain a comparison result; if the comparison result indicates that the difference is greater than the preset difference, a determination result that there are abnormal pixels in the feature data group is generated; and generating the detection result of the wafer access abnormality based on the determination result.

[0062] In a specific example, after obtaining a feature data group based on the first feature data set, the feature data group is input into the correction module to obtain a corrected feature data group. After obtaining the corrected feature data group, the corrected feature data group and the feature data group are input into the abnormality detection module, and whether there are abnormal pixels in the feature data group is determined according to the corrected feature data group and the feature data group. Specifically, the abnormality detection module can measure the difference between the corrected feature data group and the feature data group through a loss function. For example, the mean square error, mean absolute error, etc. can be selected as the loss function, which is not limited here. The difference between the two sets is calculated by the loss function, so as to determine the coordinate difference value of the corresponding pixel points between the feature data group and the corrected feature data group. That is, the pixel point whose coordinate difference value is greater than the preset difference value is set as an abnormal pixel point. If it is determined that there is an abnormal pixel point, it means that the wafer access is abnormal, and then the detection result of the wafer access abnormality is generated.

[0063] It can be seen that in this example, the difference in corresponding pixel points between the corrected feature data group and the feature data group is determined by the loss function of the anomaly detection module, thereby improving the accuracy of the determination result and further improving the accuracy of the generated detection result.

[0064] In one possible example, after obtaining the first feature data set, the feature data in the first feature data set is convolved based on a preset convolution kernel to obtain the first feature data feature set after the convolution processing, and then a feature data group is obtained based on the first feature data feature set after the convolution processing to further enhance the expressiveness and generalization capabilities of the target detection model.

[0065] In a possible example, before determining the image to be detected and the target detection model based on the execution action of the robot arm, the method also includes: obtaining a training model, a first training set and a second training set, the first training set including multiple images of the robot arm grabbing the wafer from the material box, and the second training set including multiple images of the wafer being located in the material box; inputting the first training set and the second training set into the training model respectively to obtain a reference detection model corresponding to the first training set and a reference detection model corresponding to the second training set; associating the reference detection model corresponding to the first training set with the execution action representing the robot arm grabbing the wafer from the material box to obtain a first association relationship; associating the reference detection model corresponding to the second training set with the execution action representing the robot arm placing the wafer into the material box to obtain a second association relationship; storing the first association relationship, the second association relationship, the reference detection model corresponding to the first training set, and the reference detection model corresponding to the second training set in a preset database.

[0066] In a specific example, before determining the image to be detected and the target detection model based on the execution action of the robot arm, a training model, a first training set, and a second training set are obtained. The first training set includes multiple images of the robot arm grabbing the wafer from the material box, and each image in the first training set shows the content of the robot arm grabbing the wafer from the front of the material box, and the front of the material box is the surface where the opening for accessing the wafer is located. The second training set includes multiple images of the wafer in the material box, and each image in the second training set includes the front of the material box. The images of the first training set and the second training set are all images obtained under the normal access state of the robot arm, avoiding model instability caused by abnormal images and improving training efficiency. And the number of images in the first training set and the second training set can be a small number such as 100 or 200, which further improves training efficiency. The first training set is input into the training model to obtain a reference detection model corresponding to the first training set, and the second training set is input into the training model to obtain a reference detection model corresponding to the second training set. The reference detection model corresponding to the first training set is associated with the execution action characterizing the robot arm grabbing the wafer from the material box to obtain a first association relationship; the reference detection model corresponding to the second training set is associated with the execution action characterizing the robot arm placing the wafer into the material box to obtain a second association relationship; the first association relationship, the second association relationship, the reference detection model corresponding to the first training set, and the reference detection model corresponding to the second training set are stored in a preset database to facilitate determining the corresponding target detection model according to the execution action of the robot arm.

[0067] It can be seen that in this example, different training sets are determined based on different execution actions, so that the training model is trained using different training sets to obtain detection models corresponding to different execution actions, thereby improving the accuracy and efficiency of detection.

[0068] In a possible example, after generating the detection result of the wafer access abnormality, the method further includes: generating alarm information based on the detection result; and sending the alarm information to a device connected to the server.

[0069] In a specific example, after the detection results are generated, an alarm message is generated based on the detection results; the alarm message is sent to a device connected to the server to prompt personnel to handle it in a timely manner, thereby improving intelligence and thereby improving the user experience.

[0070] The above mainly introduces the scheme of the embodiment of the present application from the perspective of the execution process on the method side. It is understandable that, in order to realize the above functions, the electronic device includes a hardware structure and / or software module corresponding to the execution of each function. It should be easily appreciated by those skilled in the art that, in combination with the units and algorithm steps of each example described in the embodiments provided herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present application.

[0071] The embodiment of the present application can divide the electronic device into functional units according to the above method example. For example, each functional unit can be divided according to each function, or two or more functions can be integrated into one processing unit. The above integrated unit can be implemented in the form of hardware or in the form of software functional units. It should be noted that the division of units in the embodiment of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation.

[0072] In the case of dividing each functional module according to each function, the following is combined Figure 5 The wafer access abnormality detection device in the embodiment of the present application is described in detail. Figure 5 A functional unit composition block diagram of a wafer access abnormality detection device provided in an embodiment of the present application, a wafer access abnormality detection device, applied to a server of a wafer test system, the wafer test system comprising the server, a wafer, a material box for storing the wafer, and a robotic arm communicatively connected to the server, the device comprising:

[0073] A determination unit 510 is used to determine an image to be detected and a target detection model based on an execution action of the robot arm, wherein the execution action represents an action of the robot arm grabbing the wafer from the material box or placing the wafer into the material box, and the target detection model includes a feature extraction module, a feature combination module, a correction module and an abnormality detection module;

[0074] A feature extraction unit 520 is used to input the image to be detected into the feature extraction module, and process the image to be detected by the feature extraction module to obtain a first feature data set, wherein the first feature data set includes different feature data, each of which includes image width, image length, and number of image channels, and the image width of each of the feature data is a preset image width, and the image length of each of the feature data is a preset image length;

[0075] A combining unit 530 is used to input the first feature data set into the feature combining module, and combine the feature data in the first feature data set through the feature combining module to obtain a feature data group, wherein the number of image channels of the feature data group is the sum of the number of image channels of each feature data in the first feature data set;

[0076] A correction processing unit 540, configured to input the feature data group into the correction module, perform data correction processing on the feature data group through the generative adversarial network of the correction module, and obtain a corrected feature data group;

[0077] The generating unit 550 is used to input the corrected feature data group and the feature data group into the abnormality detection module, and if the abnormality detection module determines that there are abnormal pixels in the feature data group, a detection result of the wafer access abnormality is generated.

[0078] In a possible example, the feature extraction module includes multiple feature extraction submodules and a multi-scale feature adjustment submodule, and the feature extraction unit 520 is further used to input the image to be detected into the multiple feature extraction submodules of the feature extraction module for feature extraction to obtain reference feature data of multiple different scales; and obtain a reference feature data set composed of the reference feature data of multiple different scales; and input the reference feature data set into the multi-scale feature adjustment submodule, and adjust the image width and image length of each reference feature data in the reference feature data set to the preset image width and the preset image length through the multi-scale feature adjustment submodule, so as to obtain the first feature data set composed of the adjusted reference feature data.

[0079] In a possible example, the determination unit 510 is also used to obtain video information of the robot arm performing the execution action if the execution action of the robot arm represents the action of the robot arm grabbing the wafer from the material box; and set the first image in the video information when the robot arm grabs the wafer in the material box as the image to be detected; and search for a first reference detection model corresponding to the execution action from a preset database, the preset database including the correspondence between different execution actions and different reference detection models; and set the first reference detection model to the target detection model.

[0080] In a possible example, the determination unit 510 is also used to set a second image of the wafer placed in the material box after the robot arm stops moving as the image to be detected if the execution action of the robot arm represents the action of the robot arm placing the wafer into the material box; and search a preset database for a second reference detection model corresponding to the execution action, the preset database including a correspondence between different execution actions and different reference detection models; and set the second reference detection model to the target detection model.

[0081] In a possible example, the generation unit 550 is also used to input the corrected feature data group and the feature data group into the abnormality detection module, and calculate the difference between the pixel coordinates of the corrected feature data group and the pixel coordinates of the feature data group through the abnormality detection module; and compare the difference with a preset difference to obtain a comparison result; and if the comparison result indicates that the difference is greater than the preset difference, generate a determination result that an abnormal pixel point exists in the feature data group; and generate a detection result of the wafer access abnormality based on the determination result.

[0082] In a possible example, the device also includes a training unit, which is used to obtain a training model, a first training set and a second training set, the first training set including multiple images of the robot arm grabbing the wafer from the material box, and the second training set including multiple images of the wafer being located in the material box; and inputting the first training set and the second training set into the training model respectively to obtain a reference detection model corresponding to the first training set and a reference detection model corresponding to the second training set; and associating the reference detection model corresponding to the first training set with an execution action representing the robot arm grabbing the wafer from the material box to obtain a first association relationship; and associating the reference detection model corresponding to the second training set with an execution action representing the robot arm placing the wafer into the material box to obtain a second association relationship; and storing the first association relationship, the second association relationship, the reference detection model corresponding to the first training set, and the reference detection model corresponding to the second training set in a preset database.

[0083] In a possible example, the apparatus further includes a prompt unit, which is configured to generate alarm information based on the detection result; and send the alarm information to a device connected to the server.

[0084] The above embodiments may be implemented in whole or in part by software, hardware, firmware or any other combination thereof. When implemented using software, the above embodiments may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from one website, computer, server or data center to another website, computer, server or data center by wired or wireless means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that contains one or more available media sets. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium. The semiconductor medium may be a solid-state hard disk.

[0085] An embodiment of the present application also provides a computer storage medium, wherein the computer storage medium stores a computer program for electronic data exchange, wherein the computer program enables a computer to execute part or all of the steps of any method described in the above method embodiments, and the above computer includes an electronic device.

[0086] An embodiment of the present application further provides a computer program product, which includes a computer program. The computer program can be operated to enable a computer to execute part or all of the steps of any method recorded in the above method embodiments.

[0087] The computer program product may be a software installation package, and the computer includes an electronic device.

[0088] It should be understood that in the various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0089] In the several embodiments provided in the present application, it should be understood that the disclosed methods, devices and systems can be implemented in other ways. For example, the device embodiments described above are merely schematic; for example, the division of the units is only a logical function division, and there may be other division methods in actual implementation; for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0090] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0091] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may be physically included separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0092] The above-mentioned integrated unit implemented in the form of a software functional unit can be stored in a computer-readable storage medium. The above-mentioned software functional unit is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform some steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store program codes.

[0093] Although the present invention is disclosed as above, the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions without departing from the spirit and scope of the present invention, and can make various changes and modifications, including the combination of the above-mentioned different functions and implementation steps, including software and hardware implementation methods, all of which are within the scope of protection of the present invention.

Claims

1. A method for detecting abnormal wafer access, characterized in that: A server applied to a wafer testing system, the wafer testing system comprising the server, a wafer, a material box for storing the wafer, and a mechanical arm communicatively connected to the server, the method comprising: Determine an image to be detected and a target detection model based on an execution action of the robot arm, wherein the execution action represents an action of the robot arm grabbing the wafer from the material box or placing the wafer into the material box, and the target detection model includes a feature extraction module, a feature combination module, a correction module and an abnormality detection module; If the execution action of the robot arm represents an action of the robot arm grabbing the wafer from the material box, then obtaining video information of the robot arm performing the execution action; The first image in the video information when the robot arm grabs the wafer in the material box is set as the image to be detected; Searching a first reference detection model corresponding to the execution action from a preset database, wherein the preset database includes a correspondence between different execution actions and different reference detection models; Setting the first reference detection model as the target detection model; If the execution action of the robot arm represents the action of the robot arm placing the wafer into the material box, a second image of the wafer placed in the material box after the robot arm stops moving is set as the image to be detected; searching the database for a second reference detection model corresponding to the execution action; Setting the second reference detection model as the target detection model; Input the image to be detected into the feature extraction module, and process the image to be detected by the feature extraction module to obtain a first feature data set, wherein the first feature data set includes different feature data, each of the feature data includes image width, image length and number of image channels, and the image width of each of the feature data is a preset image width, and the image length of each of the feature data is a preset image length; Inputting the first feature data set into the feature combination module, combining the feature data in the first feature data set through the feature combination module to obtain a feature data group, wherein the number of image channels of the feature data group is the sum of the number of image channels of each feature data in the first feature data set; Inputting the feature data group into the correction module, performing data correction processing on the feature data group through the generative adversarial network of the correction module, and obtaining a corrected feature data group; The corrected feature data group and the feature data group are input into the abnormality detection module. If the abnormality detection module determines that there are abnormal pixels in the feature data group, a detection result of the wafer access abnormality is generated.

2. The method according to claim 1, characterized in that The feature extraction module includes a plurality of feature extraction submodules and a multi-scale feature adjustment submodule. The image to be detected is input into the feature extraction module, and the image to be detected is processed by the feature extraction module to obtain a first feature data set, including: Inputting the image to be detected into the multiple feature extraction submodules of the feature extraction module to perform feature extraction, and obtaining reference feature data of multiple different scales; Acquire a reference feature data set consisting of the reference feature data of the plurality of different scales; The reference feature data set is input into the multi-scale feature adjustment submodule, and the image width and image length of each reference feature data in the reference feature data set are respectively adjusted to the preset image width and the preset image length by the multi-scale feature adjustment submodule to obtain the first feature data set consisting of the adjusted reference feature data.

3. The method according to claim 1 or 2, characterized in that: The corrected feature data group and the feature data group are input into the abnormality detection module, and if the abnormality detection module determines that there are abnormal pixels in the feature data group, a detection result of the wafer access abnormality is generated, including: Inputting the corrected feature data group and the feature data group into the anomaly detection module, and calculating the difference between the pixel coordinates of the corrected feature data group and the pixel coordinates of the feature data group by the anomaly detection module; Comparing the difference with a preset difference to obtain a comparison result; If the comparison result indicates that the difference is greater than the preset difference, generating a determination result that an abnormal pixel exists in the feature data group; A detection result of the wafer access abnormality is generated based on the determination result.

4. The method according to claim 1 or 2, characterized in that: Before determining the image to be detected and the target detection model based on the execution action of the robotic arm, the method further includes: Acquire a training model, a first training set, and a second training set, wherein the first training set includes a plurality of images of the robotic arm grabbing the wafer from the material box, and the second training set includes a plurality of images of the wafer being in the material box; Inputting the first training set and the second training set into the training model respectively to obtain a reference detection model corresponding to the first training set and a reference detection model corresponding to the second training set; Associating a reference detection model corresponding to the first training set with an execution action representing the robot arm grabbing the wafer from the material box to obtain a first association relationship; Associating a reference detection model corresponding to the second training set with an execution action representing the robot arm placing the wafer into the material box to obtain a second association relationship; The first association relationship, the second association relationship, the reference detection model corresponding to the first training set, and the reference detection model corresponding to the second training set are stored in a preset database.

5. The method according to claim 1 or 2, characterized in that: After generating the detection result of the wafer access abnormality, the method further includes: generating an alarm message based on the detection result; The alarm information is sent to a device connected to the server.

6. A wafer access abnormality detection device, characterized in that: A server applied to a wafer testing system, the wafer testing system comprising the server, a wafer, a material box for storing the wafer, and a mechanical arm communicatively connected to the server, the device comprising: a determination unit, configured to determine an image to be detected and a target detection model based on an execution action of the robot arm, wherein the execution action represents an action of the robot arm grabbing the wafer from the material box or placing the wafer into the material box, and the target detection model includes a feature extraction module, a feature combination module, a correction module, and an anomaly detection module; The determining unit is further configured to obtain video information of the robot arm performing the execution action if the execution action of the robot arm represents an action of the robot arm grabbing the wafer from the material box; The determination unit is further configured to set the first image in the video information when the robot arm grabs the wafer in the material box as the image to be detected; The determination unit is further configured to search a preset database for a first reference detection model corresponding to the execution action, wherein the preset database includes a correspondence between different execution actions and different reference detection models; The determining unit is further configured to set the first reference detection model as the target detection model; The determining unit is further configured to set a second image of the wafer being placed in the material box after the robot arm stops moving as the image to be detected if the execution action of the robot arm represents an action of the robot arm placing the wafer into the material box; The determining unit is further configured to search the database for a second reference detection model corresponding to the execution action; The determining unit is further configured to set the second reference detection model as the target detection model; A feature extraction unit, used for inputting the image to be detected into the feature extraction module, and processing the image to be detected by the feature extraction module to obtain a first feature data set, wherein the first feature data set includes different feature data, each of the feature data includes image width, image length and number of image channels, and the image width of each of the feature data is a preset image width, and the image length of each of the feature data is a preset image length; a combining unit, configured to input the first feature data set into the feature combining module, and combine the feature data in the first feature data set through the feature combining module to obtain a feature data group, wherein the number of image channels of the feature data group is the sum of the number of image channels of each feature data in the first feature data set; A correction processing unit, used for inputting the feature data group into the correction module, performing data correction processing on the feature data group through the generative adversarial network of the correction module, and obtaining a corrected feature data group; A generating unit is used to input the corrected feature data group and the feature data group into the abnormality detection module, and if the abnormality detection module determines that there are abnormal pixels in the feature data group, a detection result of the wafer access abnormality is generated.

7. A terminal device, characterized in that: include: At least one processor and a memory, the memory is used to store computer program code, the computer program code includes computer instructions, and when the at least one processor executes the computer instructions, the terminal device executes the method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program runs on a processor, the method according to any one of claims 1 to 5 is implemented.

Images