Deep learning-based wafer acupuncture needle mark identification method and identification device

Abstract

The invention provides a wafer needling needle mark identification method and identification device based on deep learning, and the method comprises the following steps: S1, obtaining a needling region image on a wafer; s2, preprocessing the acupuncture region image to highlight a needle mark in the acupuncture region image to obtain a needle mark standard image; s3, collecting a plurality of needle mark area images, and carrying out image enhancement on each collected needle mark area image to carry out model training to obtain a trained needle mark image model; and S4, importing the needle mark standard image in the step S2 into the needle mark image model, performing identification according to a preset model weight, and returning the detection category and confidence of the needle mark standard image. Compared with a traditional image processing needle mark detection mode, the method has the advantages that the processed needle mark standard image is imported into the deep learning algorithm of the needle mark image model for recognition, so that the robustness is higher.

Description

Technical Field

[0001] The present invention belongs to the technical field of semiconductor wafer needle mark recognition, and specifically relates to a wafer needle mark recognition method and recognition device based on deep learning. Background Art

[0002] Wafer Acceptance Test (WAT) is the process of testing the electrical characteristics of semiconductor devices and circuits at the wafer level. The WAT process uses a probe station, a device that allows the user to place electrical probes at specific locations on the wafer or chip. The probe station can measure various parameters of semiconductor devices and circuits, such as resistance, capacitance, current, voltage, frequency, and noise, by sticking needles at specific locations. The probe station can also perform functional tests on microcircuits and identify any defects or changes in the manufacturing process.

[0003] Deep learning, a branch of machine learning, uses artificial neural networks to learn from large amounts of data and perform complex tasks. Deep learning methods can automatically extract high-level features from images and handle a variety of changes and noise. The algorithm can detect the needle position information in combination with the needle mark image data after the probe station is inserted.

[0004] The existing semiconductor wafer needle mark recognition device recognizes the needle mark on the surface of the semiconductor wafer by a needle mark recognition method to obtain the area information and position information of the needle mark. Among them, the surface of the semiconductor wafer has a pad, and the needle mark is located on the pad. Generally, the surface of the semiconductor wafer is photographed to obtain a wafer image, and then the needle mark contour on the pad is extracted, and the position information and area information of the needle mark are extracted according to the needle mark contour.

[0005] However, the existing needle mark recognition methods have low robustness, so this situation needs to be solved. Summary of the invention

[0006] Therefore, the present invention provides a wafer needle mark recognition method and recognition device based on deep learning, and the main technical problem to be solved is: how to improve the robustness of needle mark detection.

[0007] In order to solve the above problems, the present invention provides a wafer needle mark recognition method based on deep learning, which comprises the following steps:

[0008] Step S1: Acquire an image of the needle insertion area on the wafer;

[0009] Step S2: preprocessing the acupuncture area image to highlight the needle marks in the acupuncture area image to obtain a standard needle mark image;

[0010] Step S3: collecting multiple needle mark area images, and performing image enhancement on each of the collected needle mark area images to perform model training, thereby obtaining a trained needle mark image model;

[0011] Step S4: Import the needle mark standard image in step S2 into the needle mark image model, perform recognition according to the preset model weight, and return the detection category and confidence of the needle mark standard image.

[0012] In some embodiments, obtaining an image of the needle insertion area on the wafer in step S1 specifically includes:

[0013] Step S10: acquiring a wafer image, and performing grayscale space conversion on the wafer image;

[0014] Step S11: performing illumination uniformity processing on the wafer image;

[0015] Step S12: extracting each pad area in the wafer image to obtain each needle insertion area image.

[0016] In some implementations, the illumination uniformity processing is performed on the wafer image in step S11, specifically:

[0017] Perform grayscale image opening and closing operations on the wafer image, perform differential operations on the wafer image before and after the opening and closing operations, and perform grayscale value equalization on the image after the differential operation.

[0018] In some implementations, in step S12, each pad region in the wafer image is extracted by dynamically binarizing the wafer image.

[0019] In some embodiments, the acupuncture area image is preprocessed in step S2, specifically:

[0020] The grayscale mean of each acupuncture area image is extracted, and the grayscale value of each location in the acupuncture area image is regionally scaled according to the ratio of the preset grayscale mean to the grayscale mean of the acupuncture area image.

[0021] In some embodiments, in step S3, image enhancement is performed on the collected images of each needle mark area, specifically:

[0022] Perform HSV adjustment and / or image scaling and / or image rotation on the collected images of each needle mark area.

[0023] The present invention also provides a wafer needle mark recognition device based on deep learning, which includes a needle area image acquisition module, an image preprocessing module, a model training module and an output module;

[0024] The needle puncture area image acquisition module is used to acquire the needle puncture area image on the wafer;

[0025] The image preprocessing module is used to preprocess the acupuncture area image to highlight the needle marks in the acupuncture area image and obtain a standard needle mark image;

[0026] The model training module is used to collect multiple needle mark area images, perform image enhancement on each collected needle mark area image, so as to perform model training and obtain a trained needle mark image model;

[0027] The output module is used to import the needle mark standard image into the needle mark image model, perform recognition according to the preset model weight, and return the detection category and confidence of the needle mark standard image.

[0028] In some embodiments, the acupuncture area image acquisition module includes a wafer image acquisition module, a light processing module, and an extraction module;

[0029] The wafer image acquisition module is used to acquire the wafer image and perform grayscale space conversion on the wafer image;

[0030] The illumination processing module is used to perform illumination uniformity processing on the wafer image;

[0031] The extraction module is used to extract each pad area in the wafer image to obtain each needle insertion area image.

[0032] The wafer needle mark recognition method and recognition device based on deep learning provided by the present invention have the following beneficial effects:

[0033] 1. Compared with the traditional image processing needle mark detection method, the present invention has higher robustness because the processed needle mark standard image is imported into the deep learning algorithm of the needle mark image model for identification.

[0034] 2. The present invention combines the preprocessing of the acupuncture area image with a deep learning algorithm. When the image is affected by different light intensities and background noise, the needle mark can still be detected and the correct needle mark position information can be obtained. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the implementation methods of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for the implementation methods or the description of the prior art. The drawings described below are only exemplary, and for ordinary technicians in this field, other implementation drawings can be derived from the provided drawings without creative work.

[0036] Figure 1 It is a flowchart of a wafer needle mark recognition method based on deep learning of the present invention;

[0037] Figure 2 is a schematic diagram of a wafer image;

[0038] Figure 3 It is a schematic diagram of the wafer image after the illumination uniformity processing;

[0039] Figure 4 is a schematic diagram of extracting the pad area within the wafer image;

[0040] Figure 5 is a schematic diagram of the pad image;

[0041] Figure 6 It is a schematic diagram of the pad image after grayscale value scaling;

[0042] Figure 7 This is the effect diagram of deep learning needle mark detection.

[0043] The accompanying drawings are marked as follows:

[0044] 1. Solder pad; 2. Needle mark. DETAILED DESCRIPTION

[0045] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. The following description of at least one exemplary embodiment is actually only illustrative and is by no means intended to limit the present invention and its application or use. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0046] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0047] It should be understood that the term "and / or" used in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0048] Unless otherwise specified, the relative arrangement of the components and steps described in these embodiments, the numerical expressions and numerical values ​​do not limit the scope of the present invention. At the same time, it should be understood that for ease of description, the structures, proportions, sizes, etc. shown in the drawings are only used to match the contents disclosed in the specification, so that people familiar with this technology can understand and read, and are not used to limit the limiting conditions that can be implemented in the present invention. Any modification of the structure, change of the proportional relationship or adjustment of the size should still fall within the scope of the technical content disclosed by the present invention without affecting the efficacy and purpose that can be achieved by the present invention. The techniques, methods and equipment known to ordinary technicians in the relevant field may not be discussed in detail, but in appropriate cases, the techniques, methods and equipment should be regarded as part of the authorization specification. In all examples shown and discussed here, any specific value should be interpreted as merely exemplary, not as a limitation. Therefore, other examples of exemplary embodiments may have different values. It should be noted that similar reference numerals and letters represent similar items in the following figures, so once an item is defined in one figure, it does not need to be further discussed in subsequent figures.

[0049] In the description of the present invention, it is necessary to understand that the directions or positional relationships indicated by directional words such as "front, back, up, down, left, right", "lateral, vertical, perpendicular, horizontal" and "top, bottom" are usually based on the directions or positional relationships shown in the drawings. They are only for the convenience of describing the present invention and simplifying the description. Unless otherwise specified, these directional words do not indicate or imply that the devices or elements referred to must have a specific direction or be constructed and operated in a specific direction. Therefore, they cannot be understood as limiting the scope of protection of the present invention. The directional words "inside and outside" refer to the inside and outside relative to the contours of each component itself.

[0050] For ease of description, spatially relative terms such as "above", "above", "on the upper surface of", "above", etc. may be used here to describe the spatial positional relationship between a device or feature and other devices or features as shown in the figure. It should be understood that spatially relative terms are intended to include different orientations of the device in use or operation in addition to the orientation described in the figure. For example, if the device in the accompanying drawings is inverted, the device described as "above other devices or structures" or "above other devices or structures" will be positioned as "below other devices or structures" or "below other devices or structures". Thus, the exemplary term "above" can include both "above" and "below". The device can also be positioned in other different ways (rotated 90 degrees or in other orientations), and the spatially relative descriptions used here are interpreted accordingly.

[0051] In addition, it should be noted that the use of terms such as "first" and "second" to limit components is only for the convenience of distinguishing the corresponding components. If not otherwise stated, the above terms have no special meaning and therefore cannot be understood as limiting the scope of protection of the present invention.

[0052] See also Figure 1-7 As shown, according to an embodiment of the present invention, a wafer needle mark recognition method based on deep learning is provided, which includes the following steps:

[0053] Step S1: Acquire the needle-piercing area image on the wafer. Specifically, the following steps S10 to S12 may be used to acquire the needle-piercing area image on the wafer:

[0054] Step S10: Acquire a wafer image and perform grayscale space conversion on the wafer image.

[0055] Specifically, the wafer image can be acquired by optical, ultrasonic or CT imaging. In this case, the wafer image is generally a color image. The color wafer image can be converted into a grayscale image by grayscale space conversion. Figure 2 A schematic diagram of a wafer image after grayscale space conversion is shown.

[0056] Step S11: performing illumination uniformity processing on the wafer image.

[0057] Specifically, the wafer image can be subjected to grayscale image opening and closing operations, the wafer image before and after the opening and closing operations can be differentially operated, and the grayscale value of the image after the differential operation can be balanced. In this way, after the wafer image is processed for illumination uniformity, the illumination uniformity of the wafer image can be improved.

[0058] Figure 2 A schematic diagram of a wafer image before light uniformity treatment is shown. From the figure, it can be seen that the right part of the wafer image is dark before light uniformity treatment. Among them, the number 1 is the pad, and the number 2 is the needle mark. Figure 3 A schematic diagram of a wafer image after light uniformity processing is shown. Figure 3 It can be seen that the overall illumination uniformity of the wafer image is improved after illumination uniformity processing, making the pad area in the wafer image more prominent and obvious, making it easier to extract the pad area from the wafer image.

[0059] Step S12: extracting each pad area in the wafer image to obtain each needle insertion area image.

[0060] Each pad in the wafer image is generally rectangular, and each pad area in the wafer image can be extracted by dynamically binarizing the wafer image to obtain a needle area image (i.e., pad image). Generally, there are multiple pad areas in a wafer image, so the obtained needle area images are also multiple. Figure 4 A schematic diagram of extracting a pad area is shown. Figure 5 A schematic diagram of a pad image is shown.

[0061] Step S2: Preprocessing the acupuncture area image to highlight the needle marks in the acupuncture area image and obtain a standard needle mark image. Specifically, the acupuncture area image can be preprocessed in the following manner:

[0062] The grayscale mean of each acupuncture area image is extracted, and the grayscale value of each location in the acupuncture area image is regionally scaled according to the ratio of the preset grayscale mean to the grayscale mean of the acupuncture area image to highlight the needle marks in the acupuncture area image, thereby obtaining a standard needle mark image.

[0063] Specifically, if the grayscale mean of a pad image is A and the preset grayscale mean is B, then the ratio of the preset grayscale mean to the grayscale mean of the pad image is B / A. Multiply the grayscale values ​​at each location in the pad image by the ratio B / A to obtain the scaled needle mark standard image.

[0064] Figure 5 A schematic diagram of a pad image before grayscale scaling is shown. Figure 6 A schematic diagram of a pad image after grayscale scaling is shown. Figure 5 and Figure 6 It can be seen that after the grayscale value is scaled, the needle mark outline in the pad image is more prominent and obvious, which facilitates the subsequent matching of the needle mark outline.

[0065] Step S3: collecting multiple needle mark area images, and performing image enhancement on each of the collected needle mark area images to perform model training to obtain a trained needle mark image model.

[0066] Specifically, the following methods can be used to enhance the collected images of the needle mark regions: perform HSV adjustment and / or image scaling and / or image rotation on the collected images of the needle mark regions.

[0067] Step S4: Import the needle mark standard image in step S2 into the needle mark image model, perform recognition according to the preset model weights, and return the detection category and confidence of the needle mark standard image.

[0068] Figure 7A schematic diagram of the detection category and confidence of a returned needle mark standard image is shown. From the image, it can be seen that the detection category of the needle mark standard image is Bpin, and the confidence is 0.85.

[0069] In the above example, compared with the traditional image processing needle mark detection method, the present invention is more robust because the processed needle mark standard image is imported into the deep learning algorithm of the needle mark image model for recognition.

[0070] In addition, the present invention can first extract the images of each pad area, i.e., the needle piercing area, in the wafer image by performing illumination uniformity processing on the entire wafer image in step S11; then perform preprocessing such as grayscale value scaling on the pad image in step S2, so as to highlight the needle marks in the needle piercing area image. In this way, the needle mark contour can be highlighted and displayed from the wafer image by step-by-step extraction, thereby reducing the influence of uneven illumination on the accuracy of needle mark detection.

[0071] In addition, the above-mentioned preprocessing methods such as illumination uniformity processing + grayscale value scaling of the pad image highlight the needle marks in the needle puncture area image. In addition to reducing the impact of uneven illumination on the accuracy of needle mark detection, it can also reduce the impact of background noise on the needle mark detection effect; and detecting needle marks through deep learning can further reduce the impact of background noise on the needle mark detection effect. In this way, the combination of illumination uniformity processing + grayscale value scaling of the pad image + deep learning can greatly reduce the impact of background noise in the needle puncture area on the detection effect.

[0072] The present invention also provides a wafer needle mark recognition device based on deep learning, which may include a needle area image acquisition module, an image preprocessing module, a model training module and an output module. The needle area image acquisition module is used to acquire the needle area image on the wafer. The image preprocessing module is used to preprocess the needle area image to highlight the needle marks in the needle area image and obtain a needle mark standard image. The model training module is used to collect multiple needle mark area images, perform image enhancement on each collected needle mark area image, perform model training, and obtain a trained needle mark image model. The output module is used to import the needle mark standard image into the needle mark image model, perform recognition according to the preset model weights, and return the detection category and confidence of the needle mark standard image.

[0073] Among them, in order to realize the function of the above-mentioned needle area image acquisition module, the needle area image acquisition module may include a wafer image acquisition module, a light processing module and an extraction module. The wafer image acquisition module is used to acquire the wafer image and perform grayscale space conversion on the wafer image. The light processing module is used to perform light uniformity processing on the wafer image. The extraction module is used to extract each pad area in the wafer image to obtain each needle area image.

[0074] In the above example, compared with the traditional image processing needle mark detection method, the present invention is more robust because the processed needle mark standard image is imported into the deep learning algorithm of the needle mark image model for recognition.

[0075] The present invention combines the preprocessing of the acupuncture area image with a deep learning algorithm, and can still detect the needle mark and obtain the correct needle mark position information when the image is affected by different light intensities and background noise.

[0076] It is easy for those skilled in the art to understand that, without conflict, the advantageous technical features of the above-mentioned methods can be freely combined and superimposed.

[0077] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principle of the present invention shall be included in the protection scope of the present invention. The above description is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and variations can be made without departing from the technical principles of the present invention. These improvements and variations should also be regarded as the protection scope of the present invention.

Claims

1. A wafer needle mark recognition method based on deep learning, characterized by: The following steps are involved: Step S1: Acquire an image of the needle insertion area on the wafer; Step S2: preprocessing the acupuncture area image to highlight the needle marks in the acupuncture area image to obtain a standard needle mark image; Step S3: collecting multiple needle mark area images, and performing image enhancement on each of the collected needle mark area images to perform model training, thereby obtaining a trained needle mark image model; Step S4: Import the needle mark standard image in step S2 into the needle mark image model, perform recognition according to the preset model weight, and return the detection category and confidence of the needle mark standard image.

2. The wafer needle mark recognition method based on deep learning according to claim 1, characterized in that: The step S1 of obtaining the needle-piercing area image on the wafer specifically includes: Step S10: acquiring a wafer image, and performing grayscale space conversion on the wafer image; Step S11: performing illumination uniformity processing on the wafer image; Step S12: extracting each pad area in the wafer image to obtain each needle insertion area image.

3. The wafer needle mark recognition method based on deep learning according to claim 2 is characterized in that: In step S11, the wafer image is processed for illumination uniformity, specifically: Perform grayscale image opening and closing operations on the wafer image, perform differential operations on the wafer image before and after the opening and closing operations, and perform grayscale value equalization on the image after the differential operation.

4. The wafer needle mark recognition method based on deep learning according to claim 2, characterized in that: In step S12, each pad region in the wafer image is extracted, specifically, each pad region in the wafer image is extracted by dynamically binarizing the wafer image.

5. The wafer needle mark recognition method based on deep learning according to any one of claims 1 to 3, characterized in that: In step S2, the acupuncture area image is preprocessed, specifically: The grayscale mean of each acupuncture area image is extracted, and the grayscale value of each location in the acupuncture area image is regionally scaled according to the ratio of the preset grayscale mean to the grayscale mean of the acupuncture area image.

6. The wafer needle mark recognition method based on deep learning according to any one of claims 1 to 3, characterized in that: In step S3, image enhancement is performed on the collected images of each needle mark area, specifically: Perform HSV adjustment and / or image scaling and / or image rotation on the collected images of each needle mark area.

7. A wafer needle mark recognition device based on deep learning, characterized in that: It includes a needle-piercing area image acquisition module, an image preprocessing module, a model training module and an output module; The needle puncture area image acquisition module is used to acquire the needle puncture area image on the wafer; The image preprocessing module is used to preprocess the acupuncture area image to highlight the needle marks in the acupuncture area image and obtain a standard needle mark image; The model training module is used to collect multiple needle mark area images, perform image enhancement on each collected needle mark area image, so as to perform model training and obtain a trained needle mark image model; The output module is used to import the needle mark standard image into the needle mark image model, perform recognition according to the preset model weight, and return the detection category and confidence of the needle mark standard image.

8. The wafer needle mark recognition device based on deep learning according to claim 7, characterized in that: The acupuncture area image acquisition module includes a wafer image acquisition module, a light processing module and an extraction module; The wafer image acquisition module is used to acquire the wafer image and perform grayscale space conversion on the wafer image; The illumination processing module is used to perform illumination uniformity processing on the wafer image; The extraction module is used to extract each pad area in the wafer image to obtain each needle insertion area image.

Images