A method, apparatus and device for dividing metal lines of an integrated circuit chip

Abstract

The embodiment of the specification discloses a kind of integrated circuit chip metal line segmentation method, device and equipment, scheme includes: obtaining the electron microscope image of integrated circuit chip to be handled;Based on the generative adversarial network model of pre-training, the preliminary metal line segmentation result image of the electron microscope image is obtained;The outline in the initial segmentation result image is extracted;For each outline in the initial segmentation result image, metal line outline in the each outline is identified, to obtain identification result;Based on each metal line outline obtained by the identification result, the final metal line segmentation result image of the electron microscope image is obtained.The metal line in the electron microscope image of integrated circuit chip is identified by the generative adversarial network model of pre-training, the preliminary metal line segmentation result image of electron microscope image is obtained, and further metal line identification is carried out to preliminary metal line segmentation result image, which can improve the accuracy of identifying the metal line of integrated circuit chip.

Description

Technical Field

[0001] This application relates to the field of integrated circuit chip technology, and in particular to a method, apparatus and equipment for dividing metal lines in integrated circuit chips. Background Technology

[0002] Chip reverse engineering employs a bottom-up approach, photographing the reference chip layer by layer, extracting the netlist from each layer, and then performing hierarchical analysis and organization of the netlist to ultimately derive the design concept of the reference chip. This technology is of great significance for chip industry professionals to learn advanced design concepts and improve their design experience.

[0003] In line analysis, accurately obtaining the location information of metal lines on a chip is a crucial step. However, chips often contain a massive number of complex metal lines. Furthermore, even a tiny inaccuracy can lead to incorrect integrated circuit design; therefore, this step demands high precision. Manually sketching these lines by chip analysis engineers is an incredibly time-consuming and labor-intensive task. To alleviate the workload of chip analysts, machine learning techniques have been applied to automate this step in the segmentation of metal lines on integrated circuit chips, such as methods based on support vector machines. However, machine learning techniques require manually designed features and have poor transferability.

[0004] Deep learning technology can automatically extract depth features from images, and numerous studies have demonstrated that it can achieve higher accuracy and robustness. Currently, methods exist for semantic segmentation of metal lines in electron microscope images of integrated circuit chips by constructing fully convolutional neural networks. However, semantic segmentation has poor ability to distinguish targets with unclear boundaries or blurry images. In practice, due to the physical processing of the chip and the electron microscope imaging process, we cannot guarantee that all metal lines in the image have clear boundaries, and the metal lines are often distributed in very close proximity, leading to performance limitations in semantic segmentation. Furthermore, since electron microscope images are high-resolution images, they need to be cropped before being used for training and inference of deep learning models. This results in images that may consist entirely of background or entirely of metal lines, making it difficult for the model to make accurate judgments about such images.

[0005] Therefore, it is necessary to provide a new method for dividing the metal lines of integrated circuit chips. Summary of the Invention

[0006] This specification provides a method, apparatus, and device for dividing metal lines in integrated circuit chips, in order to solve the problem of inaccurate dividing in existing integrated circuit chip metal line dividing methods.

[0007] To solve the above-mentioned technical problems, the embodiments in this specification are implemented as follows:

[0008] This specification provides an embodiment of a method for dividing metal lines in an integrated circuit chip, comprising:

[0009] Obtain electron microscope images of the integrated circuit chip to be processed;

[0010] Based on a pre-trained generative adversarial network model, a preliminary metal line segmentation result image is obtained from the electron microscope image;

[0011] For each contour in the initial segmentation result image, the metal line contour in each contour is identified to obtain the identification result;

[0012] Based on the recognition results, the contours of each metal line are obtained, and the final metal line segmentation result image of the electron microscope image is obtained.

[0013] Optionally, the preliminary metal line segmentation result image of the electron microscope image obtained based on the pre-trained generative adversarial network model specifically includes:

[0014] The electron microscope image is cropped according to a first preset cropping rule to obtain several sub-images;

[0015] The sub-images are input into a pre-trained generative adversarial network model to obtain the segmentation result sub-images corresponding to each sub-image;

[0016] The segmentation result sub-images corresponding to each sub-image are stitched together to obtain the preliminary metal wire segmentation result image.

[0017] Optionally, identifying the metal line contours within each contour in the initial segmentation result image specifically includes:

[0018] Extract each contour from the initial segmentation result image;

[0019] For any contour among the extracted contours, obtain the area of ​​the contour;

[0020] Determine whether the area of ​​the contour is greater than or equal to a first preset threshold;

[0021] If the area of ​​the contour is greater than or equal to a first preset threshold, then the contour is determined to be a metal wire contour.

[0022] or,

[0023] Obtain the roundness of the contour;

[0024] Determine whether the roundness of the contour is greater than or equal to a second preset threshold;

[0025] If the roundness of the contour is greater than or equal to a second preset threshold, then the contour is determined to be a metal wire contour.

[0026] Optionally, after the area of ​​the contour is greater than or equal to a first preset threshold, or the roundness of the contour is greater than or equal to a second preset threshold, the method further includes:

[0027] Determine whether the preliminary metal line segmentation result image contains a straight line passing through the contour;

[0028] If it exists, the contour is dilated and eroded to obtain a dilated contour image and an eroded contour image.

[0029] Subtracting the dilated contour image from the eroded contour image yields the contour ring;

[0030] Obtain the maximum and minimum pixel values ​​corresponding to the contour ring in the electron microscope image;

[0031] Determine whether the range between the maximum and minimum pixel values ​​is greater than or equal to a preset threshold;

[0032] Determining that the contour is a metal line contour specifically includes:

[0033] If the range value is greater than or equal to a preset threshold, then the contour is determined to be a metal line contour.

[0034] Optionally, the pre-trained generative adversarial network model is trained using the following method:

[0035] Obtain training sample images;

[0036] The training sample images are cropped according to the second preset cropping rule to obtain several training sub-images;

[0037] The training sub-images are preprocessed to obtain the processed training sub-images;

[0038] The initial generative adversarial network model is trained using the processed training sub-images to obtain the trained generative adversarial network model.

[0039] Optionally, the image preprocessing of the plurality of training sub-images specifically includes:

[0040] The labeled images of each electron microscope image in the plurality of training sub-images are subjected to narrow slit preprocessing, which expands the spacing of metal lines in each image that are less than or equal to n pixels to m pixels.

[0041] Optionally, the narrow slit pretreatment specifically includes:

[0042] For any training sub-image among the plurality of training sub-images, obtain the metal lines in the training sub-image that are separated by a distance of less than or equal to i pixels; i∈[1,n];

[0043] The metal lines in the training sub-image that are spaced less than or equal to i pixels apart are dilated with an m×m kernel to obtain the processed training sub-image.

[0044] Optionally, obtaining metal lines in the training sub-image that are spaced apart by a distance of less than or equal to n pixels specifically includes: establishing an auxiliary image, a slit image, and a midline image corresponding to the training sub-image; the auxiliary image is an image obtained by setting pixels with a value of 0 in the training sub-image to -1; the slit image and the midline image are images with the same size as the training sub-image and all pixel values ​​are 0.

[0045] The auxiliary image is convolved with the row vector to obtain the first convolution result image; the length of the row vector is i+2, the first and last elements are both 1, and the values ​​of the remaining elements are all -1; the variable i∈[1,n];

[0046] Set the pixel value at the position corresponding to the pixel with pixel value equal to i+2 in the first convolution result image P on the midline image of the interval to 1, and obtain the adjusted first midline image of the interval.

[0047] The auxiliary image is convolved with the column vector to obtain the second convolution result image; the length of the column vector is i+2, the first and last elements are both 1, and the values ​​of the remaining elements are all -1.

[0048] The pixel value at the position corresponding to the pixel with pixel value equal to i+2 in the second convolution result image on the adjusted first interval midline image is set to 1 to obtain the adjusted second interval midline image; the second interval midline image is used to represent the midline of the interval of metal lines in the training sub-image that are separated by a distance of less than or equal to i pixels.

[0049] Optionally, the step of dilating metal lines in the training sub-image that are spaced less than or equal to i pixels apart using an m×m kernel to obtain the processed training sub-image specifically includes:

[0050] The adjusted second interval midline image is dilated with an m×m kernel to obtain the dilated interval midline image corresponding to the training sub-image for variable i.

[0051] With a step size of i plus 1, we obtain the dilated interval midline image corresponding to each value of i in the range [1, n].

[0052] The dilated midline images corresponding to each value are superimposed on the slit image to obtain the final slit image;

[0053] Set the pixel value at the position corresponding to the pixel with a pixel value greater than 0 in the final slit image in the training sub-image to 0 to obtain the slit preprocessed training sub-image corresponding to the training sub-image.

[0054] Optionally, the image preprocessing of the plurality of training sub-images further includes:

[0055] Data augmentation processing is performed on the training sub-images after the narrow slit preprocessing;

[0056] The data augmentation processing includes at least one of the following methods: random scaling; random cropping; random brightness, contrast and saturation adjustment; random addition of Gaussian noise; random horizontal and vertical flipping; and random angle rotation with a rotation angle of [40°, 50°].

[0057] Optionally, the initial generative adversarial network (GAN) model includes a generator and a discriminator; the step of training the initial GAN ​​model using the processed training sub-images to obtain the trained GAN model specifically includes:

[0058] The processed training sub-images are used as input to the initial generative adversarial network model, and iterative training is performed for a preset number of rounds to obtain the trained generative adversarial network model.

[0059] During iterative training, every two training rounds, the performance of the generative adversarial network model trained in the last round of those two rounds is evaluated using a validation set. If the evaluation results indicate that the performance of the generative adversarial network model trained in the last round of those two rounds is better than the performance of the generative adversarial network models trained in the previous rounds, then the model parameters of the generative adversarial network model trained in the last round are saved. Based on the saved model parameters, subsequent iterative training is performed.

[0060] Optionally, during iterative training: train the generator twice and the discriminator once.

[0061] Optionally, the generator includes an encoder and a decoder; the encoder includes a residual attention module and an ASPP model connected to the residual attention module;

[0062] The decoder concatenates the first feature map output by the intermediate layer of the residual attention module and the second feature map output by the ASPP module to obtain the concatenated feature map.

[0063] Decoding is completed based on the stitched feature map.

[0064] This specification provides an embodiment of an integrated circuit chip metal line splitting device, comprising:

[0065] The image acquisition module is used to acquire electron microscope images of the integrated circuit chip to be processed;

[0066] The preliminary results module is used to obtain preliminary metal line segmentation results images of the electron microscope images based on a pre-trained generative adversarial network model.

[0067] A metal line recognition module is used to identify metal line contours in each contour of the initial segmentation result image and obtain recognition results.

[0068] The result determination module is used to obtain the final metal line segmentation result image of the electron microscope image based on the metal line contours obtained from the recognition results.

[0069] This specification provides an embodiment of an integrated circuit chip metal wire splitting device, comprising:

[0070] At least one processor; and,

[0071] A memory communicatively connected to the at least one processor; wherein,

[0072] The memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enable the at least one processor to:

[0073] Obtain electron microscope images of the integrated circuit chip to be processed;

[0074] Based on a pre-trained generative adversarial network model, a preliminary metal line segmentation result image is obtained from the electron microscope image;

[0075] For each contour in the initial segmentation result image, the metal line contour in each contour is identified to obtain the identification result;

[0076] Based on the recognition results, the contours of each metal line are obtained, and the final metal line segmentation result image of the electron microscope image is obtained.

[0077] This specification provides an embodiment of a computer-readable medium storing computer-readable instructions that can be executed by a processor to implement a method for dividing metal lines in an integrated circuit chip.

[0078] One embodiment of this specification achieves the following beneficial effects: by using a pre-trained generative adversarial network model to identify metal lines in electron microscope images of integrated circuit chips, a preliminary metal line segmentation result image of the electron microscope image is obtained, and further metal line identification is performed on the preliminary metal line segmentation result image, which can improve the accuracy of identifying metal lines of integrated circuit chips. Attached Figure Description

[0079] To more clearly illustrate the technical solutions in the embodiments or prior art of this specification, the drawings used in the description of the embodiments or prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0080] Figure 1 This is a flowchart illustrating a method for dividing metal lines in an integrated circuit chip, as provided in an embodiment of this specification.

[0081] Figure 2 A schematic diagram of a post-processing target screening process provided in the embodiments of this specification;

[0082] Figure 3 A schematic diagram of the network structure of a generator for a generative adversarial network model provided in an embodiment of this specification;

[0083] Figure 4 A schematic diagram of the network structure of a discriminator for a generative adversarial network model provided in the embodiments of this specification;

[0084] Figure 5 This specification provides an example image of a metal line segmentation result for an embodiment.

[0085] Figure 6 A flowchart illustrating a method for dividing metal lines in an integrated circuit chip, as provided in the embodiments of this specification.

[0086] Figure 7 A schematic diagram of a device for dividing metal lines of an integrated circuit chip, provided as an embodiment of this specification;

[0087] Figure 8 This is a schematic diagram of a device for dividing metal lines in an integrated circuit chip, provided as an embodiment of this specification. Detailed Implementation

[0088] To make the objectives, technical solutions, and advantages of one or more embodiments of this specification clearer, the technical solutions of one or more embodiments of this specification will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this specification, and not all of them. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort are within the protection scope of one or more embodiments of this specification.

[0089] The technical solutions provided in the various embodiments of this specification are described in detail below with reference to the accompanying drawings.

[0090] To address the shortcomings of existing technologies, this solution provides the following embodiments:

[0091] Figure 1 This is a flowchart illustrating a method for dividing metal lines in an integrated circuit chip, as provided in an embodiment of this specification. From a programming perspective, the entity executing the process can be a program hosted on an application server or an application client.

[0092] like Figure 1 As shown, the process may include the following steps:

[0093] Step 102: Obtain electron microscope images of the integrated circuit chip to be processed.

[0094] In practical applications, electron microscopes can be used to magnify and photograph integrated circuit chips to obtain electron microscope images.

[0095] Step 104: Based on the pre-trained generative adversarial network model, obtain the preliminary metal line segmentation result image of the electron microscope image.

[0096] The pre-trained generative adversarial network model can be obtained by training the initial generative adversarial network model based on labeled training samples. The pre-trained generative adversarial network model can be used to identify metal lines in electron microscope images, thus achieving preliminary identification of metal lines in electron microscope images.

[0097] Step 106: For each contour in the initial segmentation result image, identify the metal line contour in each contour to obtain the identification result.

[0098] Step 108: Based on the metal line contours obtained from the recognition results, obtain the final metal line segmentation result image of the electron microscope image.

[0099] In the embodiments of this specification, after obtaining the initial segmentation result image identified by the network model, post-processing operations can be performed on the image to further remove interference. Contour recognition tools or algorithms can be used to extract all contours in the initial segmentation result image; for example, OpenCV's findContours can be used to extract all contours in the image. The identified contours may contain misjudgments caused by interference introduced by the chip's physical processing and imaging processes, or misjudgments caused by cropping resulting in an image consisting entirely of background or entirely of metal lines. Through further recognition, metal lines in the image can be accurately filtered out.

[0100] It should be understood that the order of some steps in the methods described in one or more embodiments of this specification may be interchanged according to actual needs, or some steps may be omitted or deleted.

[0101] Figure 1 The method described above uses a pre-trained generative adversarial network model to identify metal lines in electron microscope images of integrated circuit chips, obtaining preliminary metal line segmentation results. Further metal line identification is then performed on the preliminary metal line segmentation results, which can improve the accuracy of identifying metal lines in integrated circuit chips.

[0102] based on Figure 1 In addition to the method described herein, this specification also provides some specific implementation schemes of the method, which will be described below.

[0103] In practical applications, the electron microscope images of the integrated circuit chip to be processed, acquired by electron microscope equipment, may be quite large. Due to limitations in the computing power of the hardware, the electron microscope image is usually cropped into several smaller images before recognition. Optionally, the preliminary metal line segmentation result image obtained from the electron microscope image based on a pre-trained generative adversarial network model, as described in the embodiments of this specification, may specifically include:

[0104] The electron microscope image is cropped according to a first preset cropping rule to obtain several sub-images;

[0105] The sub-images are input into a pre-trained generative adversarial network model to obtain the segmentation result sub-images corresponding to each sub-image;

[0106] The segmentation result sub-images corresponding to each sub-image are stitched together to obtain the preliminary metal wire segmentation result image.

[0107] In practical applications, the first preset cropping rule can be set according to actual needs. As one implementation method, a fixed-size window s×s can be set, and the cropping can be performed by sliding along the rows and columns starting from the upper left corner of the high-resolution electron microscope image. First, move s pixels along the column direction each time, and then move s pixels along the row direction each time, thereby traversing the entire electron microscope image to complete the cropping.

[0108] The cropped sub-images are input into a pre-trained generative adversarial network (GAN) model for recognition, yielding the recognition result for each sub-image, i.e., the segmentation result sub-image. These segmentation result sub-images are then stitched together according to a stitching rule corresponding to the first preset cropping rule, resulting in a preliminary metal wire segmentation result image corresponding to the original electron microscope image. This stitching rule ensures that the segmentation result sub-images are stitched together according to their corresponding positions in the original electron microscope image, making the stitched result image correspond to the original electron microscope image. The positions of each segmentation result sub-image in the preliminary metal wire segmentation result image are consistent with the positions of the individual sub-images in the original electron microscope image.

[0109] In the embodiments of this specification, the determination of whether a contour is a metal line contour can be made based on parameters such as the area and roundness of the contour. Optionally, in the embodiments of this specification, the identification of metal line contours in each contour of the initial segmentation result image to obtain the identification result may specifically include:

[0110] Extract each contour from the initial segmentation result image;

[0111] For any contour among the extracted contours, obtain the area of ​​the contour;

[0112] Determine whether the area of ​​the contour is greater than or equal to a first preset threshold;

[0113] If the area of ​​the contour is greater than or equal to a first preset threshold, then the contour is determined to be a metal wire contour.

[0114] or,

[0115] Obtain the roundness of the contour;

[0116] Determine whether the roundness of the contour is greater than or equal to a second preset threshold; if the roundness of the contour is greater than or equal to the second preset threshold, then determine that the contour is a metal wire contour.

[0117] In order to remove small non-metallic outlines from the preliminary metal line segmentation result image while retaining isolated small metal lines, the area and roundness of the outline can be calculated. Outlines that do not meet the conditions for area and roundness are removed, while outlines that meet any preset conditions for area and roundness in the preliminary metal line segmentation result image are retained.

[0118] To further improve the accuracy of metal wire segmentation, in the embodiments of this specification, if the area of ​​the contour is greater than or equal to a first preset threshold, or the roundness of the contour is greater than or equal to a second preset threshold, the following may also be included:

[0119] Determine whether the preliminary metal line segmentation result image contains a straight line passing through the contour;

[0120] If it exists, the contour is dilated and eroded to obtain a dilated contour image and an eroded contour image.

[0121] Subtracting the dilated contour image from the eroded contour image yields the contour ring;

[0122] Obtain the maximum and minimum pixel values ​​corresponding to the contour ring in the electron microscope image;

[0123] Determine whether the range between the maximum and minimum pixel values ​​is greater than or equal to a preset threshold;

[0124] Determining that the contour is a metal line contour specifically includes:

[0125] If the range value is greater than or equal to a preset threshold, then the contour is determined to be a metal line contour.

[0126] In the embodiments of this specification, contours whose area or roundness is greater than or equal to a threshold can be further filtered. Specifically, Hough transform can be used to find straight lines on the preliminary metal wire segmentation result image of the contour. If there are straight lines passing through the contour in the preliminary metal wire segmentation result image, the contour can be dilated and eroded to obtain a dilated contour image and an eroded contour image, respectively. The two images are subtracted to obtain a contour loop. The maximum and minimum gray values ​​within the contour loop are obtained in the original electron microscope image, and the gray range is calculated. If the range is greater than or equal to the threshold, the contour is determined to be a metal wire contour.

[0127] To more clearly illustrate the process of post-processing target selection in the preliminary metal line segmentation result image obtained from the network model in the embodiments of this specification, Figure 2 This is a schematic diagram of a post-processing target screening process provided in an embodiment of this specification. Figure 2 As shown, the preliminary metal line segmentation result image of the electron microscope image obtained based on the pre-trained generative adversarial network model can be further post-processed, which may include:

[0128] Step 202: Extract all contours from the initial segmentation result image to obtain the initial contour set C. In practical applications, the initial segmentation result image can be a binary image, and OpenCV's findContours function can be used to find all contours in the segmentation result binary image.

[0129] Step 204: Take a contour c from the contour set C.

[0130] Step 206: To remove non-metallic small outlines while retaining isolated small metallic lines, calculate the area and roundness of outline c.

[0131] Step 208: Determine whether the area is greater than or equal to the first preset threshold Ta, and whether the roundness is greater than or equal to the second preset threshold Tr.

[0132] Step 210: If the area is greater than or equal to the first preset threshold Ta, or the roundness is greater than or equal to the second preset threshold Tr, then use Hough transform to find a straight line on the binary image of contour c. Other algorithms can also be used to find the straight line; no specific limitation is made here. If the area is less than the first preset threshold Ta and the roundness is less than the second preset threshold Tr, then contour c is determined to be a contour to be removed, and we can return to step 204 to select another contour from the contour set C for processing.

[0133] Step 212: Determine whether there is a straight line passing through contour c in the preliminary metal wire segmentation result image.

[0134] Step 214: If the preliminary metal wire segmentation result image contains a straight line passing through contour c, then dilate and erode contour c to obtain a dilated contour image and an eroded contour image, respectively. Subtract the two images to obtain the contour loop. If the preliminary metal wire segmentation result image does not contain a straight line passing through contour c, then contour c is determined to be a contour to be removed. The process can return to step 204 to select another contour from the contour set C for further processing.

[0135] In practical applications, the preliminary metal wire segmentation result image containing contour c can be a binary image. The binary image of contour c in the preliminary metal wire segmentation result image can be dilated and eroded to obtain a dilated contour binary image and an eroded contour binary image, respectively. Subtracting the two binary images yields the contour loop. Specifically, dilating and eroding the binary image of contour c can be understood as dilating and eroding the region containing contour c in the preliminary metal wire segmentation result image, while other contours in the preliminary metal wire segmentation result image remain unprocessed.

[0136] Step 216: Obtain the maximum and minimum pixel values ​​within the region corresponding to the contour ring in the electron microscope image, and calculate the range between the maximum and minimum pixel values.

[0137] Step 218: Determine whether the range value is greater than or equal to the preset threshold Td.

[0138] Step 220: If the range value is greater than or equal to the preset threshold Td, then the contour c can be determined to be a metal line contour. A result contour set T can be established to store the filtered contours, and contour c is placed into the result contour set T. As one implementation, the thresholds for contour area and roundness are set to 200 and 0.9 respectively, and the threshold for the grayscale range value is set to 50. These can also be set according to actual needs; no limitation is made here.

[0139] If the range value is less than the preset threshold Td, then the contour c can be considered as a contour to be removed, and the process can return to step 204 to select another contour from the contour set C for processing.

[0140] Step 222: Perform the above steps to filter each contour in the contour set C, and remove the filtered contours from the contour set C; determine whether the contour set C is empty.

[0141] Step 224: If the contour set C is empty, it means that all contours in the initial segmentation result image have been filtered, and the target filtering process can be completed. If the contour set C is not empty, then step 204 is executed to select other contours from the contour set C for filtering.

[0142] In the embodiments of this specification, the cropped sliced ​​images are input into the generator of the generative adversarial network model to generate binary images of metal line segmentation results corresponding to each sliced ​​image. These images are then stitched together to obtain the initial segmentation result image of the original electron microscope image. Post-processing is performed on the initial segmentation result image to filter targets. The contour images of all contours in the filtered result contour set are then filled and drawn to obtain the final binary image of the metal line segmentation result.

[0143] In the embodiments of this specification, a generative adversarial network (GAN) model can be used to identify metal lines in electron microscope images. The pre-trained GAN model can be trained using the following method:

[0144] Obtain training sample images;

[0145] The training sample images are cropped according to the second preset cropping rule to obtain several training sub-images;

[0146] The training sub-images are preprocessed to obtain the processed training sub-images; the labeled images are images in which metal line regions and non-metal line regions are clearly labeled.

[0147] The initial generative adversarial network model is trained using the processed training sub-images to obtain the trained generative adversarial network model.

[0148] The training sample images can be labeled images. As one implementation method, the training sample images in this embodiment can be obtained based on expert experience. For example, electron microscopes can be used to magnify and photograph 12 integrated circuit chips, obtaining several electron microscope images with a resolution of 8192×8192. Experienced chip analysis engineers can then label the metal lines, marking the metal line areas as 1 and other background areas as 0, thus obtaining the corresponding true label images. First, the images are divided into training, validation, and test sets in a 7:2:1 ratio, based on the type of integrated circuit chip. Then, these high-resolution electron microscope images need to be cropped. In this embodiment, a sliding window can be used for cropping to obtain sliced ​​images. Further, a sliding window of size 2048×2048 is selected, and the cropping is set to start from the upper left corner of the original high-resolution image and proceed along the rows and columns of the image. The horizontal and vertical movement steps of the sliding window are set to be equal to the window size, meaning there is no overlap between the cropped sliced ​​images. Furthermore, we can first crop the image by moving the sliding window 2048 pixels at a time along the column direction of the original high-resolution image, and then move it down 2048 pixels along the row direction, thus traversing the entire original high-resolution image. Similarly, we crop the real label image to obtain several 2048×2048 resolution slice electron microscope images and their corresponding real label images.

[0149] Understandably, in practical applications, the sample type, the size of the electron microscope image, and the specific dimensions for cropping the image can be determined according to actual needs, and no specific limitations are made here.

[0150] Considering the potential for target adhesion issues in network model downsampling due to the close proximity of some metal lines in integrated circuit chips during practical applications, preprocessing of the training images can be performed, and the processed images can be used for training. Optionally, the image preprocessing of the plurality of training sub-images described in the embodiments of this specification may specifically include:

[0151] The labeled images of each electron microscope image in the plurality of training sub-images are subjected to narrow slit preprocessing, which expands the spacing of metal lines in each image that are less than or equal to n pixels to m pixels.

[0152] In the embodiments of this specification, the real label image of the training image is a binary image. Narrow slit preprocessing can be used to expand the spacing of metal lines with a distance of less than or equal to n pixels to m pixels. Specifically, it can include the following steps:

[0153] 1) Copy the training sub-image that needs to be preprocessed into the image to be processed. Set the pixels with a value of 0 in the image to be processed to -1 to obtain the auxiliary image B. Create a new slit image M and a midline image N with the same size as the image to be processed and all pixel values ​​of 0. Initialize the variable i to 1.

[0154] 2) Create a new row vector H of length i+2, with the first and last elements of H having a value of 1 and the remaining elements having a value of -1. Perform a convolution operation between the auxiliary image B and the row vector H to obtain the first convolution result image P; set the pixel values ​​at the positions on the interval midline image N corresponding to the pixels in the first convolution result image P with a pixel value equal to i+2 to 1, to obtain the adjusted first interval midline image N1.

[0155] 3) Create a new column vector L of length i+2, with the first and last elements of L having a value of 1 and the remaining elements having a value of -1. Perform a convolution operation between the auxiliary image B and the column vector L to obtain the second convolution result image Q; set the pixel values ​​at the positions corresponding to the pixels in the second convolution result image Q with a pixel value equal to i+2 on the adjusted interval midline image N1 to 1 to obtain the adjusted second interval midline image N2; the adjusted interval midline image N2 is used to represent the contour lines in the training sub-image that are spaced less than or equal to i pixels apart.

[0156] 4) Perform a dilation operation with a kernel size of m×m on the adjusted second interval midline image N2 to obtain the dilated interval midline image N3.

[0157] 5) Add the dilated midline image N3 to the slit image M, then reset the dilated midline image N3 to all zeros, thus restoring the original midline image N with all pixel values ​​set to zero. Adding the dilated midline image N3 to the slit image represents adding the pixel values ​​at corresponding positions in the two images.

[0158] 6) Determine if i is equal to n. If they are equal, proceed to the next step; otherwise, increment i by 1 and return to step 2). Through steps 1) to 6), obtain the dilated midline image corresponding to each value of i in the range [1, n], and superimpose each dilated midline image N3 onto the slit image M to obtain the final slit image Mn.

[0159] As one implementation method, n can be set to 5 and m to 9. In practical applications, specific data can also be set according to actual needs; no specific limitations are made here.

[0160] 7) Set the pixel values ​​at the positions corresponding to the pixels with pixel values ​​greater than 0 in the final slit image Mn on the image to be processed to 0, and obtain the result image of slit preprocessing, that is, the slit preprocessed training sub-image corresponding to this training sub-image.

[0161] To effectively prevent overfitting of the network model and improve its robustness, the image preprocessing of the plurality of training sub-images described in the embodiments of this specification may further include:

[0162] Data augmentation processing is performed on the training sub-images after narrow slit preprocessing;

[0163] The data augmentation processing includes at least one of the following methods: random scaling; random cropping; random brightness, contrast and saturation adjustment; random addition of Gaussian noise; random horizontal and vertical flipping; and random angle rotation with a rotation angle of [40°, 50°].

[0164] In the embodiments of this specification, during model training, each batch of images fed into the model for training undergoes data augmentation before being sent to the model for training. The data augmentation operations specifically include: random scaling; random cropping; random adjustment of brightness, contrast, and saturation; random addition of Gaussian noise; random horizontal and vertical flipping; and random angle rotation [40°, 50°]. Different data augmentation methods can be applied to the training samples according to a certain proportion.

[0165] In one implementation, the initial input image size is 2048×2048, which can be scaled with a probability of 0.5 within a random ratio range of [0.5, 1.0). Then, a cropped image of size 1024×1024 is randomly extracted from the scaled image. Subsequently, the cropped image is adjusted for brightness, contrast, and saturation with a probability of 0.7; Gaussian noise is added with a probability of 0.3; horizontal and vertical rotation is performed with a probability of 0.5; and finally, a random angle rotation within the range of [40°, 50°] is performed with a probability of 0.1, filling unimage areas with 0. Here, probability represents the likelihood of performing the corresponding image processing; for example, adding Gaussian noise with a probability of 0.3 means there is a 30% probability that the image will undergo Gaussian noise processing.

[0166] Understandably, in practical applications, specific probability or ratio values ​​can be set according to actual needs, and no specific restrictions are made here.

[0167] In the embodiments of this specification, the training sample data can also be divided into a training set, a validation set, and a test set according to a preset ratio, wherein the data in the training set can be used to train the model.

[0168] The initial generative adversarial network model in the embodiments of this specification may include a generator and a discriminator; wherein, the generator is used to generate a binary image of the metal line segmentation result of the electron microscope image, and the discriminator is used to distinguish the difference between the generated segmentation result and the real label image. The two modules compete against each other, and finally a generator that can generate a segmentation result image that is highly similar to the real label image is obtained through training.

[0169] The above-mentioned use of the processed training sub-images to train the initial generative adversarial network model to obtain the trained generative adversarial network model may specifically include:

[0170] The processed training sub-images are used as input to the initial generative adversarial network model, and iterative training is performed for a preset number of rounds to obtain the trained generative adversarial network model.

[0171] During iterative training, the performance of the generative adversarial network model trained in the last round is evaluated every two training rounds using a validation set. If the evaluation results indicate that the performance of the generative adversarial network model trained in the last round is better than that of the generative adversarial network models trained in the previous rounds, the model parameters of the generative adversarial network model trained in the last round are saved. Subsequent iterative training is then performed based on the saved model parameters.

[0172] Throughout the iterative training process, the generator and discriminator compete against each other. In each training round, the discriminator can be trained once after every two generator training rounds, and finally a generator that can generate images that are highly similar to the real metal line segmentation results is obtained.

[0173] The generator can be a segmentation model with an encoder-decoder structure; the encoder includes a residual attention module and an ASPP model connected to the residual attention module.

[0174] The decoder concatenates the first feature map output by the intermediate layer of the residual attention module and the second feature map output by the ASPP module to obtain a concatenated feature map; decoding is then performed based on the concatenated feature map.

[0175] Figure 3 This is a schematic diagram of the network structure of a generator for a generative adversarial network model provided in an embodiment of this specification. For example... Figure 3 As shown:

[0176] Encoder: The encoder's main body consists of an attention-based ResNet50 model, where an attention module is added to each residual module of the ResNet50 model. The ASPP module is then connected after the attention-based ResNet50. The ASPP module mainly consists of a regular convolutional branch, a pooling convolutional branch, and three dilated convolutional branches. The output feature maps of each branch are concatenated along the channel direction and fed into a 1×1 convolution. Finally, a BN layer, a ReLU activation layer, and a Dropout layer are connected to obtain the output of the ASPP module. The attention module used in the attention-based ResNet50 is the Squeeze-and-Excitation attention module. The number of output channels for each branch convolution in the ASPP module is set to 256, and the scaling factors for the three dilated convolutions are set to 6, 12, and 18, respectively. The final 1×1 convolution also has 256 output channels, and the coefficient of the Dropout layer is set to 0.1.

[0177] Decoder: The low-level feature map output from the intermediate layer of the attention ResNet50 model is extracted, reduced in dimensionality by a 1×1 convolution, and then concatenated along the channel direction with the feature map obtained by upsampling the high-level feature map output from the ASPP module. The concatenated feature map is then subjected to a 3×3 convolution and upsampling to complete the decoding. Specifically, the low-level feature map taken from the intermediate layer of the attention ResNet50 model in the decoder comes from the output of its first residual module, i.e., a feature map with 256 channels, which is then reduced to 48 channels by a 1×1 convolution. The high-level feature map output from the ASPP module is upsampled by a factor of 4 and concatenated along the channel direction with the aforementioned dimensionality-reduced feature map. The concatenated feature map is fed into a 3×3 convolution with 256 output channels, and after BN and ReLU activation, it is upsampled again to the original image size. Finally, it is fed into a 1×1 convolution with the number of output channels equal to the number of classes to obtain the metal line segmentation result image of the encoder input image. Since we need to generate a binary image of the segmentation result, the number of classes in this embodiment can be 1.

[0178] In convolutional networks, the dimensionality of feature images increases with each convolutional layer. Generally, the lower-dimensional feature maps at the beginning of the model are called low-level feature maps, while the higher-dimensional feature maps at the end are called high-level feature maps.

[0179] The discriminator described in the embodiments of this specification can be a fully convolutional network structure, including a preset number of convolutional modules. The discriminator is used to flatten the input image and the output feature maps of each convolutional module, multiply them by their respective weight coefficients, and then concatenate them as the final output of the discriminator.

[0180] Figure 4 This is a schematic diagram of the network structure of a discriminator for a generative adversarial network model provided in an embodiment of this specification. Figure 4As shown, the discriminator does not use a traditional classifier, but instead constructs a fully convolutional network. The input is a stitched image of the electron microscope image and the segmentation binary image along the channel direction, and the output is a feature map. The discriminator can consist of six convolutional modules connected in series. The input of the discriminator and the output feature maps of each convolutional module are flattened, multiplied by their respective weight coefficients, and then stitched together to form the final output of the discriminator. For example, in this example, the seven coefficients W1 to W7 used are 1.0, 1.0, 2.0, 2.0, 2.0, 2.0, and 4.0, respectively.

[0181] Each convolutional module consists of a convolutional layer, a batch normalization (BN) layer, and an activation layer. The activation layer uses the LeakyReLU activation function. The input / output channels, kernel size, stride, and padding size of the convolutional layers in the six convolutional modules are as follows: Figure 4 As shown.

[0182] It is understood that the above is merely an example to more clearly illustrate the model structure in the embodiments of this specification. In actual applications, parameters such as the number of network layers, number of channels, convolutional kernel size, padding size, and weight coefficients can be set according to actual needs, without specific limitations here.

[0183] The training set data, after slit preprocessing and data augmentation, is input into the generative adversarial network model to train the generator and discriminator, as follows:

[0184] 1) Optimizer and hyperparameters: The model optimizer for both the generator and discriminator is Adam optimizer. The training is set to 600 rounds. The performance of the model is evaluated using a validation set every two rounds of training. The Dice score is used as the evaluation metric. If the model performance is better than the model obtained from previous training, the model parameters are saved.

[0185] The initial learning rate is set to 1e-4, and the learning rate is updated using a cosine learning rate strategy. The learning rate is set in each round as follows:

[0186]

[0187] Among them, lr max and lr min Let E represent the initial learning rate and the minimum learning rate, respectively. The minimum learning rate is set to 0.001 times the initial learning rate. cur and E max These represent the current training round and the total number of training rounds, respectively.

[0188] 2) Selection of iteration rounds: During the entire iterative training process, the generator and the discriminator compete against each other. In each training round, after training the generator twice, the discriminator is trained once. Finally, a generator that can generate images that are highly similar to the real metal line segmentation results is obtained.

[0189] 3) Loss Function: The loss function design includes the loss functions for the generator and discriminator, as detailed below:

[0190] The generator's loss function: The generator's output image of the metal line segmentation is concatenated with the original electron microscope image used for training and then input into the discriminator. The discriminator's output is required to be highly similar to the output obtained when the ground truth label image is concatenated with the ground truth label image as input. Simultaneously, the generator's output image of the metal line segmentation is required to be highly similar to the ground truth label image. Therefore, the generator's loss consists of three parts. The first part is the L1 loss, calculated using the following formula:

[0191]

[0192] Where g is the output feature map of the discriminator when the metal wire segmentation result image output by the generator is concatenated with the original electron microscope image and used as input to the discriminator. r is the output feature map of the discriminator when the ground truth label image is concatenated with the original electron microscope image and used as input to the discriminator. i,j can represent the i-th row and j-th column of the output feature map; r i，j This can represent the pixel in the i-th row and j-th column of the discriminator's output feature map when the real label image and the original electron microscope image are stitched together as input to the discriminator; g i,j The first part represents the pixel in the i-th row and j-th column of the discriminator's output feature map when the metal wire segmentation result image output by the generator is stitched together with the original electron microscope image and used as input to the discriminator. The second part is the perceptual loss, which involves extracting the intermediate feature maps of the discriminator model and calculating the same L1 loss as the first part. This includes the output feature maps of the 1st, 2nd, 5th, and 6th convolutional modules of the discriminator model, and summing them with weights of 5.0, 1.5, 1.5, and 1.0 respectively as the final perceptual loss. The third part is the SSIM loss, calculated using the following formula:

[0193]

[0194] Where p and t represent the segmentation result image generated by the generator and the real label image, respectively, and μ p and μ t σ represents the mean pixel value of p and t, respectively. p and σ t σ represents the standard deviation of pixel values ​​for p and t, respectively. pt C1 and C2 are the pixel value covariance of p and t, and are preset constants, usually much smaller than 1, to prevent the denominator from being zero.

[0195] The discriminator's loss function is designed to distinguish the segmented image generated by the generator from the real label image, thus creating a game-like competition with the generator. Therefore, the loss function is the inverse of the L1 loss. The calculation of the L1 loss is the same as that for the generator.

[0196] Specifically, in this example, to fully illustrate the beneficial effects of the proposed method, electron microscopy images of integrated circuit chips without real segmentation labels are not used in subsequent steps. Instead, a randomly partitioned test set is used to evaluate the performance of the generator of the generative adversarial model. Similar to the validation process, the Dice score is used to evaluate the generator's performance, which may specifically include the following steps:

[0197] 1) Input the slice images from the test set into the generator of the trained generative adversarial model to obtain the corresponding metal line segmentation result slice images. After stitching them together, obtain the metal line segmentation result image of the original electron microscope image of the test set.

[0198] 2) Post-processing is performed on the metal line segmentation results of the original electron microscope images of the obtained test set to select the final metal line segmentation results. In this example, the thresholds for contour area and roundness are set to 200 and 0.9, respectively, and the threshold for grayscale range is set to 50.

[0199] 3) After obtaining the final metal line segmentation result image, the Dice score is used to evaluate the performance of the generator of the generative adversarial model. The calculation formula is as follows:

[0200]

[0201] Where T is the real label image, P is the final metal line segmentation result image, and area() is the area of ​​the target in the image.

[0202] In this example, the Dice score obtained by using the test set to evaluate the model performance is 0.9751. Figure 5This specification provides an example image of a metal line segmentation result. The first row shows the original electron microscope (EM) image, and the second row shows the metal line segmentation result image generated by the model. Each image represents a segmented image. From left to right, the first three images in the first row represent the original EEM images of different chips. All three images contain both background and metal lines. Specifically, the first and second images are examples of very small intervals between metal lines. The fourth image can represent a cropped original EEM image consisting entirely of metal lines, and the fifth image can represent a cropped original EEM image consisting entirely of non-metallic lines (i.e., background). The images in the second row represent the segmentation result images corresponding to the original images in the first row. In the segmentation result images, the black areas represent the background, and the white areas represent the segmented metal line portions.

[0203] To more clearly illustrate the methods provided in the embodiments of this specification, Figure 6 This is a flowchart illustrating a method for dividing metal lines in an integrated circuit chip, as provided in the embodiments of this specification. Figure 6 As shown, the method may include a model training phase and a metal line segmentation phase using the model, specifically including:

[0204] Step 602: Obtain electron microscope images of integrated circuit chips containing labels as training data;

[0205] Step 604: Crop the training data; use a sliding window to crop the training data to obtain several slice images;

[0206] Step 606: Perform preprocessing operations on the training set images, which may include slit preprocessing and / or data augmentation, to obtain processed training data;

[0207] Step 608: Train the initial generative adversarial network model using the processed training data to obtain the trained generative adversarial network model;

[0208] Step 610: Evaluate the trained generative adversarial network model;

[0209] Step 612: When the evaluation result indicates that the trained generative adversarial network model meets the requirements, it can be understood that the evaluation result of the model is excellent, and a usable trained generative adversarial network model is obtained; when the evaluation result indicates that the trained generative adversarial network model does not meet the requirements, it can be understood that the evaluation result of the model is poor, and the model can continue to be trained.

[0210] Step 614: Obtain an electron microscope image of the integrated circuit chip to be segmented, as the image to be segmented;

[0211] Step 616: Crop the image to be segmented. This can be done using the same cropping method as the training data.

[0212] Step 618: Input the cropped sub-images into the trained generative adversarial network model to obtain the segmentation result images corresponding to each sub-image; stitch the segmentation result images corresponding to each sub-image together to obtain the preliminary metal wire segmentation result image;

[0213] Step 620: Perform post-processing and filtering on the preliminary metal wire segmentation result image;

[0214] Step 622: Output the final metal wire segmentation result image.

[0215] The above methods can achieve at least one of the following beneficial effects:

[0216] 1) The narrow slit preprocessing method solves the problem of target sticking of real labels during the sampling process of network model due to the close distribution of some metal lines in integrated circuit chips.

[0217] 2) The method of segmenting metal lines of integrated circuit chips using generative adversarial networks is more suitable for processing electron microscope images of metal lines of integrated circuit chips compared with previous methods, and effectively improves the segmentation accuracy.

[0218] 3) Training methods for generative adversarial network (GAN) models: Based on electron microscope images of metal lines on integrated circuit chips, this paper presents the necessary hyperparameters and data augmentation designs for model training, such as learning rate adjustment mechanisms and loss function design, as well as the addition of Gaussian noise and rotations within the [40°, 50°] range. These methods enable more accurate model training results and effectively avoid overfitting.

[0219] 4) After the generator of the generative adversarial network outputs the initial result image, there are some areas that the model cannot accurately judge after cropping the electron microscope image of the metal lines of the high-resolution integrated circuit chip. A post-processing target screening method is provided to effectively solve the problem of reduced segmentation accuracy caused by this issue.

[0220] Based on the same idea, embodiments of this specification also provide apparatus corresponding to the above methods. Figure 7 This is a schematic diagram of a device for dividing metal lines in an integrated circuit chip, provided as an embodiment of this specification. Figure 7 As shown, the device may include:

[0221] Image acquisition module 702 is used to acquire electron microscope images of the integrated circuit chip to be processed;

[0222] The preliminary results module 704 is used to obtain a preliminary metal line segmentation result image of the electron microscope image based on a pre-trained generative adversarial network model.

[0223] The metal line recognition module 706 is used to recognize the metal line contours in each contour of the initial segmentation result image and obtain the recognition result.

[0224] The result determination module 708 is used to obtain the final metal line segmentation result image of the electron microscope image based on the metal line contours obtained from the recognition results.

[0225] Based on the same idea, this specification also provides devices corresponding to the above methods in its embodiments.

[0226] Figure 8 This is a schematic diagram of a device for dividing metal lines in an integrated circuit chip, provided as an embodiment of this specification. Figure 8 As shown, device 800 may include:

[0227] At least one processor 810; and,

[0228] Memory 830 communicatively connected to the at least one processor; wherein,

[0229] The memory 830 stores instructions 820 that can be executed by the at least one processor 810, the instructions being executed by the at least one processor 810 to enable the at least one processor 810 to:

[0230] Obtain electron microscope images of the integrated circuit chip to be processed;

[0231] Based on a pre-trained generative adversarial network model, a preliminary metal line segmentation result image is obtained from the electron microscope image;

[0232] For each contour in the initial segmentation result image, the metal line contour in each contour is identified to obtain the identification result;

[0233] Based on the recognition results, the contours of each metal line are obtained, and the final metal line segmentation result image of the electron microscope image is obtained.

[0234] Based on the same approach, embodiments of this specification also provide a computer-readable medium corresponding to the above-described method. The computer-readable medium stores computer-readable instructions that can be executed by a processor to implement the above-described method for dividing metal lines in an integrated circuit chip.

[0235] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on its differences from other embodiments. In particular, for... Figure 8 As the device shown is basically similar to the method embodiment, the description is relatively simple, and relevant parts can be found in the description of the method embodiment.

[0236] For ease of description, the above devices are described separately by function as various units. Of course, in implementing this application, the functions of each unit can be implemented in one or more software and / or hardware.

[0237] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0238] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0239] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0240] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital character versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0241] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0242] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0243] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A method for dividing metal lines in an integrated circuit chip, characterized in that, include: Obtain electron microscope images of the integrated circuit chip to be processed; Based on a pre-trained generative adversarial network model, a preliminary metal line segmentation result image of the electron microscope image is obtained, specifically including: cropping the electron microscope image according to a first preset cropping rule to obtain several sub-images; inputting the several sub-images into the pre-trained generative adversarial network model to obtain a segmentation result sub-image corresponding to each sub-image; and stitching the segmentation result sub-images corresponding to each sub-image together to obtain the preliminary metal line segmentation result image. For each contour of the preliminary metal line segmentation result image, the metal line contours in each contour are identified to obtain the identification result; Based on the recognition results, the contours of each metal line are obtained, and the final metal line segmentation result image of the electron microscope image is obtained.

2. The method according to claim 1, characterized in that, The step of identifying the metal line contours within each contour of the preliminary metal line segmentation result image specifically includes: Extract each contour from the preliminary metal wire segmentation result image; For any contour among the extracted contours, obtain the area of ​​the contour; Determine whether the area of ​​the contour is greater than or equal to a first preset threshold; If the area of ​​the contour is greater than or equal to a first preset threshold, then the contour is determined to be a metal wire contour. or, Obtain the roundness of the contour; Determine whether the roundness of the contour is greater than or equal to a second preset threshold; If the roundness of the contour is greater than or equal to a second preset threshold, then the contour is determined to be a metal wire contour.

3. The method according to claim 2, characterized in that, If the area of ​​the contour is greater than or equal to a first preset threshold, or the roundness of the contour is greater than or equal to a second preset threshold, the method further includes: Determine whether the preliminary metal line segmentation result image contains a straight line passing through the contour; If it exists, the contour is dilated and eroded to obtain a dilated contour image and an eroded contour image. Subtracting the dilated contour image from the eroded contour image yields the contour ring; Obtain the maximum and minimum pixel values ​​corresponding to the contour ring in the electron microscope image; Determine whether the range between the maximum and minimum pixel values ​​is greater than or equal to a preset threshold; Determining that the contour is a metal line contour specifically includes: If the range value is greater than or equal to a preset threshold, then the contour is determined to be a metal line contour.

4. The method according to claim 1, characterized in that, The pre-trained generative adversarial network model is trained using the following method: Obtain training sample images; The training sample images are cropped according to the second preset cropping rule to obtain several training sub-images; The training sub-images are preprocessed to obtain the processed training sub-images; The initial generative adversarial network model is trained using the processed training sub-images to obtain the trained generative adversarial network model.

5. The method according to claim 4, characterized in that, The image preprocessing of the plurality of training sub-images specifically includes: The labeled images of each electron microscope image in the plurality of training sub-images are subjected to narrow slit preprocessing, which expands the spacing of metal lines in each image that are less than or equal to n pixels to m pixels.

6. The method according to claim 5, characterized in that, The narrow slit pretreatment specifically includes: For any training sub-image among the plurality of training sub-images, obtain the metal lines in the training sub-image that are separated by a distance of less than or equal to i pixels; i∈[1,n]; The metal lines in the training sub-image that are spaced less than or equal to i pixels apart are dilated with an m×m kernel to obtain the processed training sub-image.

7. The method according to claim 6, characterized in that, The step of obtaining metal lines in the training sub-image that are spaced apart by a distance of less than or equal to n pixels specifically includes: Establish an auxiliary image, a slit image, and a midline image corresponding to the training sub-image; the auxiliary image is an image obtained by setting the pixels with a value of 0 in the training sub-image to -1; the slit image and the midline image are images with the same size as the training sub-image and all pixel values ​​are 0. The auxiliary image is convolved with the row vector to obtain the first convolution result image; the length of the row vector is i+2, the first and last elements are both 1, and the values ​​of the remaining elements are all -1; the variable i∈[1,n]; Set the pixel value at the position corresponding to the pixel with a pixel value of i+2 in the first convolution result image on the midline image of the interval to 1, and obtain the adjusted first midline image of the interval. The auxiliary image is convolved with the column vector to obtain the second convolution result image; the length of the column vector is i+2, the first and last elements are both 1, and the values ​​of the remaining elements are all -1. The pixel value at the position corresponding to the pixel with pixel value equal to i+2 in the second convolution result image on the adjusted first interval midline image is set to 1 to obtain the adjusted second interval midline image; the second interval midline image is used to represent the midline of the interval of metal lines in the training sub-image that are separated by a distance of less than or equal to i pixels.

8. The method according to claim 7, characterized in that, The step of dilating the metal lines in the training sub-image that are spaced less than or equal to i pixels apart using an m×m kernel to obtain the processed training sub-image specifically includes: The adjusted second interval midline image is dilated with an m×m kernel to obtain the dilated interval midline image corresponding to the training sub-image for variable i. With a step size of i plus 1, we obtain the dilated interval midline image corresponding to each value of i in the range [1, n]. The dilated midline images corresponding to each value are superimposed on the slit image to obtain the final slit image; Set the pixel value at the position corresponding to the pixel with a pixel value greater than 0 in the final slit image in the training sub-image to 0 to obtain the slit preprocessed training sub-image corresponding to the training sub-image.

9. The method according to claim 5, characterized in that, The image preprocessing of the plurality of training sub-images specifically includes: Data augmentation processing is performed on the training sub-images after narrow slit preprocessing; The data augmentation processing includes at least one of the following methods: random scaling; random cropping; random brightness, contrast and saturation adjustment; random addition of Gaussian noise; random horizontal and vertical flipping; and random angle rotation with a rotation angle of [40°, 50°].

10. The method according to claim 4, characterized in that, The initial generative adversarial network model includes a generator and a discriminator; The step of training the initial generative adversarial network model using the processed training sub-images to obtain the trained generative adversarial network model specifically includes: The processed training sub-images are used as input to the initial generative adversarial network model, and iterative training is performed for a preset number of rounds to obtain the trained generative adversarial network model. During iterative training, every two training rounds, the performance of the generative adversarial network model trained in the last round of those two rounds is evaluated using a validation set. If the evaluation results indicate that the performance of the generative adversarial network model trained in the last round of those two rounds is better than the performance of the generative adversarial network models trained in the previous rounds, then the model parameters of the generative adversarial network model trained in the last round are saved. Based on the saved model parameters, subsequent iterative training is performed.

11. The method according to claim 10, characterized in that, During the iterative training process: the generator is trained twice, and the discriminator is trained once.

12. The method according to claim 10, characterized in that, The generator includes an encoder and a decoder; The encoder includes a residual attention module and an ASPP model connected to the residual attention module; The decoder concatenates the first feature map output by the intermediate layer of the residual attention module and the second feature map output by the ASPP module to obtain the concatenated feature map. Decoding is completed based on the stitched feature map.

13. An apparatus for dividing metal lines of an integrated circuit chip, characterized in that, include: The image acquisition module is used to acquire electron microscope images of the integrated circuit chip to be processed; The preliminary results module is used to obtain a preliminary metal line segmentation result image of the electron microscope image based on a pre-trained generative adversarial network model. Specifically, it includes: cropping the electron microscope image according to a first preset cropping rule to obtain several sub-images; inputting the several sub-images into the pre-trained generative adversarial network model to obtain a segmentation result sub-image corresponding to each sub-image; and stitching the segmentation result sub-images corresponding to each sub-image together to obtain the preliminary metal line segmentation result image. The metal line recognition module is used to identify the metal line contours in each contour of the preliminary metal line segmentation result image and obtain the recognition result. The result determination module is used to obtain the final metal line segmentation result image of the electron microscope image based on the metal line contours obtained from the recognition results.

14. A device for dividing metal wires in an integrated circuit chip, characterized in that, include: At least one processor; as well as, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enable the at least one processor to: Obtain electron microscope images of the integrated circuit chip to be processed; Based on a pre-trained generative adversarial network model, a preliminary metal line segmentation result image of the electron microscope image is obtained, specifically including: cropping the electron microscope image according to a first preset cropping rule to obtain several sub-images; inputting the several sub-images into the pre-trained generative adversarial network model to obtain a segmentation result sub-image corresponding to each sub-image; and stitching the segmentation result sub-images corresponding to each sub-image together to obtain the preliminary metal line segmentation result image. For each contour of the preliminary metal line segmentation result image, the metal line contours in each contour are identified to obtain the identification result; Based on the recognition results, the contours of each metal line are obtained, and the final metal line segmentation result image of the electron microscope image is obtained.

Images