Contour extraction method and device based on lithography SEM image, equipment and storage medium

Abstract

The application discloses a profile extraction method and device based on lithography SEM images, equipment and a storage medium. The method comprises the following steps: training an initial semantic segmentation model according to a labeled lithography SEM image, obtaining an intermediate semantic segmentation model; inputting an unlabeled lithography SEM image into the intermediate semantic segmentation model to obtain a segmentation image carrying pseudo-labels; determining available labels from the pseudo-labels, training the intermediate semantic segmentation model according to the segmentation image carrying the available labels and the labeled lithography SEM image to obtain a target semantic segmentation model; inputting a to-be-detected lithography SEM image into the target semantic segmentation model to obtain a target segmentation image; and obtaining a target profile extraction result according to a profile extraction algorithm and the target segmentation image. The application extracts the profile of the lithography SEM image based on the semantic segmentation model of the semi-supervised learning mode, reduces the dependence on the labeled data set, reduces the cost, and improves the efficiency of the image profile extraction.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and particularly relates to a contour extraction method and device based on lithography SEM images, equipment and a storage medium. BACKGROUND

[0002] Contour extraction based on lithography scanning electron microscope (SEM) images is crucial for the evaluation and monitoring of the quality of a patterning process. However, the general lithography SEM image has the problems of poor image quality and complex patterns. The existing method is to extract the contour of the lithography SEM image through a deep learning model. However, the deep learning model needs a large number of labeled lithography image datasets for training, and the labeling of the lithography image needs a large amount of human cost, thereby affecting the efficiency of the image contour extraction for the lithography SEM image. SUMMARY

[0003] The embodiment of the present application provides a contour extraction method and device based on lithography SEM images, equipment and a storage medium, which can effectively improve the contour extraction efficiency for the lithography SEM image.

[0004] In a first aspect, the embodiment of the present application provides a contour extraction method based on lithography SEM images, comprising:

[0005] Obtaining an initial training dataset, wherein the initial training dataset comprises labeled lithography SEM images and unlabeled lithography SEM images;

[0006] Training a preset initial semantic segmentation model according to the labeled lithography SEM images to obtain an intermediate semantic segmentation model;

[0007] Inputting the unlabeled lithography SEM images into the intermediate semantic segmentation model for image segmentation processing to obtain a segmentation image carrying pseudo-labels;

[0008] Determining available labels from the pseudo-labels, and determining the segmentation image carrying the available labels and the labeled lithography SEM images as a target dataset;

[0009] Training the intermediate semantic segmentation model according to the target dataset to obtain a target semantic segmentation model;

[0010] Obtaining a to-be-detected lithography SEM image, inputting the to-be-detected lithography SEM image into the target semantic segmentation model for image segmentation processing to obtain a target segmentation image;

[0011] Extracting the contour of the target segmentation image according to a preset contour extraction algorithm to obtain a target contour extraction result.

[0012] In some embodiments, the target semantic segmentation model comprises an encoding module, a decoding module and a skip connection module, and the inputting the to-be-detected lithography SEM image into the target semantic segmentation model for image segmentation processing to obtain a target segmentation image comprises:

[0013] inputting the to-be-detected lithography SEM image into the encoding module for image feature extraction to obtain a first feature image;

[0014] inputting the first feature image into the decoding module for data up-sampling processing to obtain a second feature image;

[0015] performing data merging processing on the first feature image and the second feature image corresponding to each other through the skip connection module to obtain a third feature image;

[0016] performing convolution processing on the third feature image, performing image classification processing on the third feature image after convolution processing according to a preset softmax activation function, and obtaining the target segmentation image.

[0017] In some embodiments, the encoding module comprises a convolution module and a max-pooling layer connected in sequence, the convolution module comprises a first convolution layer, an activation function layer and a batch normalization layer, and the inputting the to-be-detected lithography SEM image into the encoding module for image feature extraction to obtain a first feature image comprises:

[0018] inputting the to-be-detected lithography SEM image into the first convolution layer for convolution processing to obtain a first intermediate image;

[0019] inputting the first intermediate image into the activation function layer for activation operation to obtain a second intermediate image;

[0020] inputting the second intermediate image into the batch normalization layer for normalization processing to obtain the first feature image.

[0021] In some embodiments, the decoding module comprises an up-sampling layer and a second convolution layer, and the inputting the first feature image into the decoding module for data up-sampling processing to obtain a second feature image comprises:

[0022] inputting the first feature image into the up-sampling layer for up-sampling processing to obtain a third intermediate image after up-sampling;

[0023] inputting the third intermediate image into the second convolution layer for convolution processing to obtain the second feature image.

[0024] In some embodiments, the training of the intermediate semantic segmentation model according to the target data set comprises:

[0025] The convergence state information of the intermediate semantic segmentation model is determined according to a preset loss function, and when the convergence state information does not satisfy a preset model convergence condition, the label-free lithography SEM image is input to the intermediate semantic segmentation model again for image segmentation processing to obtain a segmentation image carrying a new pseudo label;

[0026] A new available label is determined from the new pseudo label, and the segmentation image carrying the new available label and the labeled lithography SEM image are determined as a new target data set;

[0027] The intermediate semantic segmentation model is trained according to the new target data set to obtain a new intermediate semantic segmentation model, and when the new intermediate semantic segmentation model satisfies the model convergence condition, the new intermediate semantic segmentation model is determined as the target semantic segmentation model.

[0028] In some embodiments, before the training of the preset initial semantic segmentation model according to the labeled lithography SEM image, the method further comprises:

[0029] The labeled lithography SEM image is image preprocessed according to a preset image preprocessing rule to obtain a preprocessed labeled lithography SEM image.

[0030] In a second aspect, the embodiments of the present application provide a contour extraction device based on a lithography SEM image, comprising:

[0031] A data acquisition module is configured to acquire an initial training data set, wherein the initial training data set comprises a labeled lithography SEM image and a label-free lithography SEM image;

[0032] A first data processing module is configured to train a preset initial semantic segmentation model according to the labeled lithography SEM image to obtain an intermediate semantic segmentation model;

[0033] A second data processing module is configured to input the label-free lithography SEM image to the intermediate semantic segmentation model for image segmentation processing to obtain a segmentation image carrying a pseudo label;

[0034] A third data processing module is configured to determine an available label from the pseudo label, and determine a segmentation image carrying the available label and the labeled lithography SEM image as a target data set;

[0035] a fourth data processing module, configured to train the intermediate semantic segmentation model according to the target data set to obtain a target semantic segmentation model;

[0036] a fifth data processing module, configured to acquire a to-be-detected lithography SEM image, input the to-be-detected lithography SEM image into the target semantic segmentation model for image segmentation processing, and obtain a target segmentation image;

[0037] a target contour extraction result determination module, configured to perform contour extraction on the target segmentation image according to a preset contour extraction algorithm to obtain a target contour extraction result.

[0038] In a third aspect, an electronic device is provided, which includes a memory, a processor, and a computer program stored in the memory and capable of running on the processor, and the processor implements the contour extraction method based on a lithography SEM image as described in the first aspect when executing the computer program.

[0039] In a fourth aspect, a computer readable storage medium is provided, which stores computer executable instructions for executing the contour extraction method based on a lithography SEM image as described in the first aspect.

[0040] The contour extraction method, device, electronic device, and storage medium based on a lithography SEM image provided in the embodiments of the present application include the following steps: acquiring an initial training data set, the initial training data set including labeled lithography SEM images and unlabeled lithography SEM images; training a preset initial semantic segmentation model according to the labeled lithography SEM images to obtain an intermediate semantic segmentation model; inputting the unlabeled lithography SEM images into the intermediate semantic segmentation model for image segmentation processing to obtain segmentation images carrying pseudo labels; determining usable labels from the pseudo labels, and determining segmentation images carrying the usable labels and the labeled lithography SEM images as a target data set; training the intermediate semantic segmentation model according to the target data set to obtain a target semantic segmentation model; acquiring a to-be-detected lithography SEM image, inputting the to-be-detected lithography SEM image into the target semantic segmentation model for image segmentation processing to obtain a target segmentation image; and performing contour extraction on the target segmentation image according to a preset contour extraction algorithm to obtain a target contour extraction result. The embodiments of the present application perform image contour extraction on a to-be-detected lithography SEM image based on a semantic segmentation model of a semi-supervised learning mode, which can effectively reduce the dependence on a labeled data set, reduce costs, and improve the efficiency of image contour extraction. BRIEF DESCRIPTION OF DRAWINGS

[0041] Figure 1is a step flow chart of a profile extraction method based on lithography SEM images provided by an embodiment of the present application;

[0042] Figure 2 is a step flow chart of obtaining a target segmentation image provided by another embodiment of the present application;

[0043] Figure 3 is a step flow chart of obtaining a first feature image provided by another embodiment of the present application;

[0044] Figure 4 is a step flow chart of obtaining a second feature image provided by another embodiment of the present application;

[0045] Figure 5 is a step flow chart of training a target semantic segmentation model provided by another embodiment of the present application;

[0046] Figure 6 is a step flow chart of image pre-processing on a labeled lithography SEM image provided by another embodiment of the present application;

[0047] Figure 7 is a module schematic diagram of a profile extraction device based on lithography SEM images provided by another embodiment of the present application;

[0048] Figure 8 is a hardware schematic diagram of an electronic device provided by another embodiment of the present application. DETAILED DESCRIPTION

[0049] In order to make the purposes, technical solutions and advantages of the present application clearer, the present application is further described in detail below in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, and are not used to limit the present application.

[0050] It can be understood that, although the functional modules are divided in the device schematic diagram, and the logical order is shown in the flow chart, in some cases, the steps shown or described can be executed in a different order from the module division in the device or the order in the flow chart. The terms "first", "second", etc. in the specification, claims or above-described drawings are used to distinguish similar objects, and do not necessarily describe a specific order or sequence.

[0051] The embodiment of the application provides a profile extraction method, device and equipment based on a lithography SEM image and a storage medium. The method comprises the following steps: obtaining an initial training data set, wherein the initial training data set comprises a labeled lithography SEM image and an unlabeled lithography SEM image; training a preset initial semantic segmentation model according to the labeled lithography SEM image to obtain an intermediate semantic segmentation model; inputting the unlabeled lithography SEM image into the intermediate semantic segmentation model for image segmentation processing to obtain a segmentation image carrying pseudo-labels; determining available labels from the pseudo-labels, and determining the segmentation image carrying the available labels and the labeled lithography SEM image as a target data set; training the intermediate semantic segmentation model according to the target data set to obtain a target semantic segmentation model; obtaining a to-be-detected lithography SEM image, inputting the to-be-detected lithography SEM image into the target semantic segmentation model for image segmentation processing to obtain a target segmentation image; and performing profile extraction on the target segmentation image according to a preset profile extraction algorithm to obtain a target profile extraction result. The embodiment of the application performs image profile extraction on the to-be-detected lithography SEM image based on the semantic segmentation model of the semi-supervised learning mode, can effectively reduce the dependence on the labeled data set, and can reduce the cost while improving the efficiency of image profile extraction.

[0052] The embodiment of the application will be further described below with reference to the drawings.

[0053] As shown in the figure, Figure 1 Figure 1 is a step flowchart of a profile extraction method based on a lithography SEM image provided by an embodiment of the application. The embodiment of the application provides a profile extraction method based on a lithography SEM image, which comprises but is not limited to the following steps:

[0054] In step S110, an initial training data set is obtained, and the initial training data set comprises a labeled lithography SEM image and an unlabeled lithography SEM image.

[0055] In step S120, an initial semantic segmentation model is trained according to the labeled lithography SEM image to obtain an intermediate semantic segmentation model.

[0056] In step S130, the unlabeled lithography SEM image is input into the intermediate semantic segmentation model for image segmentation processing to obtain a segmentation image carrying pseudo-labels.

[0057] In step S140, available labels are determined from the pseudo-labels, and the segmentation image carrying the available labels and the labeled lithography SEM image are determined as a target data set.

[0058] In step S150, the intermediate semantic segmentation model is trained according to the target data set to obtain a target semantic segmentation model. ​

[0059] Step S160, obtaining a lithography SEM image to be detected, inputting the lithography SEM image to be detected into the target semantic segmentation model for image segmentation processing to obtain a target segmentation image;

[0060] Step S170, performing contour extraction on the target segmentation image according to a preset contour extraction algorithm to obtain a target contour extraction result.

[0061] It can be understood that an initial training data set is obtained, the initial training data set includes labeled lithography SEM images and unlabeled lithography SEM images, and the embodiments of the present application do not limit the specific number of the labeled lithography SEM images and the unlabeled lithography SEM images; the preset initial semantic segmentation model is trained according to the labeled lithography SEM images to obtain an intermediate semantic segmentation model, the unlabeled lithography SEM images are predicted using the intermediate semantic segmentation model after optimization training to obtain a segmentation image carrying pseudo-labels; the pseudo-labels are screened to determine high-confidence and relatively accurate available labels from the pseudo-labels, the segmentation image carrying the available labels and the labeled lithography SEM images are determined as a target data set, the intermediate semantic segmentation model is trained according to the target data set to realize re-optimization of the model, a target semantic segmentation model is obtained, the lithography SEM image to be detected is obtained, the lithography SEM image to be detected is input into the target semantic segmentation model for image segmentation processing to obtain a target segmentation image, and the target segmentation image is subjected to contour extraction according to a preset contour extraction algorithm to obtain a target contour extraction result. The semantic segmentation model based on the semi-supervised learning mode in the embodiments of the present application can accurately extract the contours of lithography images with poor picture quality and complex patterns, and at the same time, solves the problem that a neural network model needs a large amount of data and is difficult to label when processing lithography SEM images.

[0062] It should be noted that the embodiments of the present application do not limit the specific way of obtaining the initial training data set, which can be by collecting original lithography images generated by SEM, sampling the original lithography images according to a preset image size, the preset image size can be a pixel size of 512*512, which is not limited herein, obtaining an aggregated image set D, determining the types of lithography patterns in the image set D through a preset image classification rule, and selecting typical images corresponding to each pattern type, labeling the contours of the typical images and aggregating to form a labeled lithography SEM image set D1, dividing the images in the labeled lithography SEM image set D1 into a training set T1 and a verification set V1 according to a preset proportion, and aggregating the remaining images in the image set D except the typical images to form an unlabeled lithography SEM image set D2.

[0063] It should be noted that the initial semantic segmentation model of the embodiment of the present application is a U-Net model, and can also be a mask-rcnn model, which is not limited here.

[0064] It should be noted that the distribution condition of the target label in the labeled lithography SEM image can be a Bernoulli distribution, and the distribution expression can be:

[0065]

[0066] wherein f θ (x n ) is the result output by the pixel n after the softmax function calculation in the output layer of the target semantic segmentation model, and θ is the parameter vector of the U-Net model.

[0067] It should be noted that the embodiment of the present application does not limit the specific contour extraction algorithm, which can be a Canny edge detection algorithm, and can also be a whole nested edge detection HED algorithm.

[0068] In addition, in an embodiment, the target semantic segmentation model includes an encoding module, a decoding module and a skip connection module, and step S160 in the embodiment shown in Figure 2 , Figure 1 includes but is not limited to the following steps:

[0069] Step S210, input the to-be-detected lithography SEM image into the encoding module for image feature extraction to obtain a first feature image;

[0070] Step S220, input the first feature image into the decoding module for data up-sampling processing to obtain a second feature image;

[0071] Step S230, data merging processing of the one-to-one corresponding first feature image and second feature image is performed through the skip connection module to obtain a third feature map;

[0072] Step S240, performing convolution processing on the third feature image, and performing image classification processing on the third feature image after the convolution processing according to a preset softmax activation function to obtain a target segmentation image.

[0073] In addition, in an embodiment, the encoding module includes a convolution module and a maximum pooling layer connected in sequence, the convolution module includes a first convolution layer, an activation function layer and a batch normalization layer, and step S210 in the embodiment shown in Figure 3 , Figure 2 includes but is not limited to the following steps:

[0074] Step S310, input the lithography SEM image to be detected into the first convolution layer for convolution processing to obtain a first intermediate image.

[0075] Step S320, input the first intermediate image into the activation function layer for activation operation to obtain a second intermediate image.

[0076] Step S330, input the second intermediate image into the batch normalization layer for normalization processing to obtain a first feature image.

[0077] In addition, in an embodiment, the decoding module includes an up-sampling layer and a second convolution layer, and the specific steps of obtaining the second feature image include but are not limited to the following steps: Figure 4 , Figure 2

[0078] Step S410, input the first feature image into the up-sampling layer for up-sampling processing to obtain a third intermediate image after up-sampling.

[0079] Step S420, input the third intermediate image into the second convolution layer for convolution processing to obtain a second feature image.

[0080] It should be noted that, since the encoding module includes the convolution module and the max-pooling layer connected in sequence, the convolution module includes the first convolution layer, the activation function layer and the batch normalization layer, and the specific steps of inputting the lithography SEM image to be detected into the encoding module for image feature extraction to obtain the first feature image can be: inputting the lithography SEM image to be detected into the first convolution layer for convolution processing to obtain a first intermediate image; inputting the first intermediate image into the activation function layer for activation operation to obtain a second intermediate image; and inputting the second intermediate image into the batch normalization layer for normalization processing to obtain a first feature image. The specific number of the encoding module is not limited in the embodiment of the application, and the model can include four encoding modules. The size of the first convolution kernel is not limited in the embodiment, and the first convolution layer can be a convolution layer with a convolution kernel size of 3*3. Meanwhile, the specific structure of the activation function layer is not limited, and the activation function layer can be a RELU activation function layer.

[0081] It should be noted that, since the decoding module includes the up-sampling layer and the second convolution layer, the specific steps of obtaining the second feature image can be: inputting the first feature image into the up-sampling layer for up-sampling processing to obtain a third intermediate image after up-sampling, and inputting the third intermediate image into the second convolution layer for convolution processing to obtain a second feature image. The specific number of the encoding module is not limited in the embodiment of the application, and the model can include four encoding modules. The size of the up-sampling layer in the embodiment can be 2*2, and the size of the convolution kernel of the second convolution layer can be 3*3, which is not limited herein.

[0082] ​It can be understood that after the first feature image output by the encoding module and the second feature image output by the decoding module are obtained, the first feature image and the second feature image corresponding to each other are data-merged by the skip connection module to obtain a third feature image, and the third feature image is subjected to convolution processing (which can be realized by a convolution layer with a convolution kernel size of 1*1). Since the lithography SEM image in the embodiment only distinguishes between foreground and background, it is essentially a binary classification task. The third feature image after convolution processing can be subjected to image classification processing according to a preset softmax activation function to obtain a target segmentation image, thereby providing an effective data basis for obtaining a target contour extraction result.

[0083] In addition, with reference to Figure 5 In an embodiment, Figure 1 The step S150 in the illustrated embodiment includes but is not limited to the following steps:

[0084] Step S510, determining the convergence state information of the intermediate semantic segmentation model according to a preset loss function;

[0085] Step S520, when the convergence state information does not satisfy the preset model convergence condition, re-inputting the unlabeled lithography SEM image into the intermediate semantic segmentation model for image segmentation processing to obtain a segmentation image carrying a new pseudo label;

[0086] Step S530, determining a new available label from the new pseudo label, and determining the segmentation image carrying the new available label and the labeled lithography SEM image as a new target data set;

[0087] Step S540, training the intermediate semantic segmentation model according to the new target data set to obtain a new intermediate semantic segmentation model, and determining the new intermediate semantic segmentation model as the target semantic segmentation model when the new intermediate semantic segmentation model satisfies the model convergence condition.

[0088] It should be noted that the target data set can be represented as D1={x n ,y n} N Considering that the labels carried in each image in the target data set are independent of each other, the expression of the loss function can be determined based on the target data set as follows:

[0089]

[0090] It should be noted that the embodiments of the present application also minimize the value of the loss function by optimizing the parameter θ, so as to realize the convergence of the intermediate semantic segmentation model. The parameter θ optimization method can be: using the random gradient descent method with a fixed learning rate γ = 0.01 to realize, and the optimized parameter θ can be: the sample number selected in the single training of the intermediate semantic segmentation model is set to 2, the epoch number is set to 50, and the maximum iteration number is set to 5. Based on the optimized parameter θ, the intermediate semantic segmentation model is iteratively trained until the convergence state information meets the preset model convergence condition, so as to obtain the target semantic segmentation model.

[0091] It should be noted that the pseudo label can be obtained according to the following formula:

[0092] y m *=f θ (x m );

[0093] The expression of the unfiltered pseudo label data set can be as follows:

[0094] P={x m ,y m *}M,m=1;

[0095] Wherein, x m is a segmentation image carrying a pseudo label, y m * is a pseudo label. It can be understood that determining the available label with high confidence from the pseudo label can reduce the possibility of learning the wrong label by the neural network, and thus improve the accuracy of the contour extraction. The embodiments of the present application do not limit the specific way of determining the available label from the pseudo label. The pseudo label can be filtered according to a preset threshold λt∈[0.5,1). For example, the condition for the availability of the pseudo label is set as foreground y m *>λ t , and background y m *<1-λ t In the case where the convergence state information does not meet the preset model convergence condition, the unlabeled lithography SEM image is input to the intermediate semantic segmentation model for image segmentation processing, to obtain a segmentation image carrying a new pseudo label. The new available label needs to be determined from the new pseudo label. In this process, λ t can control the proportion of the segmentation image carrying the pseudo label selected as the target data set for the next round of training of the intermediate semantic segmentation model. The threshold λ t decreases with the increase of time t, which means that as the number of model training increases, the accuracy of the model increases, and more pseudo labels can be considered as labels with high confidence. In this example, the value of λ t increases by 0.1 in each iteration, and the expression of the available label data set with high confidence after filtering is as follows:

[0096] P1={x m ,y m}M1,m1=1;

[0097] Further, the segmented image carrying the new available label, i.e. the available label data set P1 and the labeled lithography SEM image are merged to obtain a new target data set, and the intermediate semantic segmentation model is trained based on the new target data set to obtain a new intermediate semantic segmentation model. When the new intermediate semantic segmentation model meets the model convergence condition, the new intermediate semantic segmentation model is determined as the target semantic segmentation model, thereby providing an effective data basis for obtaining an accurate target contour extraction result.

[0098] In addition, with reference to Figure 6 , in an embodiment, before step S120 in the embodiment shown in the figure is performed, the profile extraction method based on the lithography SEM image provided by the embodiment of the present application further includes but is not limited to the following steps: Figure 1

[0099] Step S610: performing image preprocessing on the labeled lithography SEM image according to a preset image preprocessing rule to obtain a preprocessed labeled lithography SEM image.

[0100] It can be understood that the main purpose of performing image preprocessing on the to-be-detected image is to eliminate irrelevant information in the labeled lithography SEM image, restore useful real information, enhance the detectability of relevant information and maximize data simplification, thereby improving the reliability of subsequent application of the labeled lithography SEM image.

[0101] It should be noted that the embodiment of the present application does not limit the specific method of performing image preprocessing on the labeled lithography SEM image. The labeled lithography SEM image can be subjected to a mirror operation to increase the mirror edge of the labeled lithography SEM image, so as to retain sufficient information at the picture boundary.

[0102] In addition, with reference to Figure 7 , Figure 7 is a module schematic diagram of the profile extraction device based on the lithography SEM image provided by another embodiment of the present application. In an embodiment, the embodiment of the present application provides a profile extraction device 700 based on the lithography SEM image, which includes:

[0103] The data acquisition module 710 is configured to acquire an initial training data set, and the initial training data set includes a labeled lithography SEM image and an unlabeled lithography SEM image.

[0104] ​The first data processing module 720 is used to train a preset initial semantic segmentation model based on labeled photolithography SEM images to obtain an intermediate semantic segmentation model.

[0105] The second data processing module 730 is used to input the unlabeled photolithography SEM image into the intermediate semantic segmentation model for image segmentation processing to obtain a segmented image carrying pseudo-labels.

[0106] The third data processing module 740 is used to determine available labels from pseudo-labels and to identify the segmented images carrying available labels and the labeled lithographic SEM images as the target dataset.

[0107] The fourth data processing module 750 is used to train the intermediate semantic segmentation model based on the target dataset to obtain the target semantic segmentation model;

[0108] The fifth data processing module 760 is used to acquire the SEM image of the lithography to be inspected, input the SEM image of the lithography to be inspected into the target semantic segmentation model for image segmentation processing, and obtain the target segmented image;

[0109] The target contour extraction result determination module 770 is used to extract the contour of the target segmentation image according to a preset contour extraction algorithm to obtain the target contour extraction result.

[0110] The specific implementation of the contour extraction device based on photolithography SEM images is basically the same as the specific implementation of the contour extraction method based on photolithography SEM images described above, and will not be repeated here.

[0111] Additionally, refer to Figure 8 , Figure 8 This is a hardware structure diagram of an electronic device provided in another embodiment of this application. Another embodiment of this application also provides an electronic device 800, which includes: a memory 810, a processor 820, and a computer program stored in the memory 810 and executable on the processor 820.

[0112] The processor 820 and memory 810 can be connected via a bus or other means.

[0113] The non-transient software program and instructions required to implement the contour extraction method based on photolithography SEM images in the above embodiments are stored in memory 810. When executed by processor 820, the contour extraction method based on photolithography SEM images in the above embodiments is executed, for example, the method described above is executed. Figure 1 Method steps S110 to S170, Figure 2 Method steps S210 to S240, Figure 3 Method steps S310 to S330, Figure 4the method steps S410-S420 in the method 400, Figure 5 the method steps S510-S540 in the method 500, and Figure 6 the method step S610 in the method 600.

[0114] The device embodiments described above are merely illustrative, wherein the units described as separate components can or can not be physically separate, i.e., can be located in one place, or can be distributed to multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0115] In addition, an embodiment of the present application also provides a computer readable storage medium, which stores computer executable instructions, the computer executable instructions are executed by a processor or a controller, for example, executed by a processor in the above-mentioned electronic device 800 embodiment, so that the above-mentioned processor executes the profile extraction method based on the lithography SEM image in the above-mentioned embodiment, for example, executes the above-mentioned Figure 1 the method steps S110-S170 in the method 100, Figure 2 the method steps S210-S240 in the method 200, Figure 3 the method steps S310-S330 in the method 300, Figure 4 the method steps S410-S420 in the method 400, Figure 5 the method steps S510-S540 in the method 500, and Figure 6The method steps S610 in the above disclosed methods can be implemented as software, firmware, hardware, and appropriate combinations thereof. Some or all of the physical components can be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or as hardware, or as an integrated circuit, such as an application specific integrated circuit. Such software can be distributed on computer readable media, which can include computer storage media (or non-transitory media) and communication media (or transitory media). As is well known to those of ordinary skill in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by a computer. Further, as is well known to those of ordinary skill in the art, communication media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media.

Claims

1. A method of profile extraction based on lithography SEM images, characterized by, The method comprises the following steps: acquiring an initial training data set comprising labeled lithography SEM images and unlabeled lithography SEM images; training a preset initial semantic segmentation model according to the labeled lithography SEM images to obtain an intermediate semantic segmentation model; inputting the unlabeled lithography SEM images into the intermediate semantic segmentation model for image segmentation processing to obtain a segmentation image carrying pseudo-labels; determining available labels from the pseudo-labels, and determining the segmentation image carrying the available labels and the labeled lithography SEM images as a target data set; training the intermediate semantic segmentation model according to the target data set to obtain a target semantic segmentation model; acquiring a to-be-detected lithography SEM image, inputting the to-be-detected lithography SEM image into the target semantic segmentation model for image segmentation processing to obtain a target segmentation image; performing contour extraction on the target segmentation image according to a preset contour extraction algorithm to obtain a target contour extraction result; the target semantic segmentation model comprises an encoding module, a decoding module and a skip connection module, and the inputting of the to-be-detected lithography SEM image into the target semantic segmentation model for image segmentation processing to obtain a target segmentation image comprises: inputting the to-be-detected lithography SEM image into the encoding module for image feature extraction to obtain a first feature image; inputting the first feature image into the decoding module for data up-sampling processing to obtain a second feature image; performing data merging processing on the first feature image and the second feature image corresponding to each other through the skip connection module to obtain a third feature map; performing convolution processing on the third feature map, performing image classification processing on the third feature map after convolution processing according to a preset softmax activation function to obtain the target segmentation image.

2. The profile extraction method based on lithography SEM images according to claim 1, wherein, the encoding module comprises a convolution module and a max-pooling layer connected in sequence, the convolution module comprises a first convolution layer, an activation function layer and a batch normalization layer, and the inputting of the to-be-detected lithography SEM image into the encoding module for image feature extraction to obtain a first feature image comprises: inputting the to-be-detected lithography SEM image into the first convolution layer for convolution processing to obtain a first intermediate image; inputting the first intermediate image into the activation function layer for activation operation to obtain a second intermediate image; inputting the second intermediate image into the batch normalization layer for normalization processing to obtain the first feature image.

3. The profile extraction method based on lithography SEM images according to claim 1, wherein, the decoding module comprises an up-sampling layer and a second convolution layer, and the inputting of the first feature image into the decoding module for data up-sampling processing to obtain a second feature image comprises: inputting the first feature image into the up-sampling layer for up-sampling processing to obtain a third intermediate image after up-sampling; inputting the third intermediate image into the second convolution layer for convolution processing to obtain the second feature image.

4. The profile extraction method based on lithography SEM images according to claim 1, wherein, the training of the intermediate semantic segmentation model according to the target data set to obtain a target semantic segmentation model comprises: determine the convergence state information of the intermediate semantic segmentation model according to a preset loss function; when the convergence state information does not satisfy a preset model convergence condition, re-input the unlabeled lithography SEM image into the intermediate semantic segmentation model for image segmentation processing to obtain a segmentation image carrying a new pseudo label; determine a new available label from the new pseudo label, and determine the segmentation image carrying the new available label and the labeled lithography SEM image as a new target data set; train the intermediate semantic segmentation model according to the new target data set to obtain a new intermediate semantic segmentation model, and when the new intermediate semantic segmentation model satisfies the model convergence condition, determine the new intermediate semantic segmentation model as the target semantic segmentation model.

5. The profile extraction method based on lithography SEM images according to claim 1, wherein, Before the training of the preset initial semantic segmentation model according to the labeled lithography SEM image, the method further comprises: perform image preprocessing on the labeled lithography SEM image according to a preset image preprocessing rule to obtain a preprocessed labeled lithography SEM image.

6. The lithography SEM image-based profile extraction method according to claim 1, wherein, The initial semantic segmentation model is a U-Net model.

7. A profile extraction apparatus based on lithography SEM images, characterized by, Comprise: a data acquisition module configured to acquire an initial training data set, the initial training data set comprising a labeled lithography SEM image and an unlabeled lithography SEM image; a first data processing module configured to train a preset initial semantic segmentation model according to the labeled lithography SEM image to obtain an intermediate semantic segmentation model; a second data processing module configured to input the unlabeled lithography SEM image into the intermediate semantic segmentation model for image segmentation processing to obtain a segmentation image carrying a pseudo label; a third data processing module configured to determine an available label from the pseudo label, and determine the segmentation image carrying the available label and the labeled lithography SEM image as a target data set; a fourth data processing module configured to train the intermediate semantic segmentation model according to the target data set to obtain a target semantic segmentation model; a fifth data processing module configured to acquire a to-be-detected lithography SEM image, input the to-be-detected lithography SEM image into the target semantic segmentation model for image segmentation processing to obtain a target segmentation image; a target contour extraction result determination module configured to perform contour extraction on the target segmentation image according to a preset contour extraction algorithm to obtain a target contour extraction result; The target semantic segmentation model comprises an encoding module, a decoding module and a skip connection module, and the input of the to-be-detected lithography SEM image into the target semantic segmentation model for image segmentation processing to obtain a target segmentation image comprises: input the to-be-detected lithography SEM image into the encoding module for image feature extraction to obtain a first feature image; input the first feature image into the decoding module for data up-sampling processing to obtain a second feature image; perform data merging processing on the first feature image and the second feature image corresponding to each other through the skip connection module to obtain a third feature image; The third feature image is subjected to convolution processing, and the third feature image after the convolution processing is subjected to image classification processing according to a preset softmax activation function, so as to obtain the target segmentation image.

8. An electronic device comprising: The memory, the processor and the computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement the profile extraction method based on lithography SEM images according to any one of claims 1 to 6. 9.A computer readable storage medium storing computer executable instructions for performing the profile extraction method based on lithography SEM images according to any one of claims 1 to 6.

Images